Docker: работа с контейнерами
Mar 20, 2017 13:54 · 525 words · 3 minute read docker
Ранее мы рассматривали процесс установки Docker в операционной системе Ubuntu 16.04 и для проверки работоспособности запускали тестовый контейнер hello-world — это был очень простой пример контейнера, который запускался, выводил сообщение на экран и завершал работу.
Как правило, контейнеры выполняют более полезные действия, ведь они очень похожи на виртуальные машины (но менее требовательны к ресурсам). Давайте разберемся с запуском контейнеров и рассмотрим полезные команды для работы с ними!
Проверить сколько места на жестком диске занимает Docker:
Очистить неиспользуемые данные (будьте внимательны с использованием данной команды):
Для поиска образов в реестрах используется команда search , например:
Загрузить образ можно с помощью команды pull :
Просмотреть список уже загруженных образов можно так:
Для удаления образа используется команда rmi с указанием имени и версии образа:
Для удаления всех неиспользуемых образов введите:
Для запуска контейнера и выполнения внутри него команды используется:
Примечание. В данном примере для запуска контейнера используется образ itscaro/debian-ssh.
Подключиться к уже запущенному контейнеру и запустить shell :
Список запущенных контейнеров можно увидеть так:
Примечание. Следует обратить внимание на первую колонку ( CONTAINER ID ), в которой указаны идентификаторы контейнеров — они часто используются для управления Docker-контейнерами.
Список запущенных процессов внутри контейнера смотрим так:
Просмотреть логи контейнера можно так:
или использовать ключ -f (аналог tail -f в linux):
Для остановки контейнера используется команда stop , например:
Для остановки всех запущенных контейнеров используйте такую команду:
Удалить остановленный контейнер можно так:
Удалить все остановленные контейнеры можно командой:
Просмотреть историю образа, который использует контейнер:
Просмотреть изменения файловой системы контейнера по сравнению с оригинальным образом:
Выполнить резервное копирование каталога с данными из контейнера в хост-машину:
Восстановить данные из резервной копии на хост-машине внутрь контейнера:
Вывод имени хранилища ( volume ), которое использует контейнер:
Вывод имени хранилища ( volume ), которое использует контейнер и точек монтирования:
Получить IP-адрес контейнера:
Еще больше полезных команд и примеров работы с docker-контейнерами можно найти на StackOverflow или в официальной документации Docker.
Работаем с логами контейнеров Docker
Перевод: 
Reviewing Docker Logs
Docker
Многие знают, что есть команда docker logs. Чуть меньшее количество людей знает, что этот функционал можно расширить для отслеживания новых сообщений в контейнере (примерно как работает tail -f на обычных файлах), и уж совсем немногие знают и помнят, что можно указывать временные метки для просмотра логов контейнера только с определённого промежутка времени.
Show Docker Container Logs
Самая простая команда для просмотра логов контейнера:
Так можно посмотреть все доступные логи контейнера mycontainer1, самые свежие сообещния будут в конце.
Отслеживание новых сообщений в логах Docker
Чтобы следить за появлением новых сообщений в контейнере, нужно указать опцию -f (от английского follow — следовать):
Сначала результат команды будет такой же, как и в предыдущем примере. Но во-первых, команда не завершится и вы не получите приглашения командной строки (это будет ждущий режим). А во-вторых, скоро должны появиться новые логи из контейнера.
Просмотр логов в промежутке времени
А вот это по-настоящему полезная весч. Можно получить только те сообщения в логах, которые произошли между указанными точками времени.
Для доступа к логам определённого рвемени, используйте параметры –since (от даты) и –until (до даты) для команды docker logs.
ЗАМЕЧУ: можно и не даты указвыать, а даты и время — но для моих нужд и дат хватает.
Вот пример просмотра логов контейнера для Java приложения. Как видите, благодаря моим параметрам мы получили только сообщения между 30м марта и 1м апреля:
Complete Beginner's Guide to Docker Logging
Explore and learn some interesting ways to make log management easier on Docker.
Knowledge on logging with Docker can be very helpful for day-to-day system administration activities, be it troubleshooting or basic monitoring. Without tracking logs on Docker, mitigating issues would be a lot more difficult when investigating anomalies.
This write-up explores some important information on that area to make it easier to understand how to manage Docker associated log files on your Linux system.
Let me start by exploring the most basic parts and gradually go deeper into some specifics.
How to view Docker logs
You can use the docker logs command to fetch whatever is going on with the service inside a running container.
The syntax is simple:
You probably already know that you can use the docker ps command to view the running containers’ name and ID.
Let’s take a real-world example. I have deployed Nextcloud with Docker. The container is named nextcloud, unsurprisingly.
Now, to view the logs of the container named nextcloud , use:
Here’s a truncated version of the output of the above command:
Too much of logs? The good thing is that there are various options that make viewing docker logs a lot easier, just like logs in Linux.
Tail Docker logs to view only certain number of lines
Docker logs command has —tail attribute that can be used in a fashion similar to the tail command.
Which means, you can use it to display only a certain number of lines of Docker logs from the end.
For example, to view the last 50 lines of a container, you can use:
You may use the above command in this fashion as well
Viewing Docker logs in real-time on a live container
You can totally view the container logs in real time. To “follow” the logs, use the —follow or the -f attribute.
When you run docker logs with follow option, you’ll notice that the new log lines will be reported from the container as time progresses.
To exit, you can simply use the Ctrl+C key combination and drop back to the terminal prompt.
Most of the options in the docker logs sub-command can be combined to produce a desired result.
For example, docker logs -f will flood the screen with all the logs from the beginning. You wouldn’t want that. What you can do is to combine both tail and follow options like this:
The above command will show the last 20 lines of the logs and then follow the logs in real time.
View timestamp in Docker logs
If you also want to see the timestamp of the log entries, you can use the -t option.
This can easily be combined with other options like —follow and —tail .
The command below will show the last 10 lines of the log file of a given container with timestamps attached to each of them
Viewing Docker logs in a specified time period
When you are investigating an issue, time can be a critical factor, in which case the —since and —until flags could prove very helpful.
For example, if you are investigating an issue that occurred in the past 24 hours, the following command will show the logged contents of whatever happened only during that time:
1440m in the above command denotes 24×60 minutes. Adding the -t or the —timestamps flag ensures the reported lines get labeled with a timestamp for you to comprehend incidents or errors even more easily. I’d suggest adding it when inquiring any container.
Similarly, if you require the log details after the first 24 hours of initiating deployment, the equivalent command would be:
Other than specifying time in minutes, the above two flags can also be used with a specific timestamp that -t generates. The format is like 2021-07-28T06:20:00 .
The above command will show the logs from 28th July 2021.
Accessing Docker logs from within the container
In some interesting use-cases, you may want to go for a hybrid approach, where you access the application specific logs from within the containers.
To do that, you enter the Docker container:
And then you can use the regular Linux tools to get application related logs. One approach would be to get the process ID of the desired service using the ps command:
Get the details on this process:
Docker system service logs
If you wish to see the logs of the Docker service itself on the host, you can use journalctl if you are on Ubuntu 16.04 or later:
Linux Handbook Abhishek Prakash
Where are Docker logs stored?
You should find the docker logs in the /var/lib/docker/containers directory on the host system. This directory contains the log files related to all containers in their individual directories. You can identify the directories with the container ID.
I used the docker ps to find that 70f19fde9076 is the container ID of the Nextcloud container.
If I look into the directory where docker logs are stored:
There are currently four containers running, and you can see that the third one matches the one we need to look into (it begins with 70f19fde9076 ).
If you check the contents of this directory, you can see that our log file sits right there!
Therefore, the corresponding log file is 70f19fde907672b9a6e5ff3b7db0c9ecbcb68d419712cb04d03d77694cd2ca4e-json.log which is the same file that you were reading when I ran the docker logs command in the beginning.
Let me quickly verify that:
The log contents are the same, but since I’m reading the log file directly from the host, this has a lot of jargon compared to the docker logs command used earlier.
Enabling Log Rotation for Docker (JSON)
By default, for JSON file based logging, log rotation on Docker is disabled.
This may pose a problem with the disk space if the log files grows bigger in size. Our Docker based Ghost instance saw its log files going up to 20 GB in size. You would want to avoid such situation by enabling log rotation.
To enable log rotation for Docker, edit the /etc/docker/daemon.json file:
Append the following lines and save the file:
Restart Docker daemon:
The above setup is of course relevant to a generic Docker installation, which has JSON logging set as default. So do note that every other logging mechanism (listed below) has its own way of implementing rotation.
A quick word about logging drivers
Did you notice “json” in the log filename from earlier? That’s because JSON File is the default logging driver on Docker.
Apart from JSON, there are many other logging mechanisms to choose from as well:
For large-scale deployments, GELF, AWS, GCP and Fluentd are recommended as they implement a centralized approach. But for small scale, JSON does the job just fine, while Local, Syslog and Journald are also quite suitable. Syslog and Logstash are particularly very helpful if complex log analysis becomes a requirement. ETW is Windows specific, whereas Splunk is focused on remote logging.
Starting from Docker Engine 20.10, a new feature called “dual-logging” has been introduced that ensures executing the docker logs command as well as performing log rotation irrespective of the logging driver in effect.
Summary
In this article, I’ve begun with the basics of how to view the logs for any Docker container, both at the container and host-system level. I’ve also showed you how to locate the physical location of container-specific log files residing on your host-system. I briefly showed you how to check the logs for the Docker service on the host itself.
Later, I’ve discussed logging drivers with focus on the default mechanism and also highlighted the new dual logging feature.
I hope these pointers help you in your daily monitoring or learning activities. Any more thoughts and suggestions are more than welcome! Please share them in the comment section below.
A Comprehensive Guide to Logging in Docker
Containers have become an increasingly popular way to deploy applications due to their portability and reproducibility across different environments. However, one important consideration when containerizing an application is logging. Logging plays a critical role in troubleshooting, identifying trends, and optimizing application performance. In the case of Docker containers, logging presents unique challenges as they are ephemeral and logs stored in the container's filesystem are lost once the container is terminated. To address this, it's important to transport logs to a more permanent location.
This article will cover the basics of logging in Docker containers, including how to access and view log messages, as well as crafting an optimal logging strategy tailored to your application's needs, and transporting your logs to a log management service for further processing and analysis. Note that this tutorial will not cover the logs generated by the docker daemon itself, as their management falls outside the scope of this article.
�� Want to centralize and monitor your Docker logs?
Head over to Logtail and start ingesting your logs in 5 minutes.
Prerequisites
Before you proceed with this tutorial, you will access to a system that supports Docker (v20.10 or later), preferably a Linux server that includes a non-root user with sudo access. A basic understanding of how Docker containers work is also assumed.
Setting up a demo container (optional)
To demonstrate the concepts described in this article, we will use a simple NGINX hello world application image to set up a Docker container. If you want to follow along, use the docker pull command with the image name ( karthequian/helloworld ) to download it from the Docker registry to your server.
You'll see the program's output appear on the screen:
The output above shows the process of fetching an image and storing it locally to be available for running containers. If you get some permissions error, you may need to prefix the command above with sudo or better still, add the current user to the docker group:
Once you've done that, log out and log back into your system again, and the docker pull command should work without prefixing it with sudo . You can subsequently use the docker images command to verify that the karthequian/helloworld image is present:
You should see the following output:
At this point, you can create and run a new container from the karthequian/helloworld container image by executing the docker run command as shown below:
The argument to the -p flag maps port 80 in the container to port 80 on your machine, while the -d option runs the container in the background and prints its ID (typical usage for web service). Assuming there were no errors, the container ID will be displayed on your screen.
If you get the error below, it means that some other program is already using port 80 on your server, so you should stop that program before rerunning the command.
You can verify the status of your Docker containers with the docker ps command:
You'll see the program's output appear on the screen:
The ps command describes a few details about your running containers. You can see the Container ID, the image running inside the container, the command used to start the container, its creation date, the status, exposed ports and the auto-generated name of the container.
If you visit the following URI in your browser http://<your_server_ip> in a browser, you'll see the sample NGINX hello world app.
You can reload the page a few times so that several log entries are generated. At this point, you're all setup to view and configure the logs generated by your running containers.
Viewing container logs with the Docker CLI
Docker provides a docker logs command for viewing the log messages produced within a container. You can also use the docker service logs command to display the logs produced by all containers participating in a service but we won't demonstrate that in this article.
By default, the logs command display all the messages sent to the standard output ( stdout ) and standard error ( stderr ) streams within a container, so if the services running on the container do not log to either stream (perhaps when they log to files instead), you may not see any useful output from the command.
For the NGINX hello world container, you will observe the following output:
Notice how we're able to view the NGINX logs for the application through the docker logs command despite the fact that NGINX writes its logs to access.log and error.log files in the /var/log/nginx directory by default. It works because the helloworld image configures NGINX to send the access log to /dev/stdout and the error log to /dev/stderr instead.
The official NGINX Docker image takes a different approach to achieve the same effect. It creates a symbolic link from /var/log/nginx/access.log to /dev/stdout and another one from /var/log/nginx/error.log to /dev/stderr so that the logs are collected by Docker.
These are two approaches you can use to ensure that you can access the relevant logs you need through docker logs .
Filtering your logs with the Docker CLI
If your application generates a lot of logs, the output from command docker logs command above may be huge so it's probably best to limit the output in some way using the available options:
1. Display the only the most recent logs
The —tail option can be used to display only the latest entries:
2. Limit the log output to a specific time range
Docker also provides the option to limit the log output by time so that you can quickly examine the entries that came through within the period you're interested in without being distracted by other entries.
There are two options for this task: —since and —until . The former specifies the lower time limit, while the latter sets an upper time limit for which logs should be displayed. The argument to both flags can be The argument to both flags can be a RFC 3339 date, a UNIX timestamp, or a Go duration string (e.g. 1m30s, 3h)
For example, the command below will only display log entries that were produced in the last 15 minutes:
This one will show all logs except those produced in the last 1 hour:
You can also derive a more specific output by combining both —since and —until . For example, the command below only prints the entries logged between 12PM and 1PM on Feb 16, 2023:
3. View container logs in real-time
The docker logs command displays only the log entries present at the time of execution. If you want to continue streaming output from the containers stdout and stderr , use the —follow option:
You can also combine this with either or both of the —tail or —since options to narrow down the initial output, while printing subsequent entries.
4. Filter Docker logs with grep
You can also filter the contents of docker logs through grep if you're only searching for lines containing a specific pattern:
Logging drivers in Docker
Docker uses logging drivers to retrieve logs from running containers and services. The default logging driver for a Docker containers is the json-file driver which internally caches container logs as JSON and stores them in files designed to be accessed through the docker daemon. These files can be found in the path below but take care not to modify them with external tools as it may interfere with Docker's logging system.
Also note that the json-file driver does not perform log rotation by default so it must be configured to do so, so that you don't end up running out of disk space (see the next section). Some of the other supported logging drivers in Docker are listed below:
- none : container logs are disabled. This causes docker logs to stop producing output.
- local : logs are stored in a custom format designed for minimal overhead. Preserves up to 100MB of logs by default (20 MB in a maximum of five files).
- syslog : writes log entries to the syslog facility on the host machine. You can read our tutorial on syslog if you are not familiar with it.
- journald : writes log messages to the journald daemon on the host machine. See our tutorial on journald for more details.
- fluentd : writes log messages to the fluentd daemon running on the host machine.
- gelf : sends log entries to a Graylog Extended Log Format (GELF) endpoint such as Graylog or Logstash.
- awslogs : sends log entries to AWS CloudWatch.
- gcplogs : sends log entries to the Google Cloud Platform.
- logentries : sends log entries to Rapid7 Logentries.
- splunk : uses the HTTP Event Collector to write log messages to Splunk.
- etwlogs (Windows only): writes log entries as Event Tracing for Windows (ETW) events.
You can check the current default logging driver for the Docker daemon through the command below:
This should yield the output below:
You can also find out the logging driver for a running container by using the following docker inspect command:
Configuring logging drivers in Docker
In this section, we will configure the json-file driver for our running Docker container for demonstration purposes. As mentioned earlier, log rotation and compression are not performed by default so we'll enable both options to ensure that the log files produced by Docker are kept to manageable sizes.
Start by opening or creating the daemon configuration file in /etc/docker/daemon.json :
Populate the file with the following contents:
The log-driver field sets the logging driver to json-file while the log-opts property configures the supported options for the specified logging driver. Note that each property in the log-opts object must have a string value (including boolean and numeric values as seen above).
Here's a brief explanation of each of the json-file options above:
-
max-size : the maximum size of the log file before it is rotated. An integer plus a modifier represents the measuring unit ( k for kilobytes, m for megabytes, and g for gigabytes). Defaults to -1 (unlimited).
max-file : the maximum number of log files that can be present. When log rotation creates excess files, the oldest one will be removed. The max-size option must also be specified for this setting to take effect.
compress : if this is set to true , rotated log files will be compressed to save disk space.
The complete list of all the options for the json-file driver is defined in the official documentation , so be sure to check them out.
When you modify the docker daemon configuration file as above, you must restart the docker service to apply the changes to newly created containers. Existing containers will not adopt the new configuration until they are recreated using docker run .
After restarting the docker service, all your running containers will be terminated, so you must start them again with docker run . Do explore the live restore option if you want to keep your containers running even when the daemon becomes unavailable.
Prior to Docker 20.10, the docker logs command only worked when the logging driver was set to local , json-file , or journald , but this changed with the introduction of dual logging in Docker 20.10. This feature allows docker logs to read container logs locally in a consistent format regardless of the logging driver in use. It's enabled by default, but if you prefer to disable it, use the cache-disabled property shown below.
Note that the cache-disabled option does not affect the local , json-file , or journald drivers since they do not use the dual logging feature. When cache-disabled is true for any other logging driver, the docker logs command will stop working:
Overriding the default logging driver per container
It is sometimes useful to set a logging driver (or options) other than the default for a specific container. This can be done through the —log-driver and —log-opt flags when using docker container create or docker run :
Choosing a logging delivery mode
The logging delivery mode for a Docker container refers to how it prioritizes the delivery of incoming log messages to the configured driver. The following two modes are supported, and they can be used with any logging driver:
1. Blocking mode
In blocking mode (the default), the delivery of log messages to the driver will block all other operations that the container is performing, which may impact its performance, especially with drivers that write to a remote service. The main advantage of blocking mode is that it guarantees that each log message will be delivered to the driver.
You can keep using the blocking mode when the logging driver being used writes to the local filesystem. It is unlikely that the latency introduced will be significant since these drivers are generally rapid.
2. Non-blocking mode
In non-blocking mode, incoming log entries are stored in a memory buffer until the configured logging driver is available to process them. Once the logs are processed, they are cleared from the buffer to make way for new entries.
The advantage of this mode is that the application's performance will not be impacted. However, it also introduces the possibility of losing log entries if the memory buffer is filled up so that existing entries are deleted before processing. To decrease the likelihood of losing log messages in containers that generate a significant amount of logs, you can increase the maximum memory buffer size from its default (1MB) through the max-buffer-size property:
Choosing a Docker logging strategy
There are several ways to aggregate, store and centralize your Docker container logs. Thus far, we've only covered the native Docker logging driver approach in this article since it's the most suitable for many everyday use cases. However, that strategy won't fit every project, so it's a good idea to be aware of alternative methods so that you can make an informed decision about the right logging approach for your application.
1. Using application-based logging
In this approach, the application itself handles its own logging through a logging framework. For example, a Node.js application could use Winston or Pino to format and transport its logs to a log management solution for storage and further processing.
This approach provides the greatest amount of control for application developers to generate, format, and transport the logs as they see fit. However, this control has a performance cost since everything (including the complexities of log delivery) is done at the application level.
Another consideration is that logs must be transported to a remote log management service (such as Logtail) or a data volume outside the container's filesystem to prevent loss on container termination.
2. Using a logging driver
We discussed this approach in great detail throughout this tutorial. To recap, it involves logging to the stdout and stderr streams so that they are picked up by Docker's log collector, and using a logging driver to manage how the logs are stored or transported.
This is the native way to log in Docker, and it reduces the impact of logging on your application's performance. The main downside to this approach is that it creates a dependency between your container and its host, but this is tolerable in most situations.
3. Using data volumes
If you don't want your container logs to be lost immediately after the container is terminated, you can link a directory inside the container to a directory on the Docker host where the log entries are transported to. This ensures that your logs are retained even when the container is destroyed, and it also makes it easy to aggregate logs from multiple containers in one place and make copies or transport them to some remote location.
4. Using a dedicated logging container
Setting up a dedicated container whose sole purpose is to aggregate and centralize Docker logs could be a great solution, especially when deploying a microservice architecture. This approach removes the dependency on the host machine and makes it easier to scale up your logging infrastructure by simply adding a new container when needed.
5. Using the sidecar approach
A technique used for more complicated deployments is the sidecar approach in which each Docker container has its own dedicated logging container (they are considered a single unit). The main advantage is that the logging approach can be tailored for each application, and it offers greater transparency regarding the origin of each log entry.
A drawback to this strategy is that setting up a logging container per application will consume more resources than the dedicated logging container approach, and the added complexity may not be worthwhile for smaller deployments. You also need to use Docker compose to ensure that both containers are managed as a single unit to avoid incomplete or missing log data.
Final thoughts
In conclusion, this article offered a comprehensive overview of logging in Docker containers to facilitate the quick and efficient deployment of applications in a Docker container. It covered the fundamentals of Docker container logging, including log storage and retrieval, as well as the configuration of logging drivers.
Additionally, alternative approaches were discussed for an optimal logging setup in Docker. To delve deeper into this topic, we recommend exploring other articles in this series or consulting the official documentation .