{"id":41363,"date":"2023-11-20T04:45:38","date_gmt":"2023-11-20T04:45:38","guid":{"rendered":"https:\/\/www.devopsschool.com\/blog\/?p=41363"},"modified":"2025-05-04T06:12:20","modified_gmt":"2025-05-04T06:12:20","slug":"elastic-tutorials-collects-docker-metrics-and-logs-using-elastic-agent","status":"publish","type":"post","link":"https:\/\/www.devopsschool.com\/blog\/elastic-tutorials-collects-docker-metrics-and-logs-using-elastic-agent\/","title":{"rendered":"Elastic Tutorials: Collects Docker metrics and logs using Elastic Agent"},"content":{"rendered":"\n<p>This Integration collects metrics and logs from\u00a0Docker containers. The default data streams for metrics collection are:\u00a0<code>container<\/code>,\u00a0<code>cpu<\/code>,\u00a0<code>diskio<\/code>,\u00a0<code>healthcheck<\/code>,\u00a0<code>info<\/code>,\u00a0<code>memory<\/code>\u00a0and\u00a0<code>network<\/code>. The\u00a0<code>image<\/code>\u00a0metricset is not enabled by default. The\u00a0<code>container_logs<\/code>\u00a0data stream for containers&#8217; logs collection is enabled by default.<\/p>\n\n\n\n<h4 class=\"wp-block-heading\">Compatibility<\/h4>\n\n\n\n<p>The Docker module is currently tested on Linux and Mac with the community edition engine, versions 1.11 and 17.09.0-ce. It is not tested on Windows, but it should also work there.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Step 1 &#8211; Install Docker<\/h2>\n\n\n\n<figure class=\"wp-block-embed is-type-wp-embed is-provider-devopsschool-com wp-block-embed-devopsschool-com\"><div class=\"wp-block-embed__wrapper\">\n<blockquote class=\"wp-embedded-content\" data-secret=\"kiduXJqprY\"><a href=\"https:\/\/www.devopsschool.com\/blog\/how-to-install-docker-in-linux\/\">Docker Tutorials: How to Install Docker in Ubuntu?<\/a><\/blockquote><iframe loading=\"lazy\" class=\"wp-embedded-content\" sandbox=\"allow-scripts\" security=\"restricted\" style=\"position: absolute; clip: rect(1px, 1px, 1px, 1px);\" title=\"&#8220;Docker Tutorials: How to Install Docker in Ubuntu?&#8221; &#8212; DevOpsSchool.com\" src=\"https:\/\/www.devopsschool.com\/blog\/how-to-install-docker-in-linux\/embed\/#?secret=rb3HaxGo4d#?secret=kiduXJqprY\" data-secret=\"kiduXJqprY\" width=\"600\" height=\"338\" frameborder=\"0\" marginwidth=\"0\" marginheight=\"0\" scrolling=\"no\"><\/iframe>\n<\/div><\/figure>\n\n\n\n<h2 class=\"wp-block-heading\">Step 2 &#8211; Enable Integration<\/h2>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"490\" src=\"https:\/\/www.devopsschool.com\/blog\/wp-content\/uploads\/2023\/11\/elastic-agent-docker-1-1024x490.png\" alt=\"\" class=\"wp-image-41364\" srcset=\"https:\/\/www.devopsschool.com\/blog\/wp-content\/uploads\/2023\/11\/elastic-agent-docker-1-1024x490.png 1024w, https:\/\/www.devopsschool.com\/blog\/wp-content\/uploads\/2023\/11\/elastic-agent-docker-1-300x143.png 300w, https:\/\/www.devopsschool.com\/blog\/wp-content\/uploads\/2023\/11\/elastic-agent-docker-1-768x367.png 768w, https:\/\/www.devopsschool.com\/blog\/wp-content\/uploads\/2023\/11\/elastic-agent-docker-1-1536x734.png 1536w, https:\/\/www.devopsschool.com\/blog\/wp-content\/uploads\/2023\/11\/elastic-agent-docker-1.png 1690w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"606\" src=\"https:\/\/www.devopsschool.com\/blog\/wp-content\/uploads\/2023\/11\/elastic-agent-docker-2-1024x606.png\" alt=\"\" class=\"wp-image-41365\" srcset=\"https:\/\/www.devopsschool.com\/blog\/wp-content\/uploads\/2023\/11\/elastic-agent-docker-2-1024x606.png 1024w, https:\/\/www.devopsschool.com\/blog\/wp-content\/uploads\/2023\/11\/elastic-agent-docker-2-300x178.png 300w, https:\/\/www.devopsschool.com\/blog\/wp-content\/uploads\/2023\/11\/elastic-agent-docker-2-768x454.png 768w, https:\/\/www.devopsschool.com\/blog\/wp-content\/uploads\/2023\/11\/elastic-agent-docker-2.png 1264w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<h2 class=\"wp-block-heading\">Step 3 &#8211; Work on Agent Configuration changes<\/h2>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n<pre class=\"wp-block-code\" aria-describedby=\"shcb-language-1\" data-shcb-language-name=\"PHP\" data-shcb-language-slug=\"php\"><span><code class=\"hljs language-php\">$ cd \/opt\/Elastic\/Agent\r\n$ systemctl stop elastic-agent\r\n<span class=\"hljs-comment\"># Replace the content of elastic-agent.yml which you recieved from Kibana.<\/span>\r\n$ vi elastic-agent.yml\r\n$ systemctl restart elastic-agent\r\n$ systemctl status elastic-agent\r\n<\/code><\/span><small class=\"shcb-language\" id=\"shcb-language-1\"><span class=\"shcb-language__label\">Code language:<\/span> <span class=\"shcb-language__name\">PHP<\/span> <span class=\"shcb-language__paren\">(<\/span><span class=\"shcb-language__slug\">php<\/span><span class=\"shcb-language__paren\">)<\/span><\/small><\/pre>\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h2 class=\"wp-block-heading\">Step 4 &#8211; Create docker containers<\/h2>\n\n\n<pre class=\"wp-block-code\"><span><code class=\"hljs\">$  docker run -itd ubuntu<\/code><\/span><\/pre>\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"89\" src=\"https:\/\/www.devopsschool.com\/blog\/wp-content\/uploads\/2023\/11\/image-27-1024x89.png\" alt=\"\" class=\"wp-image-41366\" srcset=\"https:\/\/www.devopsschool.com\/blog\/wp-content\/uploads\/2023\/11\/image-27-1024x89.png 1024w, https:\/\/www.devopsschool.com\/blog\/wp-content\/uploads\/2023\/11\/image-27-300x26.png 300w, https:\/\/www.devopsschool.com\/blog\/wp-content\/uploads\/2023\/11\/image-27-768x67.png 768w, https:\/\/www.devopsschool.com\/blog\/wp-content\/uploads\/2023\/11\/image-27-1536x134.png 1536w, https:\/\/www.devopsschool.com\/blog\/wp-content\/uploads\/2023\/11\/image-27.png 1629w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<h2 class=\"wp-block-heading\">Step 5 &#8211; Verify Containers<\/h2>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"416\" src=\"https:\/\/www.devopsschool.com\/blog\/wp-content\/uploads\/2023\/11\/image-28-1024x416.png\" alt=\"\" class=\"wp-image-41367\" srcset=\"https:\/\/www.devopsschool.com\/blog\/wp-content\/uploads\/2023\/11\/image-28-1024x416.png 1024w, https:\/\/www.devopsschool.com\/blog\/wp-content\/uploads\/2023\/11\/image-28-300x122.png 300w, https:\/\/www.devopsschool.com\/blog\/wp-content\/uploads\/2023\/11\/image-28-768x312.png 768w, https:\/\/www.devopsschool.com\/blog\/wp-content\/uploads\/2023\/11\/image-28-1536x624.png 1536w, https:\/\/www.devopsschool.com\/blog\/wp-content\/uploads\/2023\/11\/image-28.png 1906w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"423\" src=\"https:\/\/www.devopsschool.com\/blog\/wp-content\/uploads\/2023\/11\/image-29-1024x423.png\" alt=\"\" class=\"wp-image-41368\" srcset=\"https:\/\/www.devopsschool.com\/blog\/wp-content\/uploads\/2023\/11\/image-29-1024x423.png 1024w, https:\/\/www.devopsschool.com\/blog\/wp-content\/uploads\/2023\/11\/image-29-300x124.png 300w, https:\/\/www.devopsschool.com\/blog\/wp-content\/uploads\/2023\/11\/image-29-768x317.png 768w, https:\/\/www.devopsschool.com\/blog\/wp-content\/uploads\/2023\/11\/image-29-1536x635.png 1536w, https:\/\/www.devopsschool.com\/blog\/wp-content\/uploads\/2023\/11\/image-29.png 1933w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<h2 class=\"wp-block-heading\">Running from within Docker<\/h2>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n<pre class=\"wp-block-code\" aria-describedby=\"shcb-language-2\" data-shcb-language-name=\"JavaScript\" data-shcb-language-slug=\"javascript\"><span><code class=\"hljs language-javascript\">The docker Integration will <span class=\"hljs-keyword\">try<\/span> to connect to the docker socket, by <span class=\"hljs-keyword\">default<\/span> at unix:<span class=\"hljs-comment\">\/\/\/var\/run\/docker.sock. If Elastic Agent is running inside docker, you'll need to mount the unix socket inside the container:<\/span>\r\n\r\ndocker run -d \\\r\n  --name=metricbeat \\\r\n  --user=root \\\r\n  --volume=<span class=\"hljs-string\">\"\/var\/run\/docker.sock:\/var\/run\/docker.sock:ro\"<\/span> \\\r\n  docker.elastic.co\/beats\/metricbeat:latest metricbeat -e \\\r\n  -E output.elasticsearch.hosts=&#91;<span class=\"hljs-string\">\"elasticsearch:9200\"<\/span>]\r\n\r\nFor log collection since the discovery <span class=\"hljs-keyword\">of<\/span> the containers happen automatically, again access to unix:<span class=\"hljs-comment\">\/\/\/var\/run\/docker.sock will be needed so as Agent to be able to watch for Container events. In addition, access is required to the containers' logs files which by default follows the pattern of \/var\/lib\/docker\/containers\/${docker.container.id}\/*-json.log If Elastic Agent is running inside docker, you'll need to mount the logs' directory too inside the container:<\/span>\r\n\r\ndocker run -d \\\r\n  --name=metricbeat \\\r\n  --user=root \\\r\n  --volume=<span class=\"hljs-string\">\"\/var\/run\/docker.sock:\/var\/run\/docker.sock:ro\"<\/span> \\\r\n  --volume=<span class=\"hljs-string\">\"\/var\/lib\/docker\/containers:\/var\/lib\/docker\/containers:ro\"<\/span> \\\r\n  docker.elastic.co\/beats\/metricbeat:latest metricbeat -e \\\r\n  -E output.elasticsearch.hosts=&#91;<span class=\"hljs-string\">\"elasticsearch:9200\"<\/span>]\r\n\r\nIn all cases make sure that Agent has the proper permissions to access these files.<\/code><\/span><small class=\"shcb-language\" id=\"shcb-language-2\"><span class=\"shcb-language__label\">Code language:<\/span> <span class=\"shcb-language__name\">JavaScript<\/span> <span class=\"shcb-language__paren\">(<\/span><span class=\"shcb-language__slug\">javascript<\/span><span class=\"shcb-language__paren\">)<\/span><\/small><\/pre>\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n","protected":false},"excerpt":{"rendered":"<p>This Integration collects metrics and logs from\u00a0Docker containers. The default data streams for metrics collection are:\u00a0container,\u00a0cpu,\u00a0diskio,\u00a0healthcheck,\u00a0info,\u00a0memory\u00a0and\u00a0network. The\u00a0image\u00a0metricset is not enabled by default. The\u00a0container_logs\u00a0data stream for containers&#8217; logs collection is enabled&#8230; <\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_joinchat":[],"footnotes":""},"categories":[5936],"tags":[],"class_list":["post-41363","post","type-post","status-publish","format-standard","hentry","category-elastic"],"_links":{"self":[{"href":"https:\/\/www.devopsschool.com\/blog\/wp-json\/wp\/v2\/posts\/41363","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.devopsschool.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.devopsschool.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.devopsschool.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.devopsschool.com\/blog\/wp-json\/wp\/v2\/comments?post=41363"}],"version-history":[{"count":1,"href":"https:\/\/www.devopsschool.com\/blog\/wp-json\/wp\/v2\/posts\/41363\/revisions"}],"predecessor-version":[{"id":41369,"href":"https:\/\/www.devopsschool.com\/blog\/wp-json\/wp\/v2\/posts\/41363\/revisions\/41369"}],"wp:attachment":[{"href":"https:\/\/www.devopsschool.com\/blog\/wp-json\/wp\/v2\/media?parent=41363"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.devopsschool.com\/blog\/wp-json\/wp\/v2\/categories?post=41363"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.devopsschool.com\/blog\/wp-json\/wp\/v2\/tags?post=41363"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}