{"id":12626,"date":"2021-09-26T20:14:14","date_gmt":"2021-09-26T20:14:14","guid":{"rendered":"https:\/\/www.devopsschool.com\/blog\/?p=12626"},"modified":"2025-07-12T05:41:42","modified_gmt":"2025-07-12T05:41:42","slug":"how-to-secure-prometheus-docker-endpoint-after-enabling-through-metrics-addr-in-daemon-json","status":"publish","type":"post","link":"https:\/\/www.devopsschool.com\/blog\/how-to-secure-prometheus-docker-endpoint-after-enabling-through-metrics-addr-in-daemon-json\/","title":{"rendered":"Install and Configure Prometheus for Collecting Docker metrics"},"content":{"rendered":"\n<h2 class=\"wp-block-heading\">How to install Install Docker<\/h2>\n\n\n\n<ul class=\"wp-block-list\">\n<li><a href=\"https:\/\/devopsschool.com\/tutorial\/docker\/install-config\/docker-install-commuityedition-centos-rhel.html\" target=\"_blank\" rel=\"noreferrer noopener\">https:\/\/devopsschool.com\/tutorial\/docker\/install-config\/docker-install-commuityedition-centos-rhel.html<\/a><\/li>\n<\/ul>\n\n\n\n<p><\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Configure docker for exposing a Metrices at end point<\/h2>\n\n\n\n<p>To configure the Docker daemon as a Prometheus target, you need to specify the metrics-address. The best way to do this is via the daemon.json, which is located at one of the following locations by default. If the file does not exist, create it.<\/p>\n\n\n<pre class=\"wp-block-code\" aria-describedby=\"shcb-language-1\" data-shcb-language-name=\"JavaScript\" data-shcb-language-slug=\"javascript\"><span><code class=\"hljs language-javascript\">Linux: <span class=\"hljs-regexp\">\/etc\/<\/span>docker\/daemon.json\nWindows Server: C:\\ProgramData\\docker\\config\\daemon.json\nDocker Desktop <span class=\"hljs-keyword\">for<\/span> Mac \/ Docker Desktop <span class=\"hljs-keyword\">for<\/span> Windows: Click the Docker icon <span class=\"hljs-keyword\">in<\/span> the toolbar, select Preferences, then select Daemon. Click Advanced.\n<\/code><\/span><small class=\"shcb-language\" id=\"shcb-language-1\"><span class=\"shcb-language__label\">Code language:<\/span> <span class=\"shcb-language__name\">JavaScript<\/span> <span class=\"shcb-language__paren\">(<\/span><span class=\"shcb-language__slug\">javascript<\/span><span class=\"shcb-language__paren\">)<\/span><\/small><\/pre>\n\n<pre class=\"wp-block-code\" aria-describedby=\"shcb-language-2\" data-shcb-language-name=\"PHP\" data-shcb-language-slug=\"php\"><span><code class=\"hljs language-php\"><span class=\"hljs-keyword\">If<\/span> the file is currently <span class=\"hljs-keyword\">empty<\/span>, paste the following:\n\n{\n  <span class=\"hljs-string\">\"metrics-addr\"<\/span> : <span class=\"hljs-string\">\"127.0.0.1:9323\"<\/span>,\n  <span class=\"hljs-string\">\"experimental\"<\/span> : <span class=\"hljs-keyword\">true<\/span>\n}<\/code><\/span><small class=\"shcb-language\" id=\"shcb-language-2\"><span class=\"shcb-language__label\">Code language:<\/span> <span class=\"shcb-language__name\">PHP<\/span> <span class=\"shcb-language__paren\">(<\/span><span class=\"shcb-language__slug\">php<\/span><span class=\"shcb-language__paren\">)<\/span><\/small><\/pre>\n\n\n<p>Save the file, or in the case of Docker Desktop for Mac or Docker Desktop for Windows, save the configuration. Restart Docker.<\/p>\n\n\n\n<p>$ sudo systemctl restart docker<\/p>\n\n\n\n<p>Docker now exposes Prometheus-compatible metrics on port 9323.<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"958\" height=\"592\" src=\"https:\/\/www.devopsschool.com\/blog\/wp-content\/uploads\/2022\/11\/image-46.png\" alt=\"\" class=\"wp-image-32004\" srcset=\"https:\/\/www.devopsschool.com\/blog\/wp-content\/uploads\/2022\/11\/image-46.png 958w, https:\/\/www.devopsschool.com\/blog\/wp-content\/uploads\/2022\/11\/image-46-300x185.png 300w, https:\/\/www.devopsschool.com\/blog\/wp-content\/uploads\/2022\/11\/image-46-768x475.png 768w\" sizes=\"auto, (max-width: 958px) 100vw, 958px\" \/><\/figure>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"607\" height=\"432\" src=\"https:\/\/www.devopsschool.com\/blog\/wp-content\/uploads\/2021\/09\/image-15.png\" alt=\"\" class=\"wp-image-23926\" srcset=\"https:\/\/www.devopsschool.com\/blog\/wp-content\/uploads\/2021\/09\/image-15.png 607w, https:\/\/www.devopsschool.com\/blog\/wp-content\/uploads\/2021\/09\/image-15-300x214.png 300w\" sizes=\"auto, (max-width: 607px) 100vw, 607px\" \/><\/figure>\n\n\n\n<h2 class=\"wp-block-heading\">Configure and run Prometheus<\/h2>\n\n\n<pre class=\"wp-block-code\" aria-describedby=\"shcb-language-3\" data-shcb-language-name=\"PHP\" data-shcb-language-slug=\"php\"><span><code class=\"hljs language-php\"><span class=\"hljs-comment\"># my global config<\/span>\n<span class=\"hljs-keyword\">global<\/span>:\n  scrape_interval:     <span class=\"hljs-number\">15<\/span>s <span class=\"hljs-comment\"># Set the scrape interval to every 15 seconds. Default is every 1 minute.<\/span>\n  evaluation_interval: <span class=\"hljs-number\">15<\/span>s <span class=\"hljs-comment\"># Evaluate rules every 15 seconds. The default is every 1 minute.<\/span>\n  <span class=\"hljs-comment\"># scrape_timeout is set to the global default (10s).<\/span>\n\n  <span class=\"hljs-comment\"># Attach these labels to any time series or alerts when communicating with<\/span>\n  <span class=\"hljs-comment\"># external systems (federation, remote storage, Alertmanager).<\/span>\n  external_labels:\n      monitor: <span class=\"hljs-string\">'codelab-monitor'<\/span>\n\n<span class=\"hljs-comment\"># Load rules once and periodically evaluate them according to the global 'evaluation_interval'.<\/span>\nrule_files:\n  <span class=\"hljs-comment\"># - \"first.rules\"<\/span>\n  <span class=\"hljs-comment\"># - \"second.rules\"<\/span>\n\n<span class=\"hljs-comment\"># A scrape configuration containing exactly one endpoint to scrape:<\/span>\n<span class=\"hljs-comment\"># Here it's Prometheus itself.<\/span>\nscrape_configs:\n  <span class=\"hljs-comment\"># The job name is added as a label `job=&lt;job_name&gt;` to any timeseries scraped from this config.<\/span>\n  - job_name: <span class=\"hljs-string\">'prometheus'<\/span>\n\n    <span class=\"hljs-comment\"># metrics_path defaults to '\/metrics'<\/span>\n    <span class=\"hljs-comment\"># scheme defaults to 'http'.<\/span>\n\n    static_configs:\n      - targets: &#91;<span class=\"hljs-string\">'localhost:9090'<\/span>]\n\n  - job_name: <span class=\"hljs-string\">'docker'<\/span>\n         <span class=\"hljs-comment\"># metrics_path defaults to '\/metrics'<\/span>\n         <span class=\"hljs-comment\"># scheme defaults to 'http'.<\/span>\n\n    static_configs:\n      - targets: &#91;<span class=\"hljs-string\">'localhost:9323'<\/span>]<\/code><\/span><small class=\"shcb-language\" id=\"shcb-language-3\"><span class=\"shcb-language__label\">Code language:<\/span> <span class=\"shcb-language__name\">PHP<\/span> <span class=\"shcb-language__paren\">(<\/span><span class=\"shcb-language__slug\">php<\/span><span class=\"shcb-language__paren\">)<\/span><\/small><\/pre>\n\n\n<figure class=\"wp-block-image size-full is-resized\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/www.devopsschool.com\/blog\/wp-content\/uploads\/2021\/09\/image-16.png\" alt=\"\" class=\"wp-image-23927\" width=\"768\" height=\"501\" srcset=\"https:\/\/www.devopsschool.com\/blog\/wp-content\/uploads\/2021\/09\/image-16.png 768w, https:\/\/www.devopsschool.com\/blog\/wp-content\/uploads\/2021\/09\/image-16-300x196.png 300w\" sizes=\"auto, (max-width: 768px) 100vw, 768px\" \/><\/figure>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"647\" height=\"504\" src=\"https:\/\/www.devopsschool.com\/blog\/wp-content\/uploads\/2021\/09\/image-17.png\" alt=\"\" class=\"wp-image-23928\" srcset=\"https:\/\/www.devopsschool.com\/blog\/wp-content\/uploads\/2021\/09\/image-17.png 647w, https:\/\/www.devopsschool.com\/blog\/wp-content\/uploads\/2021\/09\/image-17-300x234.png 300w\" sizes=\"auto, (max-width: 647px) 100vw, 647px\" \/><\/figure>\n\n\n\n<p>Verify that the D<strong>ocker target is listed at http:\/\/localhost:9090\/targets\/.<\/strong><\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"566\" height=\"410\" src=\"https:\/\/www.devopsschool.com\/blog\/wp-content\/uploads\/2021\/09\/image-18.png\" alt=\"\" class=\"wp-image-23929\" srcset=\"https:\/\/www.devopsschool.com\/blog\/wp-content\/uploads\/2021\/09\/image-18.png 566w, https:\/\/www.devopsschool.com\/blog\/wp-content\/uploads\/2021\/09\/image-18-300x217.png 300w\" sizes=\"auto, (max-width: 566px) 100vw, 566px\" \/><\/figure>\n\n\n\n<h2 class=\"wp-block-heading\">Use Prometheus for Swarm only<\/h2>\n\n\n\n<p>Create a graph. Click the Graphs link in the Prometheus UI. Choose a metric from the combo box to the right of the Execute button, and click Execute. The screenshot below shows the graph for<br>engine_daemon_network_actions_seconds_count.<\/p>\n\n\n\n<p>The above graph shows a pretty idle Docker instance. Your graph might look different if you are running active workloads.<\/p>\n\n\n\n<p>To make the graph more interesting, create some network actions by starting a service with 10 tasks that just ping Docker non-stop (you can change the ping target to anything you like):<\/p>\n\n\n\n<p>$ docker service create &#8211;replicas 10 &#8211;name ping_service alpine ping docker.com<\/p>\n\n\n\n<p>Wait a few minutes (the default scrape interval is 15 seconds) and reload your graph.<\/p>\n\n\n\n<p>When you are ready, stop and remove the ping_service service, so that you are not flooding a host with pings for no reason.<\/p>\n\n\n\n<p>docker service remove ping_service<br>Wait a few minutes and you should see that the graph falls back to the idle level.<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"761\" height=\"429\" src=\"https:\/\/www.devopsschool.com\/blog\/wp-content\/uploads\/2021\/09\/image-7.png\" alt=\"\" class=\"wp-image-23903\" srcset=\"https:\/\/www.devopsschool.com\/blog\/wp-content\/uploads\/2021\/09\/image-7.png 761w, https:\/\/www.devopsschool.com\/blog\/wp-content\/uploads\/2021\/09\/image-7-300x169.png 300w, https:\/\/www.devopsschool.com\/blog\/wp-content\/uploads\/2021\/09\/image-7-355x199.png 355w\" sizes=\"auto, (max-width: 761px) 100vw, 761px\" \/><\/figure>\n\n\n\n<h2 class=\"wp-block-heading\">How to secure Prometheus Docker Endpoint after enabling through metrics-addr in daemon.json<\/h2>\n\n\n\n<p>[Experiment &#8211; NOT Tested]<\/p>\n\n\n\n<p>If you need to access the Docker daemon remotely, you need to enable the tcp Socket. Beware that the default setup provides un-encrypted and un-authenticated direct access to the Docker daemon &#8211; and should be secured either <\/p>\n\n\n\n<ol class=\"wp-block-list\">\n<li>Using the built in HTTPS encrypted socket, or <\/li>\n\n\n\n<li>By putting a secure web proxy in front of it. <\/li>\n<\/ol>\n\n\n\n<p>If you need to access the Docker daemon remotely, you need to enable the tcp Socket. Beware that the default setup provides un-encrypted and un-authenticated direct access to the Docker daemon &#8211; and should be secured either using the built in HTTPS encrypted socket, or by putting a secure web proxy in front of it. <\/p>\n\n\n\n<p>Note: If you\u2019re using an HTTPS encrypted socket, keep in mind that only TLS1.0 and greater are supported. Protocols SSLv3 and under are not supported anymore for security reasons.<\/p>\n\n\n\n<p><strong>Method 1 &#8211; Protect or Secure the Docker daemon socket <\/strong><br> https:\/\/docs.docker.com\/engine\/security\/https\/<br> https:\/\/docs.docker.com\/config\/daemon\/<br> https:\/\/docs.docker.com\/engine\/security\/https\/<br> https:\/\/gist.github.com\/kekru\/b9e4da822514df93e6fdf2f7d3d90d8a<\/p>\n\n\n\n<p><strong>Method 2 &#8211; secure web proxy<\/strong><br> One option to help secure our Prometheus server is to put it behind a reverse proxy so that we can later add SSL and an Authentication layer over the default unrestricted Prometheus web interface.<\/p>\n\n\n\n<p><strong>Example of daemon.json<\/strong><\/p>\n\n\n<pre class=\"wp-block-code\" aria-describedby=\"shcb-language-4\" data-shcb-language-name=\"JSON \/ JSON with Comments\" data-shcb-language-slug=\"json\"><span><code class=\"hljs language-json\">{\n  <span class=\"hljs-attr\">\"metrics-addr\"<\/span> : <span class=\"hljs-string\">\"127.0.0.1:9323\"<\/span>,\n  <span class=\"hljs-attr\">\"experimental\"<\/span> : <span class=\"hljs-literal\">true<\/span>\n}\n<\/code><\/span><small class=\"shcb-language\" id=\"shcb-language-4\"><span class=\"shcb-language__label\">Code language:<\/span> <span class=\"shcb-language__name\">JSON \/ JSON with Comments<\/span> <span class=\"shcb-language__paren\">(<\/span><span class=\"shcb-language__slug\">json<\/span><span class=\"shcb-language__paren\">)<\/span><\/small><\/pre>\n\n\n<p><strong>We will use Nginx.<\/strong><\/p>\n\n\n<pre class=\"wp-block-code\" aria-describedby=\"shcb-language-5\" data-shcb-language-name=\"PHP\" data-shcb-language-slug=\"php\"><span><code class=\"hljs language-php\">$ sudo apt install nginx\n<span class=\"hljs-comment\"># CD to the Nginx sites-enabled folder<\/span>\ncd \/etc\/nginx\/sites-enabled\n<span class=\"hljs-comment\"># Create a new Nginx configuration from Prometheus<\/span>\n$ sudo nano prometheus\n<span class=\"hljs-keyword\">And<\/span> copy\/paste the example below\nserver {\n    listen       <span class=\"hljs-number\">443<\/span>;   \n\n    location \/ {\n        proxy_pass           http:<span class=\"hljs-comment\">\/\/localhost:9323\/;<\/span>\n    }\n}\n<span class=\"hljs-comment\"># Save and restart Nginx<\/span>\n$ sudo service nginx restart\n$ sudo service nginx status\n<\/code><\/span><small class=\"shcb-language\" id=\"shcb-language-5\"><span class=\"shcb-language__label\">Code language:<\/span> <span class=\"shcb-language__name\">PHP<\/span> <span class=\"shcb-language__paren\">(<\/span><span class=\"shcb-language__slug\">php<\/span><span class=\"shcb-language__paren\">)<\/span><\/small><\/pre>","protected":false},"excerpt":{"rendered":"<p>How to install Install Docker Configure docker for exposing a Metrices at end point To configure the Docker daemon as a Prometheus target, you need to specify the metrics-address. The best way to do this is via the daemon.json, which is located at one of the following locations by default. If the file does not&#8230;<\/p>\n","protected":false},"author":1,"featured_media":12630,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_kad_post_transparent":"","_kad_post_title":"","_kad_post_layout":"","_kad_post_sidebar_id":"","_kad_post_content_style":"","_kad_post_vertical_padding":"","_kad_post_feature":"","_kad_post_feature_position":"","_kad_post_header":false,"_kad_post_footer":false,"_kad_post_classname":"","_joinchat":[],"footnotes":""},"categories":[4862],"tags":[95,5449,567,6014,5014,533],"class_list":["post-12626","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-docker","tag-devops","tag-devopsschool","tag-docker","tag-endppoint","tag-prometheus","tag-secure"],"_links":{"self":[{"href":"https:\/\/www.devopsschool.com\/blog\/wp-json\/wp\/v2\/posts\/12626","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.devopsschool.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.devopsschool.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.devopsschool.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.devopsschool.com\/blog\/wp-json\/wp\/v2\/comments?post=12626"}],"version-history":[{"count":7,"href":"https:\/\/www.devopsschool.com\/blog\/wp-json\/wp\/v2\/posts\/12626\/revisions"}],"predecessor-version":[{"id":32005,"href":"https:\/\/www.devopsschool.com\/blog\/wp-json\/wp\/v2\/posts\/12626\/revisions\/32005"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.devopsschool.com\/blog\/wp-json\/wp\/v2\/media\/12630"}],"wp:attachment":[{"href":"https:\/\/www.devopsschool.com\/blog\/wp-json\/wp\/v2\/media?parent=12626"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.devopsschool.com\/blog\/wp-json\/wp\/v2\/categories?post=12626"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.devopsschool.com\/blog\/wp-json\/wp\/v2\/tags?post=12626"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}