{"id":39,"date":"2026-10-03T14:49:51","date_gmt":"2026-10-03T14:49:51","guid":{"rendered":"https:\/\/hoho.live\/luda\/index.php\/2026\/10\/03\/how-promtail-reads-the-docker-api-and-pushes-logs-to-loki\/"},"modified":"2026-10-04T09:48:25","modified_gmt":"2026-10-04T01:48:25","slug":"how-promtail-reads-the-docker-api-and-pushes-logs-to-loki","status":"publish","type":"post","link":"https:\/\/hoho.live\/luda\/index.php\/2026\/10\/03\/how-promtail-reads-the-docker-api-and-pushes-logs-to-loki\/","title":{"rendered":"How Promtail reads the Docker API and pushes logs to Loki"},"content":{"rendered":"\n

Short answer:<\/strong> Promtail<\/strong> does not open Docker’s log files. It connects to the local Docker socket, lists container ids, and calls the engine logs<\/strong> API with follow<\/code>. It splits stdout<\/strong> from stderr<\/strong>, removes Docker’s timestamp prefix, and POSTs<\/strong> the lines to Loki at \/loki\/api\/v1\/push<\/code>. This page assumes that push URL is already reachable.<\/p>\n\n\n

\n\n

What Promtail asks the Docker API<\/strong><\/h4>\n\n

docker_sd_configs<\/code> with host: unix:\/\/\/var\/run\/docker.sock<\/code> talks to the engine on that socket. On each refresh it lists containers that are running then. A container that exits between refreshes is not added. The discovery labels that matter are __meta_docker_container_id<\/code> and __meta_docker_container_name<\/code>. The name label has a leading \/<\/code>.<\/p>\n\n\n

For each new id, Promtail calls inspect<\/strong> to learn whether the container has a TTY, then logs<\/strong>: stdout and stderr, follow<\/code>, timestamps on, and since<\/code> taken from the saved cursor. A missing cursor is since=0<\/code>. Docker then returns the retained log from the start.<\/p>\n\n\n

A container without a TTY sends a multiplexed stream. Each frame starts with an 8-byte header: the first byte is stdout or stderr, and the last four bytes are the frame length. A TTY container has no header. Promtail reads that stream as stdout.<\/p>\n\n\n

\n\n

What the socket is, and what Loki receives<\/strong><\/h4>\n\n

The socket is the engine HTTP API. Promtail’s own calls are list, inspect, and logs. A read-only bind mount does not remove the other methods: start, stop, and create are still requests on the same socket. The socket still has to stay on that host.<\/p>\n\n\n

Loki receives a push. It does not scrape Promtail. Promtail strips the Docker timestamp off the front of each line, uses that time as the entry time, and POSTs batches to \/loki\/api\/v1\/push<\/code>. The default body is snappy-compressed protobuf. Before that push, Promtail sets __meta_docker_container_log_stream<\/code> to stdout<\/code> or stderr<\/code> and runs relabel. Labels that still start with __<\/code> are dropped, so a relabel rule has to copy the stream onto stream<\/code> if you want to keep it.<\/p>\n\n\n

\n\n

The working path<\/strong><\/h4>\n\n
    \n
  1. docker_sd_configs.host<\/code> is unix:\/\/\/var\/run\/docker.sock<\/code>.<\/li>\n
  2. In relabel_configs<\/code>, map __meta_docker_container_name<\/code> with \/(.*)<\/code> onto container<\/code>, and copy __meta_docker_container_log_stream<\/code> onto stream<\/code>. Promtail does not keep the stream<\/code> label unless that copy is configured.<\/li>\n
  3. Promtail inspects the container id, then follows its logs from the saved cursor.<\/li>\n
  4. It demuxes non-TTY frames into stdout and stderr.<\/li>\n
  5. It stores cursor-<container id><\/code> as the Unix second of each shipped line. The id is the container id, not the name.<\/li>\n
  6. It POSTs to a reachable Loki URL, for example http:\/\/loki.example:3100\/loki\/api\/v1\/push<\/code>.<\/li>\n<\/ol>\n\n\n

    The cursor is whole seconds. After a restart, lines from that same second can be sent again. Stop Promtail before you write cursors into its positions file, or its flush overwrites them.<\/p>\n\n\n

    A process that logs only to a file, and never to stdout, is a different pipeline.<\/p>\n\n\n

    \n\n

    Approaches that look simpler and fail<\/strong><\/h4>\n\n
    \n\n\n\n\n\n\n\n\n\n
    Approach<\/th>\nWhy it fails<\/th>\n<\/tr>\n<\/thead>\n
    Tail Docker’s json log files on disk<\/td>\nThat file belongs to the json-file driver. Other drivers store logs elsewhere. The logs API is the same call for all of them.<\/td>\n<\/tr>\n
    Run docker logs -f<\/code> in a loop<\/td>\nNo cursor, no labels, and a restart repeats or skips lines<\/td>\n<\/tr>\n
    Publish the socket to the Loki host<\/td>\nThe far side gets the engine control API<\/td>\n<\/tr>\n
    Mount the socket read-only and treat that as a log-only API<\/td>\nThe read-only flag does not filter API methods<\/td>\n<\/tr>\n
    Start with an empty positions file<\/td>\nsince=0<\/code> ships the retained backlog<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/figure>\n\n\n

    A login in front of Loki protects queries. It does not replace keeping the Docker socket on the container host.<\/p>\n\n\n

    \n\n

    When you can drop Promtail<\/strong><\/h4>\n\n

    Drop Promtail when the reader already runs on the Docker host and can call the logs API itself. Also drop it when you only ship one known application file, and you do not need every container’s stdout.<\/p>\n\n\n

    Keep it when many containers on a host should appear in Loki and the socket must stay on that host. If Promtail cannot route to the push URL, fix that path first. This page assumes the URL is reachable.<\/p>\n\n\n

    \n\n

    Why this stays published<\/strong><\/h4>\n\n

    We hit this while sending container stdout from several hosts into one Loki. Promtail lists containers on the local Docker socket, follows the engine logs API, and pushes the lines. It does not open Docker’s log files, and the socket stays on that host.<\/strong><\/p>\n\n\n

    \u672c\u6587\u7531 HoHo \u8207 AI \u5354\u4f5c\u6574\u7406\uff0c\u6700\u5f8c\u66f4\u65b0\u65bc 2026 \u5e74 10 \u6708 3 \u65e5\u3002<\/p>\n","protected":false},"excerpt":{"rendered":"

    Short answer: Promtail does not open Docker’s log files. It connects to the local Docker socket, lists container ids, and calls the engine logs API with follow. It splits stdout from stderr, removes Docker’s timestamp prefix, and POSTs the lines to Loki at \/loki\/api\/v1\/push. This page assumes that push…<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[18,7,17],"tags":[15,14,13],"class_list":["post-39","post","type-post","status-publish","format-standard","hentry","category-docker","category-homelab","category-logging","tag-docker","tag-loki","tag-promtail"],"_links":{"self":[{"href":"https:\/\/hoho.live\/luda\/index.php\/wp-json\/wp\/v2\/posts\/39","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/hoho.live\/luda\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/hoho.live\/luda\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/hoho.live\/luda\/index.php\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/hoho.live\/luda\/index.php\/wp-json\/wp\/v2\/comments?post=39"}],"version-history":[{"count":2,"href":"https:\/\/hoho.live\/luda\/index.php\/wp-json\/wp\/v2\/posts\/39\/revisions"}],"predecessor-version":[{"id":51,"href":"https:\/\/hoho.live\/luda\/index.php\/wp-json\/wp\/v2\/posts\/39\/revisions\/51"}],"wp:attachment":[{"href":"https:\/\/hoho.live\/luda\/index.php\/wp-json\/wp\/v2\/media?parent=39"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/hoho.live\/luda\/index.php\/wp-json\/wp\/v2\/categories?post=39"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/hoho.live\/luda\/index.php\/wp-json\/wp\/v2\/tags?post=39"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}