{"id":48135,"date":"2025-01-12T07:20:55","date_gmt":"2025-01-12T07:20:55","guid":{"rendered":"https:\/\/www.devopsschool.com\/blog\/?p=48135"},"modified":"2026-02-21T07:25:00","modified_gmt":"2026-02-21T07:25:00","slug":"deep-dive-troubleshoot-using-kubernetes-logs-its-structure-location","status":"publish","type":"post","link":"https:\/\/www.devopsschool.com\/blog\/deep-dive-troubleshoot-using-kubernetes-logs-its-structure-location\/","title":{"rendered":"Deep Dive & Troubleshoot using Kubernetes logs & its Structure & Location"},"content":{"rendered":"\n

Here is a detailed table summarizing the log file locations, purposes, and use cases<\/strong> for Kubernetes logs:<\/p>\n\n\n\n

\n\n\n\n

Comprehensive Table of Kubernetes Logs<\/strong><\/h3>\n\n\n\n
Log Directory\/Location<\/strong><\/th>Log File(s)<\/strong><\/th>Purpose<\/strong><\/th>Use Case<\/strong><\/th><\/tr><\/thead>
\/var\/log\/containers\/<\/strong><\/td><pod-name>_<namespace>_<container-name>-<container-id>.log<\/code><\/td>Logs for each container in the pods.<\/td>Debugging container-specific issues.<\/td><\/tr>
\/var\/log\/pods\/<\/strong><\/td><namespace>_<pod-name>_<uid>\/<container-name>-stdout.log<\/code><namespace>_<pod-name>_<uid>\/<container-name>-stderr.log<\/code><\/td>stdout.log<\/code>: Logs standard output.stderr.log<\/code>: Logs standard error.<\/td>Debugging container logs specific to pods.<\/td><\/tr>
\/var\/log\/kubelet\/<\/strong><\/td>kubelet.log<\/code>audit.log<\/code>error.log<\/code><\/td>kubelet.log<\/code>: Node-level kubelet events.audit.log<\/code>: Logs kubelet actions.error.log<\/code>: Kubelet error logs.<\/td>Diagnosing issues with pod scheduling and node management.<\/td><\/tr>
\/var\/log\/kube-apiserver\/<\/strong><\/td>apiserver.log<\/code>audit.log<\/code>error.log<\/code><\/td>apiserver.log<\/code>: API server events.audit.log<\/code>: API request trails.error.log<\/code>: API server errors.<\/td>Monitoring API requests, failures, and unauthorized access.<\/td><\/tr>
\/var\/log\/kube-scheduler\/<\/strong><\/td>scheduler.log<\/code>error.log<\/code><\/td>scheduler.log<\/code>: Logs pod placement decisions.error.log<\/code>: Scheduler-related errors.<\/td>Debugging pod scheduling failures and resource allocation.<\/td><\/tr>
\/var\/log\/kube-controller-manager\/<\/strong><\/td>controller-manager.log<\/code>error.log<\/code><\/td>controller-manager.log<\/code>: Logs controller activities.error.log<\/code>: Errors in controller manager.<\/td>Troubleshooting replication, scaling, and resource management.<\/td><\/tr>
\/var\/log\/etcd\/<\/strong><\/td>etcd.log<\/code>snapshot.log<\/code>error.log<\/code><\/td>etcd.log<\/code>: Main etcd datastore logs.snapshot.log<\/code>: Logs for etcd snapshots.error.log<\/code>: Logs etcd errors.<\/td>Debugging cluster state synchronization and data corruption issues.<\/td><\/tr>
\/var\/log\/containerd\/<\/strong><\/td>containerd.log<\/code>error.log<\/code><\/td>containerd.log<\/code>: Logs container runtime events.error.log<\/code>: Errors related to container runtime.<\/td>Debugging container runtime (image pulling, container start).<\/td><\/tr>
\/var\/log\/network\/<\/strong><\/td>cni.log<\/code>flannel.log<\/code>calico.log<\/code>error.log<\/code><\/td>cni.log<\/code>: Container Network Interface logs.flannel\/calico.log<\/code>: Specific network provider logs.error.log<\/code>: Network errors.<\/td>Troubleshooting pod networking and connectivity issues.<\/td><\/tr>
\/var\/log\/node\/<\/strong><\/td>syslog<\/code>messages<\/code>dmesg.log<\/code>auth.log<\/code>error.log<\/code><\/td>syslog\/messages<\/code>: General system logs.dmesg.log<\/code>: Hardware\/boot info.auth.log<\/code>: SSH and sudo authentication logs.error.log<\/code>: Node-level error logs.<\/td>Debugging hardware, authentication, and boot issues.<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n
\n\n\n\n

Detailed Explanation of Logs<\/strong><\/h3>\n\n\n\n
Log Type<\/strong><\/th>Component\/Service<\/strong><\/th>Key Focus<\/strong><\/th><\/tr><\/thead>
Container Logs<\/strong><\/td>Individual Pods\/Containers<\/td>Application-level logs (stdout and stderr).<\/td><\/tr>
Node Logs<\/strong><\/td>Kubelet and System Logs<\/td>Logs at the node level for resource management.<\/td><\/tr>
Control Plane Logs<\/strong><\/td>Scheduler, Controller Manager<\/td>Logs for cluster-level management and decisions.<\/td><\/tr>
API Server Logs<\/strong><\/td>API Server<\/td>Tracks API requests, errors, and audit trails.<\/td><\/tr>
Networking Logs<\/strong><\/td>CNI Plugins (Flannel, Calico)<\/td>Network-related logs and troubleshooting.<\/td><\/tr>
Datastore Logs<\/strong><\/td>etcd<\/td>Logs related to cluster state storage and retrieval.<\/td><\/tr>
Runtime Logs<\/strong><\/td>containerd<\/td>Logs for container lifecycle management.<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n
\n\n\n\n

This table provides a clear breakdown of each log directory, its log files, purposes, and common use cases, making it easy for administrators to locate and analyze logs effectively in a Kubernetes environment.<\/p>\n\n\n\n

Kubernetes Logging Directory Structure Tutorial<\/h2>\n\n\n\n

Root Directory (\/var\/log\/)<\/h3>\n\n\n\n

The main logging directory in Kubernetes contains several important subdirectories:<\/p>\n\n\n\n

1. Container Logs<\/h3>\n\n\n
\/var<\/span>\/log\/containers\/<\/code><\/span>Code language:<\/span> JavaScript<\/span> (<\/span>javascript<\/span>)<\/span><\/small><\/pre>\n\n\n