{"id":1031,"date":"2026-06-26T14:04:17","date_gmt":"2026-06-26T14:04:17","guid":{"rendered":"https:\/\/www.devopsschool.com\/tutorials\/?p=1031"},"modified":"2026-06-26T14:04:19","modified_gmt":"2026-06-26T14:04:19","slug":"prometheus-step-by-step-guide-install-prometheus-on-kubernetes-using-helm-with-hostpath-pv-and-pvc","status":"publish","type":"post","link":"https:\/\/www.devopsschool.com\/tutorials\/prometheus-step-by-step-guide-install-prometheus-on-kubernetes-using-helm-with-hostpath-pv-and-pvc\/","title":{"rendered":"Prometheus – Step-by-Step Guide: Install Prometheus on Kubernetes Using Helm with hostPath PV and PVC"},"content":{"rendered":"\n

Prometheus is one of the most popular open-source monitoring systems for Kubernetes. It collects metrics from Kubernetes nodes, pods, services, and applications, and gives you a powerful query language called PromQL to analyze them.<\/p>\n\n\n\n

In this tutorial, we will install Prometheus in a Kubernetes cluster using the prometheus-community\/prometheus<\/code> Helm chart. We will also fix the common PVC pending issue by creating static hostPath<\/code> PersistentVolumes manually.<\/p>\n\n\n\n

This guide is useful for lab, demo, practice, and small Kubernetes clusters.<\/p>\n\n\n\n

\n\n\n\n

Architecture<\/h2>\n\n\n\n

We will install the following Prometheus components:<\/p>\n\n\n\n

Prometheus Server\nAlertmanager\nKube State Metrics\nNode Exporter\nPushgateway\n<\/code><\/pre>\n\n\n\n
Prometheus Server will store data under:<\/p>\n\n\n\n
\/mnt\/prometheus-server\n<\/code><\/pre>\n\n\n\n
Alertmanager will store data under:<\/p>\n\n\n\n
\/mnt\/alertmanager\n<\/code><\/pre>\n\n\n\n
These directories will be created on the Kubernetes worker node using hostPath<\/code>.<\/p>\n\n\n\n
\n\n\n\n
Important Note About hostPath<\/h2>\n\n\n\n
hostPath<\/code> storage is node-specific.<\/p>\n\n\n\n
That means data is stored directly on one Kubernetes node. If the pod moves to another node, the same data may not be available there.<\/p>\n\n\n\n
For production, use a proper storage solution such as:<\/p>\n\n\n\n
AWS EBS CSI Driver\nNFS\nLonghorn\nCeph\nOpenEBS\nCloud provider storage class\n<\/code><\/pre>\n\n\n\n
For learning and testing, hostPath<\/code> is fine.<\/p>\n\n\n\n
\n\n\n\n
Step 1: Check Kubernetes Nodes<\/h1>\n\n\n\n
First, check that your Kubernetes cluster is running.<\/p>\n\n\n\n
kubectl get nodes\n<\/code><\/pre>\n\n\n\n
Example output:<\/p>\n\n\n\n
NAME             STATUS   ROLES           AGE   VERSION\nip-172-31-5-107  Ready    control-plane   ...\nip-172-31-0-191  Ready    worker          ...\n<\/code><\/pre>\n\n\n\n
In this example, the Prometheus pod later runs on:<\/p>\n\n\n\n
ip-172-31-0-191\n<\/code><\/pre>\n\n\n\n
So the hostPath directories must be created on that node.<\/p>\n\n\n\n
\n\n\n\n
Step 2: Install Helm<\/h1>\n\n\n\n
Check if Helm is already installed:<\/p>\n\n\n\n
helm\n<\/code><\/pre>\n\n\n\n
If Helm is not installed, install it.<\/p>\n\n\n\n
curl -fsSL -o get_helm.sh https:\/\/raw.githubusercontent.com\/helm\/helm\/main\/scripts\/get-helm-4\nchmod 700 get_helm.sh\n.\/get_helm.sh\n<\/code><\/pre>\n\n\n\n
Verify Helm:<\/p>\n\n\n\n
helm version\n<\/code><\/pre>\n\n\n\n
\n\n\n\n
Step 3: Add Prometheus Helm Repository<\/h1>\n\n\n\n
Add the Prometheus Community Helm repository:<\/p>\n\n\n\n
helm repo add prometheus-community https:\/\/prometheus-community.github.io\/helm-charts\n<\/code><\/pre>\n\n\n\n
Update Helm repositories:<\/p>\n\n\n\n
helm repo update\n<\/code><\/pre>\n\n\n\n
Verify the repository:<\/p>\n\n\n\n
helm repo list\n<\/code><\/pre>\n\n\n\n
\n\n\n\n
Step 4: Install Prometheus Using Helm<\/h1>\n\n\n\n
Install Prometheus in the default<\/code> namespace:<\/p>\n\n\n\n
helm install my-prometheus prometheus-community\/prometheus --version 29.13.0\n<\/code><\/pre>\n\n\n\n
Check all pods:<\/p>\n\n\n\n
kubectl get pods\n<\/code><\/pre>\n\n\n\n
You may initially see something like this:<\/p>\n\n\n\n
NAME                                                    READY   STATUS    AGE\nmy-prometheus-alertmanager-0                            0\/1     Pending   ...\nmy-prometheus-server-67b68bd4c6-wv79t                   0\/2     Pending   ...\nmy-prometheus-kube-state-metrics-7c4457f7d6-j7c8f       1\/1     Running   ...\nmy-prometheus-prometheus-node-exporter-5gs22            1\/1     Running   ...\nmy-prometheus-prometheus-node-exporter-kzqz9            1\/1     Running   ...\nmy-prometheus-prometheus-pushgateway-78fc6d6f89-khpfd   1\/1     Running   ...\n<\/code><\/pre>\n\n\n\n
If Prometheus Server and Alertmanager are pending, it is usually because their PVCs are waiting for storage.<\/p>\n\n\n\n
\n\n\n\n
Step 5: Check PVC Status<\/h1>\n\n\n\n
Run:<\/p>\n\n\n\n
kubectl get pvc\n<\/code><\/pre>\n\n\n\n
Example output:<\/p>\n\n\n\n
NAME                                   STATUS    VOLUME   CAPACITY   ACCESS MODES   STORAGECLASS   AGE\nmy-prometheus-server                   Pending                                      <unset>        6m\nstorage-my-prometheus-alertmanager-0   Pending                                      <unset>        6m\n<\/code><\/pre>\n\n\n\n
This means Kubernetes created PVCs, but there are no matching PVs available.<\/p>\n\n\n\n
\n\n\n\n
Step 6: Describe the Prometheus Pod<\/h1>\n\n\n\n
Check the Prometheus Server pod:<\/p>\n\n\n\n
kubectl describe pod my-prometheus-server-67b68bd4c6-wv79t\n<\/code><\/pre>\n\n\n\n
You may see this error:<\/p>\n\n\n\n
Warning  FailedScheduling  default-scheduler  0\/2 nodes are available: pod has unbound immediate PersistentVolumeClaims.\n<\/code><\/pre>\n\n\n\n
This means Prometheus cannot start because its PVC is not bound.<\/p>\n\n\n\n
\n\n\n\n
Step 7: Create PersistentVolume for Prometheus Server<\/h1>\n\n\n\n
Create a file:<\/p>\n\n\n\n
cat > prometheus-server-pv.yaml <<'EOF'\napiVersion: v1\nkind: PersistentVolume\nmetadata:\n  name: prometheus-server-pv\nspec:\n  capacity:\n    storage: 8Gi\n  accessModes:\n    - ReadWriteOnce\n  persistentVolumeReclaimPolicy: Retain\n  storageClassName: \"\"\n  claimRef:\n    namespace: default\n    name: my-prometheus-server\n  hostPath:\n    path: \/mnt\/prometheus-server\n    type: DirectoryOrCreate\nEOF\n<\/code><\/pre>\n\n\n\n
Apply it:<\/p>\n\n\n\n
kubectl apply -f prometheus-server-pv.yaml\n<\/code><\/pre>\n\n\n\n
\n\n\n\n
Step 8: Create PersistentVolume for Alertmanager<\/h1>\n\n\n\n
Create another PV:<\/p>\n\n\n\n
cat > alertmanager-pv.yaml <<'EOF'\napiVersion: v1\nkind: PersistentVolume\nmetadata:\n  name: alertmanager-pv\nspec:\n  capacity:\n    storage: 2Gi\n  accessModes:\n    - ReadWriteOnce\n  persistentVolumeReclaimPolicy: Retain\n  storageClassName: \"\"\n  claimRef:\n    namespace: default\n    name: storage-my-prometheus-alertmanager-0\n  hostPath:\n    path: \/mnt\/alertmanager\n    type: DirectoryOrCreate\nEOF\n<\/code><\/pre>\n\n\n\n
Apply it:<\/p>\n\n\n\n
kubectl apply -f alertmanager-pv.yaml\n<\/code><\/pre>\n\n\n\n
\n\n\n\n
Step 9: Verify PV and PVC Binding<\/h1>\n\n\n\n
Check PVs:<\/p>\n\n\n\n
kubectl get pv\n<\/code><\/pre>\n\n\n\n
Expected output:<\/p>\n\n\n\n
NAME                   CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM\nprometheus-server-pv   8Gi        RWO            Retain           Bound    default\/my-prometheus-server\nalertmanager-pv        2Gi        RWO            Retain           Bound    default\/storage-my-prometheus-alertmanager-0\n<\/code><\/pre>\n\n\n\n
Check PVCs:<\/p>\n\n\n\n
kubectl get pvc\n<\/code><\/pre>\n\n\n\n
Expected output:<\/p>\n\n\n\n
NAME                                   STATUS   VOLUME                 CAPACITY   ACCESS MODES\nmy-prometheus-server                   Bound    prometheus-server-pv   8Gi        RWO\nstorage-my-prometheus-alertmanager-0   Bound    alertmanager-pv        2Gi        RWO\n<\/code><\/pre>\n\n\n\n
At this point, the storage issue is fixed.<\/p>\n\n\n\n
\n\n\n\n
Step 10: Create hostPath Directories on Worker Node<\/h1>\n\n\n\n
Now create the required directories on the node where Prometheus is scheduled.<\/p>\n\n\n\n
First, check where the pod is running:<\/p>\n\n\n\n
kubectl get pod -o wide\n<\/code><\/pre>\n\n\n\n
Example:<\/p>\n\n\n\n
my-prometheus-server-67b68bd4c6-wv79t   1\/2   Running   ip-172-31-0-191\n<\/code><\/pre>\n\n\n\n
SSH into that worker node:<\/p>\n\n\n\n
ssh ubuntu@ip-172-31-0-191\n<\/code><\/pre>\n\n\n\n
Create directories:<\/p>\n\n\n\n
sudo mkdir -p \/mnt\/prometheus-server\nsudo mkdir -p \/mnt\/alertmanager\nsudo mkdir -p \/mnt\/data\n<\/code><\/pre>\n\n\n\n
Set ownership:<\/p>\n\n\n\n
sudo chown -R 65534:65534 \/mnt\/prometheus-server\nsudo chown -R 65534:65534 \/mnt\/alertmanager\nsudo chown -R 65534:65534 \/mnt\/data\n<\/code><\/pre>\n\n\n\n
Set permissions:<\/p>\n\n\n\n
sudo chmod -R 775 \/mnt\/prometheus-server\nsudo chmod -R 775 \/mnt\/alertmanager\nsudo chmod -R 775 \/mnt\/data\n<\/code><\/pre>\n\n\n\n
Why 65534:65534<\/code>?<\/p>\n\n\n\n
Prometheus commonly runs as a non-root user inside the container. If the hostPath directory is owned by root and not writable, Prometheus may crash with a permission error.<\/p>\n\n\n\n
\n\n\n\n
Step 11: Check Prometheus Pods Again<\/h1>\n\n\n\n
Run:<\/p>\n\n\n\n
kubectl get pods\n<\/code><\/pre>\n\n\n\n
Example output:<\/p>\n\n\n\n
NAME                                                    READY   STATUS    RESTARTS   AGE\nmy-prometheus-alertmanager-0                            1\/1     Running   0          13m\nmy-prometheus-kube-state-metrics-7c4457f7d6-j7c8f       1\/1     Running   0          13m\nmy-prometheus-prometheus-node-exporter-5gs22            1\/1     Running   0          13m\nmy-prometheus-prometheus-node-exporter-kzqz9            1\/1     Running   0          13m\nmy-prometheus-prometheus-pushgateway-78fc6d6f89-khpfd   1\/1     Running   0          13m\nmy-prometheus-server-67b68bd4c6-b7qn2                   2\/2     Running   0          49s\n<\/code><\/pre>\n\n\n\n
This means Prometheus is now successfully running.<\/p>\n\n\n\n
\n\n\n\n
Step 12: Troubleshoot CrashLoopBackOff<\/h1>\n\n\n\n
If Prometheus Server shows:<\/p>\n\n\n\n
CrashLoopBackOff\n<\/code><\/pre>\n\n\n\n
Check the pod:<\/p>\n\n\n\n
kubectl describe pod my-prometheus-server-67b68bd4c6-wv79t\n<\/code><\/pre>\n\n\n\n
Check logs of the correct container:<\/p>\n\n\n\n
kubectl logs my-prometheus-server-67b68bd4c6-wv79t -c prometheus-server\n<\/code><\/pre>\n\n\n\n
If the pod restarted already, check previous logs:<\/p>\n\n\n\n
kubectl logs my-prometheus-server-67b68bd4c6-wv79t -c prometheus-server --previous\n<\/code><\/pre>\n\n\n\n
A common issue is permission denied on \/data<\/code>.<\/p>\n\n\n\n
Fix it on the worker node:<\/p>\n\n\n\n
sudo mkdir -p \/mnt\/prometheus-server\nsudo chown -R 65534:65534 \/mnt\/prometheus-server\nsudo chmod -R 775 \/mnt\/prometheus-server\n<\/code><\/pre>\n\n\n\n
Then delete the pod so Kubernetes recreates it:<\/p>\n\n\n\n
kubectl delete pod my-prometheus-server-67b68bd4c6-wv79t\n<\/code><\/pre>\n\n\n\n
Check again:<\/p>\n\n\n\n
kubectl get pods\n<\/code><\/pre>\n\n\n\n
\n\n\n\n
Step 13: Access Prometheus Using Port Forward<\/h1>\n\n\n\n
You can expose Prometheus locally using kubectl port-forward<\/code>.<\/p>\n\n\n\n
First, check the Prometheus Server pod name:<\/p>\n\n\n\n
kubectl get pods\n<\/code><\/pre>\n\n\n\n
Example pod name:<\/p>\n\n\n\n
my-prometheus-server-67b68bd4c6-b7qn2\n<\/code><\/pre>\n\n\n\n
Run port-forward:<\/p>\n\n\n\n
kubectl --namespace default port-forward pod\/my-prometheus-server-67b68bd4c6-b7qn2 9090:9090\n<\/code><\/pre>\n\n\n\n
Now open:<\/p>\n\n\n\n
http://localhost:9090\n<\/code><\/pre>\n\n\n\n
\n\n\n\n
Step 14: Access Prometheus from Anywhere Using 0.0.0.0<\/h1>\n\n\n\n
If you want Prometheus to be accessible using your server public IP, use:<\/p>\n\n\n\n
kubectl --namespace default port-forward --address 0.0.0.0 pod\/my-prometheus-server-67b68bd4c6-b7qn2 9090:9090\n<\/code><\/pre>\n\n\n\n
Now open:<\/p>\n\n\n\n
http://<server-public-ip>:9090\n<\/code><\/pre>\n\n\n\n
Example:<\/p>\n\n\n\n
http://13.XX.XX.XX:9090\n<\/code><\/pre>\n\n\n\n
If you are running on AWS EC2, make sure port 9090<\/code> is allowed in the Security Group.<\/p>\n\n\n\n
Allow inbound:<\/p>\n\n\n\n
Type: Custom TCP\nPort: 9090\nSource: Your IP address\n<\/code><\/pre>\n\n\n\n
For testing, you may use 0.0.0.0\/0<\/code>, but it is not recommended because Prometheus does not have authentication by default.<\/p>\n\n\n\n
\n\n\n\n
Step 15: Access Prometheus Using Service Port Forward<\/h1>\n\n\n\n
Instead of forwarding directly to a pod, it is better to forward to the service.<\/p>\n\n\n\n
Check services:<\/p>\n\n\n\n
kubectl get svc\n<\/code><\/pre>\n\n\n\n
You should see something like:<\/p>\n\n\n\n
my-prometheus-server   ClusterIP   ...   80\/TCP\n<\/code><\/pre>\n\n\n\n
Forward service port 80<\/code> to local port 9090<\/code>:<\/p>\n\n\n\n
kubectl --namespace default port-forward --address 0.0.0.0 svc\/my-prometheus-server 9090:80\n<\/code><\/pre>\n\n\n\n
Now access:<\/p>\n\n\n\n
http://<server-public-ip>:9090\n<\/code><\/pre>\n\n\n\n
This method is better than pod port-forwarding because pod names change when pods restart.<\/p>\n\n\n\n
\n\n\n\n
Step 16: Verify Prometheus UI<\/h1>\n\n\n\n
Open Prometheus in your browser:<\/p>\n\n\n\n
http://<server-public-ip>:9090\n<\/code><\/pre>\n\n\n\n
Go to:<\/p>\n\n\n\n
Status -> Targets\n<\/code><\/pre>\n\n\n\n
You should see targets such as:<\/p>\n\n\n\n
kubernetes-apiservers\nkubernetes-nodes\nkubernetes-nodes-cadvisor\nkubernetes-service-endpoints\nprometheus\nnode-exporter\nkube-state-metrics\npushgateway\n<\/code><\/pre>\n\n\n\n
Healthy targets should show:<\/p>\n\n\n\n
UP\n<\/code><\/pre>\n\n\n\n
\n\n\n\n
Step 17: Run Basic Prometheus Queries<\/h1>\n\n\n\n
In the Prometheus UI, go to the query box and try these queries.<\/p>\n\n\n\n
Check all up targets:<\/p>\n\n\n\n
up\n<\/code><\/pre>\n\n\n\n
Check Prometheus server health:<\/p>\n\n\n\n
prometheus_build_info\n<\/code><\/pre>\n\n\n\n
Check Kubernetes node CPU metrics:<\/p>\n\n\n\n
node_cpu_seconds_total\n<\/code><\/pre>\n\n\n\n
Check memory available:<\/p>\n\n\n\n
node_memory_MemAvailable_bytes\n<\/code><\/pre>\n\n\n\n
Check filesystem size:<\/p>\n\n\n\n
node_filesystem_size_bytes\n<\/code><\/pre>\n\n\n\n
Check pod metrics:<\/p>\n\n\n\n
kube_pod_info\n<\/code><\/pre>\n\n\n\n
Check deployment replicas:<\/p>\n\n\n\n
kube_deployment_status_replicas\n<\/code><\/pre>\n\n\n\n
\n\n\n\n
Step 18: Useful Kubernetes Verification Commands<\/h1>\n\n\n\n
Check pods:<\/p>\n\n\n\n
kubectl get pods\n<\/code><\/pre>\n\n\n\n
Check pods with node details:<\/p>\n\n\n\n
kubectl get pods -o wide\n<\/code><\/pre>\n\n\n\n
Check services:<\/p>\n\n\n\n
kubectl get svc\n<\/code><\/pre>\n\n\n\n
Check PV:<\/p>\n\n\n\n
kubectl get pv\n<\/code><\/pre>\n\n\n\n
Check PVC:<\/p>\n\n\n\n
kubectl get pvc\n<\/code><\/pre>\n\n\n\n
Describe Prometheus Server pod:<\/p>\n\n\n\n
kubectl describe pod <prometheus-server-pod-name>\n<\/code><\/pre>\n\n\n\n
Check Prometheus Server logs:<\/p>\n\n\n\n
kubectl logs <prometheus-server-pod-name> -c prometheus-server\n<\/code><\/pre>\n\n\n\n
Check Config Reloader logs:<\/p>\n\n\n\n
kubectl logs <prometheus-server-pod-name> -c prometheus-server-configmap-reload\n<\/code><\/pre>\n\n\n\n
Check Helm release:<\/p>\n\n\n\n
helm list\n<\/code><\/pre>\n\n\n\n
Check Helm status:<\/p>\n\n\n\n
helm status my-prometheus\n<\/code><\/pre>\n\n\n\n
\n\n\n\n
Step 19: Optional \u2014 Create Node Exporter Dashboard in Grafana<\/h1>\n\n\n\n
Prometheus collects metrics, but for dashboards, you usually install Grafana.<\/p>\n\n\n\n
You can later install Grafana using Helm:<\/p>\n\n\n\n
helm repo add grafana https:\/\/grafana.github.io\/helm-charts\nhelm repo update\nhelm install my-grafana grafana\/grafana\n<\/code><\/pre>\n\n\n\n
Then connect Grafana to Prometheus using this URL inside Kubernetes:<\/p>\n\n\n\n
http://my-prometheus-server.default.svc.cluster.local\n<\/code><\/pre>\n\n\n\n
\n\n\n\n
Step 20: Cleanup Commands<\/h1>\n\n\n\n
If you want to remove Prometheus:<\/p>\n\n\n\n
helm uninstall my-prometheus\n<\/code><\/pre>\n\n\n\n
Delete PVs:<\/p>\n\n\n\n
kubectl delete pv prometheus-server-pv\nkubectl delete pv alertmanager-pv\n<\/code><\/pre>\n\n\n\n
Delete local data from worker node carefully:<\/p>\n\n\n\n
sudo rm -rf \/mnt\/prometheus-server\nsudo rm -rf \/mnt\/alertmanager\n<\/code><\/pre>\n\n\n\n
Only run the above commands if you really want to delete Prometheus data.<\/p>\n\n\n\n
\n\n\n\n
Common Issues and Fixes<\/h1>\n\n\n\n
Issue 1: PVC Pending<\/h2>\n\n\n\n
Error:<\/p>\n\n\n\n
pod has unbound immediate PersistentVolumeClaims\n<\/code><\/pre>\n\n\n\n
Check PVC:<\/p>\n\n\n\n
kubectl get pvc\n<\/code><\/pre>\n\n\n\n
Fix:<\/p>\n\n\n\n
Create matching PVs for the PVCs:<\/p>\n\n\n\n
kubectl apply -f prometheus-server-pv.yaml\nkubectl apply -f alertmanager-pv.yaml\n<\/code><\/pre>\n\n\n\n
\n\n\n\n
Issue 2: Prometheus Server CrashLoopBackOff<\/h2>\n\n\n\n
Check logs:<\/p>\n\n\n\n
kubectl logs <prometheus-server-pod-name> -c prometheus-server\n<\/code><\/pre>\n\n\n\n
Possible reason:<\/p>\n\n\n\n
permission denied\n<\/code><\/pre>\n\n\n\n
Fix permissions on worker node:<\/p>\n\n\n\n
sudo chown -R 65534:65534 \/mnt\/prometheus-server\nsudo chmod -R 775 \/mnt\/prometheus-server\n<\/code><\/pre>\n\n\n\n
Restart pod:<\/p>\n\n\n\n
kubectl delete pod <prometheus-server-pod-name>\n<\/code><\/pre>\n\n\n\n
\n\n\n\n
Issue 3: Port Forward Works Locally but Not from Browser<\/h2>\n\n\n\n
Use:<\/p>\n\n\n\n
kubectl --namespace default port-forward --address 0.0.0.0 svc\/my-prometheus-server 9090:80\n<\/code><\/pre>\n\n\n\n
Then open:<\/p>\n\n\n\n
http://<server-public-ip>:9090\n<\/code><\/pre>\n\n\n\n
Also check cloud firewall or AWS Security Group.<\/p>\n\n\n\n
\n\n\n\n
Issue 4: Wrong Container Logs<\/h2>\n\n\n\n
If you run:<\/p>\n\n\n\n
kubectl logs my-prometheus-server-67b68bd4c6-wv79t\n<\/code><\/pre>\n\n\n\n
Kubernetes may show logs from the default container:<\/p>\n\n\n\n
prometheus-server-configmap-reload\n<\/code><\/pre>\n\n\n\n
To check actual Prometheus logs, run:<\/p>\n\n\n\n
kubectl logs my-prometheus-server-67b68bd4c6-wv79t -c prometheus-server\n<\/code><\/pre>\n\n\n\n
\n\n\n\n
Final Working Commands Summary<\/h1>\n\n\n\n
kubectl get nodes\n\ncurl -fsSL -o get_helm.sh https:\/\/raw.githubusercontent.com\/helm\/helm\/main\/scripts\/get-helm-4\nchmod 700 get_helm.sh\n.\/get_helm.sh\n\nhelm repo add prometheus-community https:\/\/prometheus-community.github.io\/helm-charts\nhelm repo update\n\nhelm install my-prometheus prometheus-community\/prometheus --version 29.13.0\n\nkubectl get pods\nkubectl get pvc\nkubectl get pv\n<\/code><\/pre>\n\n\n\n
Create Prometheus PV:<\/p>\n\n\n\n
cat > prometheus-server-pv.yaml <<'EOF'\napiVersion: v1\nkind: PersistentVolume\nmetadata:\n  name: prometheus-server-pv\nspec:\n  capacity:\n    storage: 8Gi\n  accessModes:\n    - ReadWriteOnce\n  persistentVolumeReclaimPolicy: Retain\n  storageClassName: \"\"\n  claimRef:\n    namespace: default\n    name: my-prometheus-server\n  hostPath:\n    path: \/mnt\/prometheus-server\n    type: DirectoryOrCreate\nEOF\n\nkubectl apply -f prometheus-server-pv.yaml\n<\/code><\/pre>\n\n\n\n
Create Alertmanager PV:<\/p>\n\n\n\n
cat > alertmanager-pv.yaml <<'EOF'\napiVersion: v1\nkind: PersistentVolume\nmetadata:\n  name: alertmanager-pv\nspec:\n  capacity:\n    storage: 2Gi\n  accessModes:\n    - ReadWriteOnce\n  persistentVolumeReclaimPolicy: Retain\n  storageClassName: \"\"\n  claimRef:\n    namespace: default\n    name: storage-my-prometheus-alertmanager-0\n  hostPath:\n    path: \/mnt\/alertmanager\n    type: DirectoryOrCreate\nEOF\n\nkubectl apply -f alertmanager-pv.yaml\n<\/code><\/pre>\n\n\n\n
Create directories on worker node:<\/p>\n\n\n\n
sudo mkdir -p \/mnt\/prometheus-server \/mnt\/alertmanager \/mnt\/data\n\nsudo chown -R 65534:65534 \/mnt\/prometheus-server \/mnt\/alertmanager \/mnt\/data\nsudo chmod -R 775 \/mnt\/prometheus-server \/mnt\/alertmanager \/mnt\/data\n<\/code><\/pre>\n\n\n\n
Restart Prometheus pod if needed:<\/p>\n\n\n\n
kubectl delete pod <prometheus-server-pod-name>\n<\/code><\/pre>\n\n\n\n
Verify:<\/p>\n\n\n\n
kubectl get pods\nkubectl get pv,pvc\n<\/code><\/pre>\n\n\n\n
Expose Prometheus:<\/p>\n\n\n\n
kubectl --namespace default port-forward --address 0.0.0.0 svc\/my-prometheus-server 9090:80\n<\/code><\/pre>\n\n\n\n
Access:<\/p>\n\n\n\n
http://<server-public-ip>:9090\n<\/code><\/pre>\n\n\n\n
\n\n\n\n
Conclusion<\/h1>\n\n\n\n
In this tutorial, we installed Prometheus on Kubernetes using Helm. We also fixed the most common storage issue where Prometheus pods remain pending because PVCs are not bound.<\/p>\n\n\n\n
We manually created hostPath<\/code> PersistentVolumes for Prometheus Server and Alertmanager, created the required host directories on the worker node, fixed permissions, verified all pods, and finally accessed Prometheus from the browser.<\/p>\n\n\n\n
This setup is good for learning and practice. For production, replace hostPath<\/code> with a proper dynamic storage provisioner such as AWS EBS CSI Driver or another production-grade storage backend.<\/p>\n","protected":false},"excerpt":{"rendered":"
Prometheus is one of the most popular open-source monitoring systems for Kubernetes. It collects metrics from Kubernetes nodes, pods, services, and applications, and gives you a powerful… <\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-1031","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"_links":{"self":[{"href":"https:\/\/www.devopsschool.com\/tutorials\/wp-json\/wp\/v2\/posts\/1031","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.devopsschool.com\/tutorials\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.devopsschool.com\/tutorials\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.devopsschool.com\/tutorials\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.devopsschool.com\/tutorials\/wp-json\/wp\/v2\/comments?post=1031"}],"version-history":[{"count":1,"href":"https:\/\/www.devopsschool.com\/tutorials\/wp-json\/wp\/v2\/posts\/1031\/revisions"}],"predecessor-version":[{"id":1032,"href":"https:\/\/www.devopsschool.com\/tutorials\/wp-json\/wp\/v2\/posts\/1031\/revisions\/1032"}],"wp:attachment":[{"href":"https:\/\/www.devopsschool.com\/tutorials\/wp-json\/wp\/v2\/media?parent=1031"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.devopsschool.com\/tutorials\/wp-json\/wp\/v2\/categories?post=1031"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.devopsschool.com\/tutorials\/wp-json\/wp\/v2\/tags?post=1031"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}