Introduction of Prometheus Operator

$ kubectl api-resources | grep monitoring
alertmanagerconfigs                                  monitoring.coreos.com/v1alpha1                 true         AlertmanagerConfig
alertmanagers                                        monitoring.coreos.com/v1                       true         Alertmanager
podmonitors                                          monitoring.coreos.com/v1                       true         PodMonitor
probes                                               monitoring.coreos.com/v1                       true         Probe
prometheuses                                         monitoring.coreos.com/v1                       true         Prometheus
prometheusrules                                      monitoring.coreos.com/v1                       true         PrometheusRule
servicemonitors                                      monitoring.coreos.com/v1                       true         ServiceMonitor
thanosrulers                                         monitoring.coreos.com/v1                       true         ThanosRuler
Code language: JavaScript (javascript)

The list is of Kubernetes Custom Resource Definitions (CRDs) related to the Prometheus Operator provided by CoreOS. The Prometheus Operator simplifies the deployment and operation of Prometheus, Alertmanager, and related monitoring components in Kubernetes environments.

Here’s a brief overview of each CRD:

1. AlertmanagerConfig (v1alpha1): This CRD allows for defining alert routing and receivers for Alertmanager in a more modular fashion.
2. Alertmanager (v1): Represents an Alertmanager cluster. You can define the version, replicas, and other specifications for running Alertmanager in a cluster.
3. PodMonitor (v1): Allows you to define how Prometheus should monitor specific pods based on labels and annotations.
4. Probe (v1): Defines how Prometheus should perform “blackbox” probes using the Blackbox Exporter.
5. Prometheus (v1): Represents a Prometheus instance in your cluster. You can define specifications like the version of Prometheus, storage configurations, etc.
6. PrometheusRule (v1): Represents a set of Prometheus alerting and/or recording rules.
7. ServiceMonitor (v1): A higher-level way to define how Prometheus instances should monitor Kubernetes services. Rather than writing raw Prometheus scrape configs, you can use a ServiceMonitor to automatically generate them based on service labels and annotations.
8. ThanosRuler (v1): Represents a Thanos Ruler instance, which can evaluate recording and alerting rules against Thanos Query data, especially useful in multi-cluster Thanos deployments.

These CRDs allow Kubernetes administrators and users to define monitoring configurations natively in Kubernetes manifests. Once these resources are applied, the Prometheus Operator ensures that the actual services (Prometheus, Alertmanager, etc.) are correctly configured and running as expected.

What is ServiceMonitor?

ServiceMonitor is a Custom Resource Definition (CRD) introduced by the Prometheus Operator, which simplifies the deployment and configuration of Prometheus on Kubernetes. It’s designed to help in automatically discovering and scraping metrics from Kubernetes services without having to manually write and update Prometheus scrape configurations.

You do not need Prometheus to scrape your service by modifying prometheus.yaml file.

ServiceMonitor  is A higher-level way to define how Prometheus instances should monitor Kubernetes services. Rather than writing raw Prometheus scrape configs, you can use a ServiceMonitor to automatically generate them based on service labels and annotations.

How ServiceMonitor works:

1. Service Discovery:
   • ServiceMonitor works by selecting Kubernetes services based on label selectors. This allows Prometheus to automatically discover the services and start scraping metrics without explicit configurations.
2. Configuration:
   • Once a Kubernetes service is selected, ServiceMonitor provides details on how to scrape metrics from the service’s pods. This includes the port, path (/metrics by default), interval, and other scrape configurations.
3. Integration with Prometheus:
   • When you create a Prometheus custom resource, you specify which ServiceMonitors it should look at, usually by specifying label selectors. This Prometheus instance will then automatically generate appropriate scrape configs based on the ServiceMonitors it’s observing.

ServiceMonitor works in the following way:

1. When a ServiceMonitor is created, the Prometheus Operator adds the ServiceMonitor configuration to the Prometheus scrape configuration.
2. Prometheus starts scraping the service endpoints specified in the ServiceMonitor configuration.
3. If the Prometheus Operator detects that the service has changed, it updates the Prometheus scrape configuration accordingly.
4. Prometheus continues to scrape the service endpoints specified in the Prometheus scrape configuration.

Rajesh Kumar

I’m a DevOps/SRE/DevSecOps/Cloud Expert passionate about sharing knowledge and experiences. I have worked at Cotocus. I share tech blog at DevOps School, travel stories at Holiday Landmark, stock market tips at Stocks Mantra, health and fitness guidance at My Medic Plus, product reviews at TrueReviewNow , and SEO strategies at Wizbrand.

Do you want to learn Quantum Computing?

Please find my social handles as below;

Rajesh Kumar Personal Website
Rajesh Kumar at YOUTUBE
Rajesh Kumar at INSTAGRAM
Rajesh Kumar at X
Rajesh Kumar at FACEBOOK
Rajesh Kumar at LINKEDIN
Rajesh Kumar at WIZBRAND

Rajesh Kumar DailyLogs