Creating a Simple Custom Prometheus Exporter in Python

1. Introduction

Prometheus is an open-source systems monitoring and alerting toolkit. Prometheus fundamentally stores all data as time series: streams of timestamped values belonging to the same metric and the same set of labeled dimensions. Besides stored time series, Prometheus may generate temporary derived time series as the result of queries.

Exporting metrics exposes quantifiable elements of a system periodically to be consumed by visualization engines. Exporting existing metrics from third-party systems as Prometheus metrics is useful for cases where it is not feasible to instrument a given system with Prometheus metrics directly.

2. Prerequisites

• Basic knowledge of Python.
• Prometheus setup (brief overview). The most basic way to describe it is periodically publishing text in a format that Prometheus understands (exporting) and then the Prometheus collects and stores that information in a database
• Python 3.x and necessary libraries.
• Install prometheus_client library (pip install prometheus_client).

3. Setting Up the Project

Create a new Python file (e.g., my_exporter.py).

Briefly describe the necessary imports:

from prometheus_client import start_http_server, Gauge
import time  # for simulating metrics over time

4. Define Custom Metrics

Introduction to Prometheus metric types (Counter, Gauge, Histogram, Summary).

• Counter: A counter is a cumulative metric that represents a single monotonically increasing counter whose value can only increase or be reset to zero on restart.
• Gauge: A gauge is a metric that represents a single numerical value that can arbitrarily go up and down.
• Histogram: A histogram (statistics) samples observations and counts them in configurable buckets.
• Summary: A summary samples observations, but also provides a total count of observations, a sum of all observed values, and calculates configurable quantiles over a sliding time window.

Create a simple Gauge metric to track a custom value (e.g., a random metric like "custom_temperature").

custom_temperature = Gauge('custom_temperature', 'Custom temperature metric')

5. Define Logic for Metrics Collection

Simulate or fetch the custom metric data (e.g., generate a random temperature value or pull data from an API/system).

def collect_metrics():
    while True:
        custom_temperature.set(random.uniform(20.0, 30.0))  # Random value between 20 and 30
        time.sleep(10)  # Update every 10 seconds

6. Start the HTTP Server

Use start_http_server to expose the metrics at an endpoint for Prometheus to scrape.

if __name__ == "__main__":
    start_http_server(8000)  # Expose metrics on port 8000
    collect_metrics()

7. Testing the Exporter

How to run the script and check if the metrics are accessible.

Go to http://localhost:8000/metrics to see the exposed metrics.

Ensure Prometheus can scrape the custom exporter.

8. Configuring Prometheus to Scrape the Exporter

Add the exporter endpoint to Prometheus configuration (prometheus.yml).

scrape_configs:
  - job_name: 'custom_exporter'
    static_configs:
      - targets: ['localhost:8000']

9. Conclusion

In Recapm the process of creating a simple custom Prometheus exporter in Python is to periodically publish text in a format that Prometheus understands (exporting) and then have Prometheus scrape and store that information in a database

In your experimentation with different metrics and use cases, get to know a visualization engine such as Grafana which has a number of community-provided pre-configured dashboards for all kinds of metrics.

Resources for further learning on Prometheus and Python exporters.

• Prometheus
• Grafana
• Grafana Dashboards

10. Next Steps

• Exporters, though simple in concept, can become fairly complex. Collecting data from multiple sources and formulaically manipulating the data can be complex. For example, collection metrics of a kubernetes cluster provides insight into cluster health, pod statuses, node metrics, runner pools, etc.
• Integration with Grafana for visualizing any metrics is a great next step for observing trends in data provided from exporters