{"id":53,"date":"2025-11-03T12:00:00","date_gmt":"2025-11-03T12:00:00","guid":{"rendered":"https:\/\/rezachegini.com\/?p=53"},"modified":"2026-10-30T06:41:33","modified_gmt":"2026-10-30T06:41:33","slug":"prometheus-client-libraries-and-short-lived-jobs","status":"publish","type":"post","link":"https:\/\/rezachegini.com\/?p=53","title":{"rendered":"Prometheus Client Libraries and Short-Lived Jobs"},"content":{"rendered":"<h3 class=\"wp-block-heading\">1) What Are Client Libraries<\/h3>\n<p class=\"wp-block-paragraph\">Prometheus usually collects data from exporters, like Node Exporter or cAdvisor, which always run in the background and expose system metrics.<\/p>\n<p class=\"wp-block-paragraph\">However, sometimes we want to create our own metrics directly from an application. That&#8217;s where client libraries help.<\/p>\n<p class=\"wp-block-paragraph\">A client library is a small piece of code that lets your application expose metrics in Prometheus format. It works like adding an exporter inside your own program.<\/p>\n<p class=\"wp-block-paragraph\">You can use it to track things such as:<\/p>\n<ul class=\"wp-block-list\"><li>Number of requests handled<\/li><li>Time taken to process a function<\/li><li>Current size of a queue<\/li><li>Number of errors<\/li><\/ul>\n<p class=\"wp-block-paragraph\">Using a client library gives you full control over what metrics you want to monitor from your code.<\/p>\n<h3 class=\"wp-block-heading\">2) Official Client Libraries (without Java\/C#)<\/h3>\n<p class=\"wp-block-paragraph\">Prometheus has official client libraries for:<\/p>\n<ul class=\"wp-block-list\"><li>Go<\/li><li>Python<\/li><li>Ruby<\/li><\/ul>\n<p class=\"wp-block-paragraph\">In this post I use Python because it is quick for demos and scripting.<\/p>\n<h3 class=\"wp-block-heading\">3) Python Quick Start (Summary metric)<\/h3>\n<p class=\"wp-block-paragraph\">Install and run:<\/p>\n<pre class=\"wp-block-code\"><code>python3 -m venv py-metrics\nsource py-metrics\/bin\/activate\npip install prometheus-client<\/code><\/pre>\n<p class=\"wp-block-paragraph\">Simple app with a Summary to measure time:<\/p>\n<pre class=\"wp-block-code\"><code>from prometheus_client import Summary, start_http_server\nimport random, time\n\nREQUEST_TIME = Summary('request_processing_seconds',\n                       'Time spent processing a request')\n\n@REQUEST_TIME.time()\ndef process_request(t):\n    time.sleep(t)\n\nif __name__ == \"__main__\":\n    start_http_server(8000)  # \/metrics on port 8000\n    while True:\n        process_request(random.random())<\/code><\/pre>\n<p class=\"wp-block-paragraph\">Run it:<\/p>\n<pre class=\"wp-block-code\"><code>python3 app.py<\/code><\/pre>\n<p class=\"wp-block-paragraph\">Open http:\/\/localhost:8000\/metrics and you will see metrics like:<\/p>\n<ul class=\"wp-block-list\"><li>request_processing_seconds_count<\/li><li>request_processing_seconds_sum<\/li><\/ul>\n<h3 class=\"wp-block-heading\">4) Counters and Gauges (Python)<\/h3>\n<p class=\"wp-block-paragraph\"><strong>Counters<\/strong> only go up (or reset on restart). Good for requests or errors.<\/p>\n<pre class=\"wp-block-code\"><code>from prometheus_client import Counter\nerrors_total = Counter('errors_total', 'Number of errors')\nerrors_total.inc()     # +1\nerrors_total.inc(5)    # +5<\/code><\/pre>\n<p class=\"wp-block-paragraph\"><strong>Gauges<\/strong> go up and down. Good for &#8220;current value&#8221; like queue size or memory.<\/p>\n<pre class=\"wp-block-code\"><code>from prometheus_client import Gauge\nqueue_size = Gauge('queue_size', 'Current items in queue')\nqueue_size.set(12)\nqueue_size.inc(3)\nqueue_size.dec(2)<\/code><\/pre>\n<p class=\"wp-block-paragraph\">Common tip: Prometheus adds _total to counters automatically. Do not add _total in the name yourself.<\/p>\n<h3 class=\"wp-block-heading\">5) Labels (add dimensions)<\/h3>\n<p class=\"wp-block-paragraph\">Labels help you filter and group metrics.<\/p>\n<pre class=\"wp-block-code\"><code>from prometheus_client import Counter\nhttp_requests = Counter('http_requests', 'HTTP requests', ['method', 'status'])\n\n# Set labels before increment\nhttp_requests.labels(method='GET',  status='200').inc()\nhttp_requests.labels(method='POST', status='500').inc(2)<\/code><\/pre>\n<p class=\"wp-block-paragraph\">Now you can query:<\/p>\n<pre class=\"wp-block-code\"><code>http_requests_total{method=\"GET\",status=\"200\"}<\/code><\/pre>\n<h3 class=\"wp-block-heading\">6) Add Your App to Prometheus<\/h3>\n<p class=\"wp-block-paragraph\">Edit prometheus.yml:<\/p>\n<pre class=\"wp-block-code\"><code>scrape_configs:\n  - job_name: \"python\"\n    static_configs:\n      - targets: [\"localhost:8000\"]<\/code><\/pre>\n<h3 class=\"wp-block-heading\">7) Best Practices<\/h3>\n<ul class=\"wp-block-list\"><li>Choose right type:<\/li><li>Name clearly: lowercase, snake_case, include units like _seconds or _bytes.<\/li><li>Use labels carefully: a few stable labels are good; do not add user IDs or things with many unique values.<\/li><li>Security: do not expose secrets in labels; protect the metrics endpoint if needed.<\/li><li>Performance: do not create thousands of unique label values; each unique label set is a new time series.<\/li><\/ul>\n<h3 class=\"wp-block-heading\">9) Common Issues and Fixes<\/h3>\n<ul class=\"wp-block-list\"><li>No metrics in Prometheus: open the app&#8217;s \/metrics in a browser, check port, update prometheus.yml, and restart.<\/li><li>Counter resets to zero: app restarted; use rate() or increase() in PromQL to handle restarts.<\/li><li>&#8220;Address already in use&#8221;: find the process with lsof -i :8000 (Linux\/Mac) and stop it, or choose a new port.<\/li><li>Labels not showing: you must call .labels(&#8230;).inc() with all defined labels.<\/li><\/ul>","protected":false},"excerpt":{"rendered":"<p>1) What Are Client Libraries Prometheus usually collects data from exporters, like Node Exporter or cAdvisor, which always run in the background&hellip;<\/p>\n","protected":false},"author":2,"featured_media":55,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_rezaops_part":"5","_rezaops_linkedin_url":"https:\/\/www.linkedin.com\/pulse\/prometheus-client-libraries-short-lived-jobs-reza-chegini-vk0uc\/","footnotes":""},"categories":[10,14],"tags":[24,8,16,25],"series":[15],"class_list":["post-53","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-devops","category-observability","tag-client-libraries","tag-devops","tag-prometheus","tag-python","series-prometheus"],"jetpack_featured_media_url":"https:\/\/rezachegini.com\/wp-content\/uploads\/2026\/10\/prometheus-client-libraries-short-lived-jobs-cover.png","_links":{"self":[{"href":"https:\/\/rezachegini.com\/index.php?rest_route=\/wp\/v2\/posts\/53","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/rezachegini.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/rezachegini.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/rezachegini.com\/index.php?rest_route=\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/rezachegini.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=53"}],"version-history":[{"count":1,"href":"https:\/\/rezachegini.com\/index.php?rest_route=\/wp\/v2\/posts\/53\/revisions"}],"predecessor-version":[{"id":54,"href":"https:\/\/rezachegini.com\/index.php?rest_route=\/wp\/v2\/posts\/53\/revisions\/54"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/rezachegini.com\/index.php?rest_route=\/wp\/v2\/media\/55"}],"wp:attachment":[{"href":"https:\/\/rezachegini.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=53"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/rezachegini.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=53"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/rezachegini.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=53"},{"taxonomy":"series","embeddable":true,"href":"https:\/\/rezachegini.com\/index.php?rest_route=%2Fwp%2Fv2%2Fseries&post=53"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}