# Prometheus — Boilerplate Scrapes `/metrics` from the three boilerplate apps every 15s. ## Files - `prometheus.yml` — scrape jobs (api / worker / web) - `docker-compose.yml` — Coolify-friendly compose stack - `secrets/metrics_token` — bearer token (chmod 600, do NOT commit) ## First-time deploy on Hetzner staging ```sh # 1. Create the secret file with the same value as METRICS_TOKEN in Coolify mkdir -p secrets openssl rand -hex 32 > secrets/metrics_token chmod 600 secrets/metrics_token # 2. Create the shared docker network (also used by Grafana) docker network create boilerplate-observability # 3. Start the stack docker compose up -d ``` Then expose `prometheus.boiler.saas-ctrl.com` via Coolify's proxy with **basic-auth or Cloudflare-Access** in front — Prometheus has no built-in auth and ships internal state. ## Verifying scrape targets ```sh curl -s -H "Authorization: Bearer $(cat secrets/metrics_token)" \ https://api.boiler.saas-ctrl.com/metrics | head -20 ``` The Prometheus UI at `https://prometheus.boiler.saas-ctrl.com/targets` should list all three jobs as **UP**. ## Cardinality budget - `ai_tokens_consumed_total` — capped at ~100 tenant buckets + 1 `other` - `subscription_active` — capped by the plan-id table (single digits today) - All other metrics have bounded label sets (HTTP routes, queue names) If a panel suddenly shows millions of series, the most likely culprit is a new histogram with an unbounded label — search the codebase for `labels({` and verify.