49 lines
1.5 KiB
Markdown
49 lines
1.5 KiB
Markdown
# Prometheus — Boilerplate
|
|
|
|
Scrapes `/metrics` from the three boilerplate apps every 15s.
|
|
|
|
## Files
|
|
|
|
- `prometheus.yml` — scrape jobs (api / worker / web)
|
|
- `docker-compose.yml` — Coolify-friendly compose stack
|
|
- `secrets/metrics_token` — bearer token (chmod 600, do NOT commit)
|
|
|
|
## First-time deploy on Hetzner staging
|
|
|
|
```sh
|
|
# 1. Create the secret file with the same value as METRICS_TOKEN in Coolify
|
|
mkdir -p secrets
|
|
openssl rand -hex 32 > secrets/metrics_token
|
|
chmod 600 secrets/metrics_token
|
|
|
|
# 2. Create the shared docker network (also used by Grafana)
|
|
docker network create boilerplate-observability
|
|
|
|
# 3. Start the stack
|
|
docker compose up -d
|
|
```
|
|
|
|
Then expose `prometheus.boiler.saas-ctrl.com` via Coolify's proxy with
|
|
**basic-auth or Cloudflare-Access** in front — Prometheus has no
|
|
built-in auth and ships internal state.
|
|
|
|
## Verifying scrape targets
|
|
|
|
```sh
|
|
curl -s -H "Authorization: Bearer $(cat secrets/metrics_token)" \
|
|
https://api.boiler.saas-ctrl.com/metrics | head -20
|
|
```
|
|
|
|
The Prometheus UI at `https://prometheus.boiler.saas-ctrl.com/targets`
|
|
should list all three jobs as **UP**.
|
|
|
|
## Cardinality budget
|
|
|
|
- `ai_tokens_consumed_total` — capped at ~100 tenant buckets + 1 `other`
|
|
- `subscription_active` — capped by the plan-id table (single digits today)
|
|
- All other metrics have bounded label sets (HTTP routes, queue names)
|
|
|
|
If a panel suddenly shows millions of series, the most likely culprit
|
|
is a new histogram with an unbounded label — search the codebase for
|
|
`labels({` and verify.
|