Refresh HiveGrid Cloud reference snapshot

This commit is contained in:
david committed 2026-08-03 15:31:14 +02:00
1 parent 5e99e14772
commit 7f88026f79
5955 files changed
+1177325 -188362

No files matched your search

@@ -1,10 +1,12 @@
# Prometheus — HiveGrid
Scrapes `/metrics` from the three HiveGrid apps every 15s.
Scrapes the HiveGrid runtime metrics every 15s. The web core/admin process and
the separately deployed portal process each have their own process-local
registry and therefore their own scrape target.
## Files
- `prometheus.yml` — scrape jobs (api / worker / web)
- `prometheus.yml` — scrape jobs (api / worker / web-core / web-portal)
- `docker-compose.yml` — Coolify-friendly compose stack
- `secrets/metrics_token` — bearer token (chmod 600, do NOT commit)
@@ -23,19 +25,27 @@ docker network create hivegrid-observability
docker compose up -d
```
Then expose `prometheus.saas-ctrl.com` via Coolify's proxy with
**basic-auth or Cloudflare-Access** in front — Prometheus has no
built-in auth and ships internal state.
Prometheus is deliberately reachable only as `prometheus:9090` on the
`hivegrid-observability` Docker network. It has no host-port binding and no
lifecycle HTTP endpoint. Use Grafana for dashboards; temporary operator access
must use an authenticated tunnel rather than public ingress.
## Verifying scrape targets
```sh
curl -s -H "Authorization: Bearer $(cat secrets/metrics_token)" \
https://api.hivegrid.de/metrics | head -20
curl -s -H "Authorization: Bearer $(cat secrets/metrics_token)" \
https://app.hivegrid.de/api/metrics | head -20
curl -s -H "Authorization: Bearer $(cat secrets/metrics_token)" \
https://portal.hivegrid.de/api/metrics | head -20
```
The Prometheus UI at `https://prometheus.saas-ctrl.com/targets`
should list all three jobs as **UP**.
Through an authenticated operator tunnel, the Prometheus `/targets` view should
list all four jobs as **UP**. Do not also scrape `admin.hivegrid.de`: admin and
core share the same process-local registry, so doing so would double-count.
## Cardinality budget
@@ -14,7 +14,7 @@
services:
prometheus:
image: prom/prometheus:v3.6.0
image: prom/prometheus:v3.6.0@sha256:76947e7ef22f8a698fc638f706685909be425dbe09bd7a2cd7aca849f79b5f64
container_name: hivegrid-prometheus
restart: unless-stopped
user: "65534:65534"
@@ -24,14 +24,12 @@ services:
- --storage.tsdb.retention.time=30d
- --web.console.libraries=/usr/share/prometheus/console_libraries
- --web.console.templates=/usr/share/prometheus/consoles
- --web.enable-lifecycle
- --web.external-url=https://prometheus.saas-ctrl.com
volumes:
- ./prometheus.yml:/etc/prometheus/prometheus.yml:ro
- ./secrets:/etc/prometheus/secrets:ro
- prometheus-data:/prometheus
ports:
- "9090:9090"
expose:
- "9090"
networks:
- observability
@@ -8,7 +8,8 @@
# Targets:
# * api — apps/api admin metrics surface (port 3201 default)
# * worker — apps/worker admin server (port 9101 default; 9100 is health-probe)
# * web — apps/web SSR resource route on /api/metrics
# * web-core — apps/web core SSR resource route on /api/metrics
# * web-portal — apps/web portal SSR resource route on /api/metrics
#
# Operator deploys this on the Hetzner staging server alongside Sentry.
@@ -62,20 +63,32 @@ scrape_configs:
labels:
service: worker
- job_name: hivegrid-web
- job_name: hivegrid-web-core
metrics_path: /api/metrics
scheme: https
authorization:
type: Bearer
credentials_file: /etc/prometheus/secrets/metrics_token
static_configs:
# /api/metrics is admin-only — the resource route is mounted on
# the admin host. Scraping the tenant-facing host returns 404 by
# design. Use admin.<domain> for staging; in prod set to
# admin.<prod-domain>.
- targets: ["admin.hivegrid.de"]
# Core and admin share one process-local registry. Scrape exactly one
# host so counters are not duplicated.
- targets: ["app.hivegrid.de"]
labels:
service: web
runtime_surface: core
- job_name: hivegrid-web-portal
metrics_path: /api/metrics
scheme: https
authorization:
type: Bearer
credentials_file: /etc/prometheus/secrets/metrics_token
static_configs:
# Portal runs in its own process and therefore has a separate registry.
- targets: ["portal.hivegrid.de"]
labels:
service: web
runtime_surface: portal
# When promoting to production, also scrape the production targets behind
# their own bearer-token. Add scrape_configs entries with environment=production.