Update hivegrid-cloud reference to latest main

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
davidandClaude Fable 5 committed 2026-07-08 16:15:37 +02:00
1 parent e98d38e716
commit 7b21524c86
3781 files changed
+373484 -225607

No files matched your search

@@ -16,9 +16,9 @@ AUTHENTIK_TAG=2026.2.3
# -----------------------------------------------------------------------------
# Public URL — must match the Traefik-routed hostname (see traefik.labels.yml)
# -----------------------------------------------------------------------------
AUTHENTIK_BASE_URL=https://auth.boiler.saas-ctrl.com
AUTHENTIK_BASE_URL=https://auth.tool-ctrl.de
# Optional: cookie domain for cross-subdomain SSO (only set if you have
# multiple boiler.saas-ctrl.com subdomains that should share the IdP cookie).
# multiple app.hivegrid.de subdomains that should share the IdP cookie).
AUTHENTIK_COOKIE_DOMAIN=
# -----------------------------------------------------------------------------
@@ -32,7 +32,7 @@ AUTHENTIK_COOKIE_DOMAIN=
AUTHENTIK_SECRET_KEY=replace-me-with-60-bytes-of-base64-entropy
# -----------------------------------------------------------------------------
# Postgres (Authentik-internal, NOT shared with the boilerplate app)
# Postgres (Authentik-internal, NOT shared with the HiveGrid app)
# -----------------------------------------------------------------------------
AUTHENTIK_POSTGRESQL__USER=authentik
AUTHENTIK_POSTGRESQL__NAME=authentik
@@ -65,8 +65,8 @@ AUTHENTIK_LOG_LEVEL=info
# -----------------------------------------------------------------------------
# Sentry self-hosted DSN (Phase 02 #29). Optional — leave blank to disable.
# DSN format: https://<key>@sentry.boiler.saas-ctrl.com/<project-id>
# Create a "boilerplate-authentik" project in the Sentry UI and paste its DSN.
# DSN format: https://<key>@sentry.saas-ctrl.com/<project-id>
# Create a "hivegrid-authentik" project in the Sentry UI and paste its DSN.
# -----------------------------------------------------------------------------
AUTHENTIK_SENTRY__DSN=
AUTHENTIK_SENTRY__ENVIRONMENT=staging
@@ -83,7 +83,7 @@ AUTHENTIK_EMAIL__PASSWORD=
AUTHENTIK_EMAIL__USE_TLS=true
AUTHENTIK_EMAIL__USE_SSL=false
AUTHENTIK_EMAIL__TIMEOUT=10
AUTHENTIK_EMAIL__FROM=authentik@boiler.saas-ctrl.com
AUTHENTIK_EMAIL__FROM=authentik@tool-ctrl.de
# -----------------------------------------------------------------------------
# OIDC client secret for the Toolctrl admin-panel provider
@@ -94,17 +94,17 @@ AUTHENTIK_EMAIL__FROM=authentik@boiler.saas-ctrl.com
# -----------------------------------------------------------------------------
TOOLCTRL_ADMIN_OIDC_CLIENT_ID=toolctrl-admin-panel
TOOLCTRL_ADMIN_OIDC_CLIENT_SECRET=replace-me-64-hex-chars
TOOLCTRL_ADMIN_OIDC_REDIRECT_URI=https://admin.boiler.saas-ctrl.com/auth/callback
TOOLCTRL_ADMIN_OIDC_REDIRECT_URI=https://admin.hivegrid.de/auth/callback
# -----------------------------------------------------------------------------
# S3 backup target (consumed by infra/authentik/backup/)
# Endpoint format examples:
# Hetzner: https://nbg1.your-objectstorage.com
# AWS: https://s3.eu-central-1.amazonaws.com
# MinIO local: https://minio.boiler.saas-ctrl.com
# MinIO local: https://minio.saas-ctrl.com
# -----------------------------------------------------------------------------
AUTHENTIK_S3_BACKUP_ENDPOINT=https://nbg1.your-objectstorage.com
AUTHENTIK_S3_BACKUP_BUCKET=boilerplate-authentik-backups
AUTHENTIK_S3_BACKUP_BUCKET=hivegrid-authentik-backups
AUTHENTIK_S3_BACKUP_ACCESS_KEY=replace-me
AUTHENTIK_S3_BACKUP_SECRET_KEY=replace-me
@@ -1,7 +1,7 @@
# `infra/authentik` — Self-Hosted Authentik IdP Stack
This directory contains the artifacts to deploy a **self-hosted Authentik**
identity-provider instance on the staging Hetzner host (`boilerplate-staging`,
identity-provider instance on the staging Hetzner host (`HiveGrid staging`,
see `docs/improvement-plan/archive/STAGING-ACCESS.md`) via the existing
Coolify-managed Docker-Compose runtime.
@@ -29,11 +29,11 @@ the default — it lacks OIDC, MFA-enforcement, brand-flows, and blueprints.
- Docker Engine ≥ 24 + Docker Compose v2 (provided by Coolify)
- Traefik v3.6 reverse-proxy from `coolify-proxy` (Let's Encrypt ACME via
the `letsencrypt` cert-resolver)
- DNS record `auth.boiler.saas-ctrl.com` → `46.225.5.93`
- DNS record `auth.tool-ctrl.de` → `46.225.5.93`
(Cloudflare-managed; see `docs/operations/authentik-setup.md` for the
`cf-cli` snippet that creates it)
- Sentry self-hosted stack from Phase 02 #29 reachable at
`sentry.boiler.saas-ctrl.com` (Authentik containers report their own
`sentry.saas-ctrl.com` (Authentik containers report their own
errors to it via `AUTHENTIK_SENTRY__DSN`)
## Layout
@@ -87,7 +87,7 @@ High-level:
## Scope
- Authentik **server** + **worker** + dedicated **postgresql** + dedicated
**redis** — these databases MUST NOT be shared with the boilerplate
**redis** — these databases MUST NOT be shared with HiveGrid
application's own Postgres/Redis (separate compromise blast-radius +
separate backup/restore cycle).
- **No Outposts** — pure OIDC use-case; the reverse-proxy outpost concept
@@ -10,7 +10,7 @@
| Release line | `2026.2.x` (the latest stable line as of 2026-05-17 per `gh release list --repo goauthentik/authentik`) |
| Why not `2026.5.0-rc2` | RC; we wait for the GA promotion of the `2026.5.x` line before upgrading. |
| Why not `2025.12.5` | Older minor; `2026.2.3` is the latest stable and contains the security + flow-stage fixes we care about. |
| Postgres image | `postgres:16-alpine` (Authentik supports 14/15/16; we standardise on 16 to match the rest of the boilerplate) |
| Postgres image | `postgres:16-alpine` (Authentik supports 14/15/16; we standardise on 16 to match the rest of HiveGrid) |
| Redis image | `redis:7-alpine` |
## Why we DO vendor the compose
@@ -27,7 +27,7 @@ based on the upstream reference at
disk plane so backups + log-rotation use the same path as every other
Coolify stack.
- The Postgres + Redis instances stay strictly **Authentik-internal** —
we never share them with the boilerplate app's databases.
we never share them with the HiveGrid app's databases.
- No `ports:` mappings for Postgres/Redis → only reachable on the internal
Docker network.
@@ -116,7 +116,7 @@ REQUIRED for restore:
OPTIONAL but recommended:
- TOOLCTRL_ADMIN_OIDC_CLIENT_SECRET
The OIDC provider's client_secret is stored encrypted in Postgres;
the plaintext lives in Coolify-Vault and in the boilerplate app's
the plaintext lives in Coolify-Vault and in the HiveGrid app's
.env (consumed by Phase 09 #82).
- AUTHENTIK_SENTRY__DSN
@@ -13,7 +13,7 @@
# * token TTLs: code 1m, access 10m, refresh 7d, id-token 10m
# * include_claims_in_id_token=true
# - Application `Toolctrl Admin Panel`
# * launchURL points at https://admin.boiler.saas-ctrl.com/
# * launchURL points at https://admin.hivegrid.de/
# * Access binding: only members of toolctrl-admin OR toolctrl-staff
#
# Also creates a DISABLED placeholder provider `toolctrl-tenant-sso` reserved
@@ -78,7 +78,7 @@ entries:
attrs:
name: "Toolctrl Admin Panel"
provider: !Find [authentik_providers_oauth2.oauth2provider, [name, toolctrl-admin-panel]]
meta_launch_url: "https://admin.boiler.saas-ctrl.com/"
meta_launch_url: "https://admin.hivegrid.de/"
meta_description: "Toolctrl admin panel — super-admin + staff console"
policy_engine_mode: any
open_in_new_tab: false
@@ -15,13 +15,13 @@
#
# Run this from the operator's machine OR inside a Coolify post-deploy hook:
#
# export AUTHENTIK_BASE_URL=https://auth.boiler.saas-ctrl.com
# export AUTHENTIK_BASE_URL=https://auth.tool-ctrl.de
# export AUTHENTIK_BOOTSTRAP_TOKEN=$(cat ~/.cli-tokens/authentik-bootstrap)
# bash infra/authentik/bootstrap/bootstrap.sh
set -Eeuo pipefail
: "${AUTHENTIK_BASE_URL:?AUTHENTIK_BASE_URL must be set (e.g. https://auth.boiler.saas-ctrl.com)}"
: "${AUTHENTIK_BASE_URL:?AUTHENTIK_BASE_URL must be set (e.g. https://auth.tool-ctrl.de)}"
: "${AUTHENTIK_BOOTSTRAP_TOKEN:?AUTHENTIK_BOOTSTRAP_TOKEN must be set (see .env.example)}"
WAIT_MAX_SECONDS="${WAIT_MAX_SECONDS:-300}"
@@ -83,7 +83,7 @@ cat <<'EONOTE'
----------------------------------------------------------------------
NEXT STEPS (operator)
----------------------------------------------------------------------
1. Log into https://auth.boiler.saas-ctrl.com using AUTHENTIK_BOOTSTRAP_EMAIL
1. Log into https://auth.tool-ctrl.de using AUTHENTIK_BOOTSTRAP_EMAIL
+ AUTHENTIK_BOOTSTRAP_PASSWORD. The first login forces a password change.
2. In the Authentik UI, enroll TOTP for your account
@@ -100,6 +100,6 @@ NEXT STEPS (operator)
on each.
5. Verify OIDC discovery from the admin-panel side once #82 is wired:
curl -fsS https://auth.boiler.saas-ctrl.com/application/o/toolctrl-admin-panel/.well-known/openid-configuration | jq .issuer
curl -fsS https://auth.tool-ctrl.de/application/o/toolctrl-admin-panel/.well-known/openid-configuration | jq .issuer
----------------------------------------------------------------------
EONOTE
@@ -8,7 +8,7 @@
# between staging and any future dedicated-host deployment).
#
# Postgres + Redis are strictly Authentik-internal — never share with the
# boilerplate app's databases. No `ports:` mappings anywhere; ingress is
# HiveGrid app's databases. No `ports:` mappings anywhere; ingress is
# Traefik-only.
#
# Volume convention: named volumes are managed by Docker; on staging Coolify
@@ -129,7 +129,7 @@ services:
AUTHENTIK_EMAIL__USE_TLS: ${AUTHENTIK_EMAIL__USE_TLS:-true}
AUTHENTIK_EMAIL__USE_SSL: ${AUTHENTIK_EMAIL__USE_SSL:-false}
AUTHENTIK_EMAIL__TIMEOUT: ${AUTHENTIK_EMAIL__TIMEOUT:-10}
AUTHENTIK_EMAIL__FROM: ${AUTHENTIK_EMAIL__FROM:-authentik@boiler.saas-ctrl.com}
AUTHENTIK_EMAIL__FROM: ${AUTHENTIK_EMAIL__FROM:-authentik@tool-ctrl.de}
volumes:
- authentik-media:/media
- authentik-certs:/certs
@@ -185,7 +185,7 @@ services:
AUTHENTIK_EMAIL__USE_TLS: ${AUTHENTIK_EMAIL__USE_TLS:-true}
AUTHENTIK_EMAIL__USE_SSL: ${AUTHENTIK_EMAIL__USE_SSL:-false}
AUTHENTIK_EMAIL__TIMEOUT: ${AUTHENTIK_EMAIL__TIMEOUT:-10}
AUTHENTIK_EMAIL__FROM: ${AUTHENTIK_EMAIL__FROM:-authentik@boiler.saas-ctrl.com}
AUTHENTIK_EMAIL__FROM: ${AUTHENTIK_EMAIL__FROM:-authentik@tool-ctrl.de}
volumes:
- authentik-media:/media
- authentik-certs:/certs
@@ -13,7 +13,7 @@
# 5. (optional) self-registration endpoint returns 404
#
# Usage:
# export AUTHENTIK_BASE_URL=https://auth.boiler.saas-ctrl.com
# export AUTHENTIK_BASE_URL=https://auth.tool-ctrl.de
# export AUTHENTIK_APP_SLUG=toolctrl-admin-panel
# bash infra/authentik/scripts/smoke.sh
#
@@ -23,7 +23,7 @@
set -Eeuo pipefail
: "${AUTHENTIK_BASE_URL:?AUTHENTIK_BASE_URL must be set (e.g. https://auth.boiler.saas-ctrl.com)}"
: "${AUTHENTIK_BASE_URL:?AUTHENTIK_BASE_URL must be set (e.g. https://auth.tool-ctrl.de)}"
: "${AUTHENTIK_APP_SLUG:=toolctrl-admin-panel}"
FAILED=0
@@ -10,7 +10,7 @@
# and a default ACME cert-resolver named `letsencrypt`. The labels below match
# that convention; they exactly mirror the shape used by infra/sentry/.
#
# Host: auth.boiler.saas-ctrl.com -> DNS A record points at 46.225.5.93
# Host: auth.tool-ctrl.de -> DNS A record points at 46.225.5.93
# (created via Cloudflare API — see docs/operations/authentik-setup.md).
# ---- ROUTERS ----------------------------------------------------------------
@@ -18,14 +18,14 @@
# Primary HTTPS router — terminates TLS, forwards to authentik-server:9000.
traefik.enable: "true"
traefik.docker.network: "coolify"
traefik.http.routers.authentik.rule: "Host(`auth.boiler.saas-ctrl.com`)"
traefik.http.routers.authentik.rule: "Host(`auth.tool-ctrl.de`)"
traefik.http.routers.authentik.entrypoints: "https"
traefik.http.routers.authentik.tls: "true"
traefik.http.routers.authentik.tls.certresolver: "letsencrypt"
traefik.http.services.authentik.loadbalancer.server.port: "9000"
# HTTP -> HTTPS redirect (port 80 catches plain-HTTP hits and 308s them to TLS).
traefik.http.routers.authentik-http.rule: "Host(`auth.boiler.saas-ctrl.com`)"
traefik.http.routers.authentik-http.rule: "Host(`auth.tool-ctrl.de`)"
traefik.http.routers.authentik-http.entrypoints: "http"
traefik.http.routers.authentik-http.middlewares: "authentik-https-redirect"
traefik.http.middlewares.authentik-https-redirect.redirectscheme.scheme: "https"
@@ -0,0 +1,26 @@
# Cap Standalone env for staging/prod. Copy to `.env` next to the compose file,
# chmod 600, and replace every CHANGE_ME value.
CAP_IMAGE=tiago2/cap:3.1.5
ADMIN_KEY=CHANGE_ME_RANDOM_MIN_32_CHARS
REDIS_URL=redis://valkey:6379
REDIS_PREFIX=hivegrid:cap:
# Serve widget.js and WASM from the self-hosted Cap instance instead of jsDelivr.
ENABLE_ASSETS_SERVER=true
WIDGET_VERSION=0.1.56
WASM_VERSION=0.0.7
# Comma-separated browser origins allowed to generate/redeem challenges.
# Staging example:
# CORS_ORIGIN=https://staging-app.hivegrid.de,https://staging-api.hivegrid.de
# Production example:
# CORS_ORIGIN=https://app.hivegrid.de,https://admin.hivegrid.de,https://api.hivegrid.de
CORS_ORIGIN=
# Cloudflare-proxied hosts should rate-limit by the real visitor IP header.
RATELIMIT_IP_HEADER=cf-connecting-ip
# Optional diagnostics. Keep false in production unless actively debugging.
SHOW_ERRORS=false
DISABLE_ERROR_LOGGING=false
@@ -0,0 +1,7 @@
# Staging Cap on node1. DNS should point captcha-staging.hivegrid.de to node1
# while the record is DNS-only until Caddy has issued the certificate.
captcha-staging.hivegrid.de {
encode zstd gzip
reverse_proxy 127.0.0.1:7300
}
@@ -0,0 +1,136 @@
# Self-hosted Cap
Cap is the replacement candidate for mCaptcha. It keeps the same HiveGrid
principle as mCaptcha: self-hosted, no Google/Cloudflare CAPTCHA dependency,
and no third-party widget CDN. The app loads `/assets/widget.js` and WASM from
the Cap instance itself.
## Domain requirement
Cap needs a public HTTPS URL. The browser calls the Cap instance to generate and
solve challenges, and the app server calls the same instance for `siteverify`.
A raw private IP is not a good production setup because browser TLS, CORS,
Cloudflare proxying, and asset loading all depend on a stable public origin.
Recommended hostnames:
- Staging: `https://captcha-staging.hivegrid.de` on node1 (`176.9.111.179`)
- Production: `https://captcha.hivegrid.de` on `hivegrid-prod-worker`
## Staging on node1
node1 runs Cap as a plain docker-compose stack and exposes it through the
existing Coolify Traefik proxy (`coolify-proxy`). The compose service also
joins the external `coolify` network with the stable alias `cap-staging`, so
Traefik can reach it without publishing Cap directly to the public interface.
1. Create DNS `captcha-staging.hivegrid.de -> 176.9.111.179`, DNS-only until
Traefik has issued the certificate.
2. Copy `docker-compose.node1.yml` and `.env.example` to `/opt/cap/` on node1.
3. Create `/opt/cap/.env` from `.env.example`, set `ADMIN_KEY`, and set:
```env
CORS_ORIGIN=https://staging-app.hivegrid.de,https://staging-api.hivegrid.de,https://staging.hivegrid.de
```
4. Start Cap:
```sh
docker compose -f docker-compose.node1.yml up -d
```
5. Install the Traefik dynamic config on node1, then restart the proxy:
```sh
cp traefik-staging.yaml /data/coolify/proxy/dynamic/cap-staging.yaml
docker restart coolify-proxy
```
6. Open `https://captcha-staging.hivegrid.de`, log in with `ADMIN_KEY`, and
create site keys for `login`, `forms`, and `booking`. Keep each public
sitekey and its matching site secret.
7. Set the staging web app env:
```env
CAPTCHA_PROVIDER=cap
CAP_INSTANCE_URL=https://captcha-staging.hivegrid.de
CAP_SITEKEY_LOGIN=<login sitekey>
CAP_SITE_SECRET_LOGIN=<login secret>
CAP_SITEKEY_FORMS=<forms sitekey>
CAP_SITE_SECRET_FORMS=<forms secret>
CAP_SITEKEY_BOOKING=<booking sitekey>
CAP_SITE_SECRET_BOOKING=<booking secret>
CAPTCHA_FAIL_OPEN=true
```
## Production on worker/services VM
Production should run as a Coolify compose service on `hivegrid-prod-worker`
(`49.13.24.190`, private `10.0.0.5`) with FQDN `captcha.hivegrid.de`.
1. In Coolify, create a Docker Compose service from
`infra/cap/docker-compose.coolify.yml`.
2. Set `SERVICE_FQDN_CAP_3000=captcha.hivegrid.de`.
3. Create the service `.env` from `.env.example` and set:
```env
CORS_ORIGIN=https://app.hivegrid.de,https://admin.hivegrid.de,https://api.hivegrid.de
```
4. Deploy and verify `https://captcha.hivegrid.de` returns the Cap dashboard.
5. Create the same three site keys (`login`, `forms`, `booking`).
6. Set the production web env to the `CAP_*` values and redeploy web.
7. Keep the old `MCAPTCHA_*` values available until the Cap rollout is verified.
Rollback is `CAPTCHA_PROVIDER=mcaptcha` plus a web redeploy.
## App integration contract
The web app reads:
```env
CAPTCHA_PROVIDER=cap
CAP_INSTANCE_URL=https://...
CAP_SITEKEY_LOGIN=...
CAP_SITE_SECRET_LOGIN=...
CAP_SITEKEY_FORMS=...
CAP_SITE_SECRET_FORMS=...
CAP_SITEKEY_BOOKING=...
CAP_SITE_SECRET_BOOKING=...
CAPTCHA_FAIL_OPEN=true
```
When Cap is active, the UI renders:
```html
<cap-widget data-cap-api-endpoint="https://<cap-host>/<sitekey>/"></cap-widget>
```
The widget injects `cap-token` into the enclosing form. Server-side verification
uses:
```http
POST https://<cap-host>/<sitekey>/siteverify
Content-Type: application/json
{ "secret": "<site secret>", "response": "<cap-token>" }
```
The expected valid response is `{ "success": true }`.
`Caddyfile.staging.snippet` is kept as a fallback for a non-Coolify node1
setup. The active staging setup uses `traefik-staging.yaml`.
## Cutover checklist
- Staging Cap dashboard reachable over HTTPS.
- `/assets/widget.js` and `/assets/cap_wasm_bg.wasm` return 200 from the Cap
host.
- Staging web env uses `CAPTCHA_PROVIDER=cap`.
- Register/login captcha path verifies with a real browser.
- Public form with captcha enabled submits successfully once and rejects token
replay.
- Production Cap deploy is healthy behind Cloudflare.
- Production web deploy completed after `CAP_*` env is set.
- mCaptcha rollback env remains available until Cap has run through one live
deploy cycle.
@@ -0,0 +1,35 @@
# Self-hosted Cap Standalone for HiveGrid production.
#
# Intended as a Coolify Docker Compose service on hivegrid-prod-worker
# (worker/services VM). Coolify/Traefik terminates TLS for captcha.hivegrid.de.
name: hivegrid-prod-cap
volumes:
cap-valkey-data:
services:
cap:
image: ${CAP_IMAGE:-tiago2/cap:3.1.5}
restart: unless-stopped
env_file: [.env]
expose:
- "3000"
environment:
# Coolify generates the Traefik route for captcha.hivegrid.de from this.
- SERVICE_FQDN_CAP_3000=${SERVICE_FQDN_CAP_3000:-captcha.hivegrid.de}
depends_on:
valkey:
condition: service_healthy
valkey:
image: valkey/valkey:9-alpine
restart: unless-stopped
volumes:
- cap-valkey-data:/data
command: valkey-server --save 60 1 --loglevel warning --maxmemory-policy noeviction
healthcheck:
test: ["CMD", "valkey-cli", "ping"]
interval: 5s
timeout: 3s
retries: 10
@@ -0,0 +1,46 @@
# Self-hosted Cap Standalone for staging/node1.
#
# node1 runs this as a plain docker-compose stack behind the existing
# Coolify Traefik proxy. The Cap container joins the external coolify network
# as cap-staging; the host port is kept on localhost for direct diagnostics.
name: cap
networks:
cap-net:
name: cap-net
coolify:
external: true
volumes:
cap-valkey-data:
name: cap-valkey-data
services:
cap:
image: ${CAP_IMAGE:-tiago2/cap:3.1.5}
restart: unless-stopped
networks:
cap-net:
coolify:
aliases:
- cap-staging
env_file: [.env]
ports:
- "127.0.0.1:7300:3000"
depends_on:
valkey:
condition: service_healthy
valkey:
image: valkey/valkey:9-alpine
restart: unless-stopped
networks: [cap-net]
volumes:
- cap-valkey-data:/data
command: valkey-server --save 60 1 --loglevel warning --maxmemory-policy noeviction
healthcheck:
test: ["CMD", "valkey-cli", "ping"]
interval: 5s
timeout: 3s
retries: 10
@@ -0,0 +1,33 @@
http:
routers:
cap-staging-secure:
rule: "Host(`captcha-staging.hivegrid.de`)"
entryPoints:
- https
service: cap-staging
middlewares:
- cap-staging-security-headers
tls:
certResolver: letsencrypt
cap-staging-web:
rule: "Host(`captcha-staging.hivegrid.de`)"
entryPoints:
- http
service: cap-staging
middlewares:
- cap-staging-https
services:
cap-staging:
loadBalancer:
servers:
- url: "http://cap-staging:3000"
middlewares:
cap-staging-https:
redirectScheme:
scheme: https
cap-staging-security-headers:
headers:
customResponseHeaders:
X-Robots-Tag: "noindex, nofollow"
contentTypeNosniff: true
referrerPolicy: "no-referrer"
@@ -1,7 +1,7 @@
{
"uid": "boilerplate-ai-cost",
"title": "Boilerplate — AI Cost per Tenant",
"tags": ["boilerplate", "ai"],
"uid": "hivegrid-ai-cost",
"title": "HiveGrid — AI Cost per Tenant",
"tags": ["hivegrid", "ai"],
"timezone": "browser",
"schemaVersion": 39,
"refresh": "1m",
@@ -17,7 +17,7 @@
"refId": "A",
"expr": "topk(20, sum by (tenant_bucket) (increase(ai_tokens_consumed_total[24h])))",
"legendFormat": "{{tenant_bucket}}",
"datasource": { "uid": "boilerplate-prometheus", "type": "prometheus" }
"datasource": { "uid": "hivegrid-prometheus", "type": "prometheus" }
}
]
},
@@ -27,7 +27,7 @@
"title": "Tokens by provider (24h)",
"gridPos": { "h": 10, "w": 12, "x": 12, "y": 0 },
"targets": [
{ "refId": "A", "expr": "sum by (provider) (increase(ai_tokens_consumed_total[24h]))", "legendFormat": "{{provider}}", "datasource": { "uid": "boilerplate-prometheus", "type": "prometheus" } }
{ "refId": "A", "expr": "sum by (provider) (increase(ai_tokens_consumed_total[24h]))", "legendFormat": "{{provider}}", "datasource": { "uid": "hivegrid-prometheus", "type": "prometheus" } }
]
},
{
@@ -36,7 +36,7 @@
"title": "Daily token-consumption trend (per model)",
"gridPos": { "h": 10, "w": 24, "x": 0, "y": 10 },
"targets": [
{ "refId": "A", "expr": "sum by (model) (rate(ai_tokens_consumed_total[1h]))", "legendFormat": "{{model}}", "datasource": { "uid": "boilerplate-prometheus", "type": "prometheus" } }
{ "refId": "A", "expr": "sum by (model) (rate(ai_tokens_consumed_total[1h]))", "legendFormat": "{{model}}", "datasource": { "uid": "hivegrid-prometheus", "type": "prometheus" } }
]
},
{
@@ -45,7 +45,7 @@
"title": "Cardinality of tenant_bucket (should stay ≤ 101)",
"gridPos": { "h": 4, "w": 6, "x": 0, "y": 20 },
"targets": [
{ "refId": "A", "expr": "count(count by (tenant_bucket) (ai_tokens_consumed_total))", "datasource": { "uid": "boilerplate-prometheus", "type": "prometheus" } }
{ "refId": "A", "expr": "count(count by (tenant_bucket) (ai_tokens_consumed_total))", "datasource": { "uid": "hivegrid-prometheus", "type": "prometheus" } }
],
"fieldConfig": { "defaults": { "thresholds": { "mode": "absolute", "steps": [{ "color": "green", "value": null }, { "color": "yellow", "value": 80 }, { "color": "red", "value": 101 }] } } }
}
@@ -1,7 +1,7 @@
{
"uid": "boilerplate-api-latency",
"title": "Boilerplate — API Latency",
"tags": ["boilerplate", "api"],
"uid": "hivegrid-api-latency",
"title": "HiveGrid — API Latency",
"tags": ["hivegrid", "api"],
"timezone": "browser",
"schemaVersion": 39,
"refresh": "30s",
@@ -13,9 +13,9 @@
"title": "p50 / p95 / p99 (s) per route",
"gridPos": { "h": 10, "w": 24, "x": 0, "y": 0 },
"targets": [
{ "refId": "A", "expr": "histogram_quantile(0.50, sum by (le, route) (rate(http_request_duration_seconds_bucket{service=\"api\"}[5m])))", "legendFormat": "p50 {{route}}", "datasource": { "uid": "boilerplate-prometheus", "type": "prometheus" } },
{ "refId": "B", "expr": "histogram_quantile(0.95, sum by (le, route) (rate(http_request_duration_seconds_bucket{service=\"api\"}[5m])))", "legendFormat": "p95 {{route}}", "datasource": { "uid": "boilerplate-prometheus", "type": "prometheus" } },
{ "refId": "C", "expr": "histogram_quantile(0.99, sum by (le, route) (rate(http_request_duration_seconds_bucket{service=\"api\"}[5m])))", "legendFormat": "p99 {{route}}", "datasource": { "uid": "boilerplate-prometheus", "type": "prometheus" } }
{ "refId": "A", "expr": "histogram_quantile(0.50, sum by (le, route) (rate(http_request_duration_seconds_bucket{service=\"api\"}[5m])))", "legendFormat": "p50 {{route}}", "datasource": { "uid": "hivegrid-prometheus", "type": "prometheus" } },
{ "refId": "B", "expr": "histogram_quantile(0.95, sum by (le, route) (rate(http_request_duration_seconds_bucket{service=\"api\"}[5m])))", "legendFormat": "p95 {{route}}", "datasource": { "uid": "hivegrid-prometheus", "type": "prometheus" } },
{ "refId": "C", "expr": "histogram_quantile(0.99, sum by (le, route) (rate(http_request_duration_seconds_bucket{service=\"api\"}[5m])))", "legendFormat": "p99 {{route}}", "datasource": { "uid": "hivegrid-prometheus", "type": "prometheus" } }
],
"fieldConfig": { "defaults": { "unit": "s" } }
},
@@ -25,7 +25,7 @@
"title": "Request rate per route",
"gridPos": { "h": 8, "w": 12, "x": 0, "y": 10 },
"targets": [
{ "refId": "A", "expr": "sum by (route) (rate(http_requests_total{service=\"api\"}[1m]))", "legendFormat": "{{route}}", "datasource": { "uid": "boilerplate-prometheus", "type": "prometheus" } }
{ "refId": "A", "expr": "sum by (route) (rate(http_requests_total{service=\"api\"}[1m]))", "legendFormat": "{{route}}", "datasource": { "uid": "hivegrid-prometheus", "type": "prometheus" } }
]
},
{
@@ -34,7 +34,7 @@
"title": "Status-code distribution",
"gridPos": { "h": 8, "w": 12, "x": 12, "y": 10 },
"targets": [
{ "refId": "A", "expr": "sum by (status) (rate(http_requests_total{service=\"api\"}[1m]))", "legendFormat": "{{status}}", "datasource": { "uid": "boilerplate-prometheus", "type": "prometheus" } }
{ "refId": "A", "expr": "sum by (status) (rate(http_requests_total{service=\"api\"}[1m]))", "legendFormat": "{{status}}", "datasource": { "uid": "hivegrid-prometheus", "type": "prometheus" } }
]
}
]
@@ -1,7 +1,7 @@
{
"uid": "boilerplate-business-kpi",
"title": "Boilerplate — Business KPIs",
"tags": ["boilerplate", "business"],
"uid": "hivegrid-business-kpi",
"title": "HiveGrid — Business KPIs",
"tags": ["hivegrid", "business"],
"timezone": "browser",
"schemaVersion": 39,
"refresh": "1m",
@@ -13,7 +13,7 @@
"title": "Active tenants",
"gridPos": { "h": 4, "w": 6, "x": 0, "y": 0 },
"targets": [
{ "refId": "A", "expr": "active_tenants", "datasource": { "uid": "boilerplate-prometheus", "type": "prometheus" } }
{ "refId": "A", "expr": "active_tenants", "datasource": { "uid": "hivegrid-prometheus", "type": "prometheus" } }
]
},
{
@@ -22,16 +22,16 @@
"title": "Active users (24h)",
"gridPos": { "h": 4, "w": 6, "x": 6, "y": 0 },
"targets": [
{ "refId": "A", "expr": "active_users_24h", "datasource": { "uid": "boilerplate-prometheus", "type": "prometheus" } }
{ "refId": "A", "expr": "active_users_24h", "datasource": { "uid": "hivegrid-prometheus", "type": "prometheus" } }
]
},
{
"id": 3,
"type": "timeseries",
"title": "Stripe webhook events / min by type",
"title": "Billing webhook events / min by type",
"gridPos": { "h": 8, "w": 12, "x": 0, "y": 4 },
"targets": [
{ "refId": "A", "expr": "sum by (event_type) (rate(stripe_webhook_events_total[5m]))", "legendFormat": "{{event_type}}", "datasource": { "uid": "boilerplate-prometheus", "type": "prometheus" } }
{ "refId": "A", "expr": "sum by (event_type) (rate(billing_provider_webhook_events_total[5m]))", "legendFormat": "{{event_type}}", "datasource": { "uid": "hivegrid-prometheus", "type": "prometheus" } }
]
},
{
@@ -40,7 +40,7 @@
"title": "Mail delivery rate by status",
"gridPos": { "h": 8, "w": 12, "x": 12, "y": 4 },
"targets": [
{ "refId": "A", "expr": "sum by (status) (rate(mail_sent_total[5m]))", "legendFormat": "{{status}}", "datasource": { "uid": "boilerplate-prometheus", "type": "prometheus" } }
{ "refId": "A", "expr": "sum by (status) (rate(mail_sent_total[5m]))", "legendFormat": "{{status}}", "datasource": { "uid": "hivegrid-prometheus", "type": "prometheus" } }
]
},
{
@@ -49,7 +49,7 @@
"title": "Outbound webhook attempts by status",
"gridPos": { "h": 8, "w": 12, "x": 0, "y": 12 },
"targets": [
{ "refId": "A", "expr": "sum by (status) (rate(webhook_delivery_attempts_total[5m]))", "legendFormat": "{{status}}", "datasource": { "uid": "boilerplate-prometheus", "type": "prometheus" } }
{ "refId": "A", "expr": "sum by (status) (rate(webhook_delivery_attempts_total[5m]))", "legendFormat": "{{status}}", "datasource": { "uid": "hivegrid-prometheus", "type": "prometheus" } }
]
},
{
@@ -58,7 +58,7 @@
"title": "Active subscriptions by plan",
"gridPos": { "h": 8, "w": 12, "x": 12, "y": 12 },
"targets": [
{ "refId": "A", "expr": "subscription_active", "legendFormat": "{{plan_id}}", "datasource": { "uid": "boilerplate-prometheus", "type": "prometheus" } }
{ "refId": "A", "expr": "subscription_active", "legendFormat": "{{plan_id}}", "datasource": { "uid": "hivegrid-prometheus", "type": "prometheus" } }
]
}
]
@@ -1,7 +1,7 @@
{
"uid": "boilerplate-overview",
"title": "Boilerplate — Overview",
"tags": ["boilerplate", "overview"],
"uid": "hivegrid-overview",
"title": "HiveGrid — Overview",
"tags": ["hivegrid", "overview"],
"timezone": "browser",
"schemaVersion": 39,
"refresh": "30s",
@@ -13,7 +13,7 @@
"title": "Services up",
"gridPos": { "h": 4, "w": 6, "x": 0, "y": 0 },
"targets": [
{ "refId": "A", "expr": "count(up{job=~\"boilerplate-.*\"} == 1)", "datasource": { "uid": "boilerplate-prometheus", "type": "prometheus" } }
{ "refId": "A", "expr": "count(up{job=~\"hivegrid-.*\"} == 1)", "datasource": { "uid": "hivegrid-prometheus", "type": "prometheus" } }
],
"fieldConfig": { "defaults": { "mappings": [], "thresholds": { "mode": "absolute", "steps": [{ "color": "red", "value": null }, { "color": "green", "value": 3 }] } } }
},
@@ -23,7 +23,7 @@
"title": "HTTP request rate (per service)",
"gridPos": { "h": 8, "w": 12, "x": 6, "y": 0 },
"targets": [
{ "refId": "A", "expr": "sum by (service) (rate(http_requests_total[1m]))", "legendFormat": "{{service}}", "datasource": { "uid": "boilerplate-prometheus", "type": "prometheus" } }
{ "refId": "A", "expr": "sum by (service) (rate(http_requests_total[1m]))", "legendFormat": "{{service}}", "datasource": { "uid": "hivegrid-prometheus", "type": "prometheus" } }
]
},
{
@@ -36,7 +36,7 @@
"refId": "A",
"expr": "sum by (service) (rate(http_requests_total{status=~\"5..\"}[5m])) / clamp_min(sum by (service) (rate(http_requests_total[5m])), 0.001)",
"legendFormat": "{{service}}",
"datasource": { "uid": "boilerplate-prometheus", "type": "prometheus" }
"datasource": { "uid": "hivegrid-prometheus", "type": "prometheus" }
}
],
"fieldConfig": { "defaults": { "unit": "percentunit", "max": 0.5 } }
@@ -51,7 +51,7 @@
"refId": "A",
"expr": "histogram_quantile(0.95, sum by (le, service) (rate(http_request_duration_seconds_bucket[5m])))",
"legendFormat": "{{service}}",
"datasource": { "uid": "boilerplate-prometheus", "type": "prometheus" }
"datasource": { "uid": "hivegrid-prometheus", "type": "prometheus" }
}
],
"fieldConfig": { "defaults": { "unit": "s" } }
@@ -1,7 +1,7 @@
{
"uid": "boilerplate-queue-health",
"title": "Boilerplate — Queue Health",
"tags": ["boilerplate", "queue"],
"uid": "hivegrid-queue-health",
"title": "HiveGrid — Queue Health",
"tags": ["hivegrid", "queue"],
"timezone": "browser",
"schemaVersion": 39,
"refresh": "30s",
@@ -13,7 +13,7 @@
"title": "Active jobs (gauge) per queue",
"gridPos": { "h": 8, "w": 12, "x": 0, "y": 0 },
"targets": [
{ "refId": "A", "expr": "queue_jobs_active", "legendFormat": "{{queue}}", "datasource": { "uid": "boilerplate-prometheus", "type": "prometheus" } }
{ "refId": "A", "expr": "queue_jobs_active", "legendFormat": "{{queue}}", "datasource": { "uid": "hivegrid-prometheus", "type": "prometheus" } }
]
},
{
@@ -22,7 +22,7 @@
"title": "Waiting jobs (gauge) per queue",
"gridPos": { "h": 8, "w": 12, "x": 12, "y": 0 },
"targets": [
{ "refId": "A", "expr": "queue_jobs_waiting", "legendFormat": "{{queue}}", "datasource": { "uid": "boilerplate-prometheus", "type": "prometheus" } }
{ "refId": "A", "expr": "queue_jobs_waiting", "legendFormat": "{{queue}}", "datasource": { "uid": "hivegrid-prometheus", "type": "prometheus" } }
]
},
{
@@ -31,7 +31,7 @@
"title": "Completed rate per queue",
"gridPos": { "h": 8, "w": 12, "x": 0, "y": 8 },
"targets": [
{ "refId": "A", "expr": "sum by (queue) (rate(queue_jobs_completed_total[5m]))", "legendFormat": "{{queue}}", "datasource": { "uid": "boilerplate-prometheus", "type": "prometheus" } }
{ "refId": "A", "expr": "sum by (queue) (rate(queue_jobs_completed_total[5m]))", "legendFormat": "{{queue}}", "datasource": { "uid": "hivegrid-prometheus", "type": "prometheus" } }
]
},
{
@@ -40,7 +40,7 @@
"title": "Failed rate per queue (by error_type)",
"gridPos": { "h": 8, "w": 12, "x": 12, "y": 8 },
"targets": [
{ "refId": "A", "expr": "sum by (queue, error_type) (rate(queue_jobs_failed_total[5m]))", "legendFormat": "{{queue}} / {{error_type}}", "datasource": { "uid": "boilerplate-prometheus", "type": "prometheus" } }
{ "refId": "A", "expr": "sum by (queue, error_type) (rate(queue_jobs_failed_total[5m]))", "legendFormat": "{{queue}} / {{error_type}}", "datasource": { "uid": "hivegrid-prometheus", "type": "prometheus" } }
]
},
{
@@ -49,7 +49,7 @@
"title": "Job duration heatmap",
"gridPos": { "h": 10, "w": 24, "x": 0, "y": 16 },
"targets": [
{ "refId": "A", "expr": "sum by (le) (rate(queue_job_duration_seconds_bucket[5m]))", "format": "heatmap", "datasource": { "uid": "boilerplate-prometheus", "type": "prometheus" } }
{ "refId": "A", "expr": "sum by (le) (rate(queue_job_duration_seconds_bucket[5m]))", "format": "heatmap", "datasource": { "uid": "hivegrid-prometheus", "type": "prometheus" } }
]
}
]
@@ -1,4 +1,4 @@
# Grafana stack for the boilerplate.
# Grafana stack for HiveGrid.
#
# Deploys alongside Prometheus on the Hetzner observability box. Joins the
# same docker network so Grafana can reach Prometheus via DNS
@@ -7,7 +7,7 @@
services:
grafana:
image: grafana/grafana-oss:11.4.0
container_name: boilerplate-grafana
container_name: hivegrid-grafana
restart: unless-stopped
user: "472:472"
environment:
@@ -17,7 +17,7 @@ services:
# Disable anonymous access — Business / per-tenant data lives in here.
GF_AUTH_ANONYMOUS_ENABLED: "false"
GF_AUTH_DISABLE_LOGIN_FORM: "false"
GF_SERVER_ROOT_URL: ${GRAFANA_ROOT_URL:-https://grafana.boiler.saas-ctrl.com}
GF_SERVER_ROOT_URL: ${GRAFANA_ROOT_URL:-https://grafana.saas-ctrl.com}
# Prometheus token used by the provisioned datasource. The yaml
# references it via env interpolation.
GRAFANA_PROMETHEUS_TOKEN: ${GRAFANA_PROMETHEUS_TOKEN:?GRAFANA_PROMETHEUS_TOKEN must be set}
@@ -42,9 +42,9 @@ services:
networks:
observability:
name: boilerplate-observability
name: hivegrid-observability
external: true
volumes:
grafana-data:
name: boilerplate-grafana-data
name: hivegrid-grafana-data
@@ -1,9 +1,9 @@
apiVersion: 1
providers:
- name: boilerplate
- name: hivegrid
orgId: 1
folder: Boilerplate
folder: HiveGrid
type: file
disableDeletion: true
updateIntervalSeconds: 30
@@ -3,7 +3,7 @@ apiVersion: 1
datasources:
- name: Prometheus
type: prometheus
uid: boilerplate-prometheus
uid: hivegrid-prometheus
access: proxy
url: http://prometheus:9090
isDefault: true
@@ -54,7 +54,7 @@ CAPTCHA_FAIL_OPEN=true
Verify flow: client solves the PoW widget → token → app server calls
`POST {MCAPTCHA_INSTANCE_URL}/api/v1/pow/siteverify` with `{ secret, token, key }`
(handled by `@toolctrl/captcha`).
(handled by `@hivegrid/captcha`).
## Fail policy
@@ -1,4 +1,4 @@
; PgBouncer config for boilerplate.
; PgBouncer config for HiveGrid.
;
; Mounted at /etc/pgbouncer/pgbouncer.ini inside the edoburu/pgbouncer container.
; See docs/ops/pgbouncer.md for tuning rationale and rollback.
@@ -10,7 +10,7 @@
; to Postgres) — see schema.prisma `directUrl`.
[databases]
boilerplate = host=postgres port=5432 dbname=boilerplate_validation
hivegrid = host=postgres port=5432 dbname=hivegrid_validation
[pgbouncer]
listen_addr = 0.0.0.0
@@ -1,6 +1,6 @@
# Prometheus — Boilerplate
# Prometheus — HiveGrid
Scrapes `/metrics` from the three boilerplate apps every 15s.
Scrapes `/metrics` from the three HiveGrid apps every 15s.
## Files
@@ -17,13 +17,13 @@ openssl rand -hex 32 > secrets/metrics_token
chmod 600 secrets/metrics_token
# 2. Create the shared docker network (also used by Grafana)
docker network create boilerplate-observability
docker network create hivegrid-observability
# 3. Start the stack
docker compose up -d
```
Then expose `prometheus.boiler.saas-ctrl.com` via Coolify's proxy with
Then expose `prometheus.saas-ctrl.com` via Coolify's proxy with
**basic-auth or Cloudflare-Access** in front — Prometheus has no
built-in auth and ships internal state.
@@ -31,10 +31,10 @@ built-in auth and ships internal state.
```sh
curl -s -H "Authorization: Bearer $(cat secrets/metrics_token)" \
https://api.boiler.saas-ctrl.com/metrics | head -20
https://api.hivegrid.de/metrics | head -20
```
The Prometheus UI at `https://prometheus.boiler.saas-ctrl.com/targets`
The Prometheus UI at `https://prometheus.saas-ctrl.com/targets`
should list all three jobs as **UP**.
## Cardinality budget
@@ -1,4 +1,4 @@
# Prometheus stack for the boilerplate.
# Prometheus stack for HiveGrid.
#
# Deploy as a Coolify "Docker Compose" service or with vanilla
# `docker compose up -d` on the Hetzner observability box. The bind mount
@@ -15,7 +15,7 @@
services:
prometheus:
image: prom/prometheus:v3.6.0
container_name: boilerplate-prometheus
container_name: hivegrid-prometheus
restart: unless-stopped
user: "65534:65534"
command:
@@ -25,7 +25,7 @@ services:
- --web.console.libraries=/usr/share/prometheus/console_libraries
- --web.console.templates=/usr/share/prometheus/consoles
- --web.enable-lifecycle
- --web.external-url=https://prometheus.boiler.saas-ctrl.com
- --web.external-url=https://prometheus.saas-ctrl.com
volumes:
- ./prometheus.yml:/etc/prometheus/prometheus.yml:ro
- ./secrets:/etc/prometheus/secrets:ro
@@ -37,9 +37,9 @@ services:
networks:
observability:
name: boilerplate-observability
name: hivegrid-observability
external: true
volumes:
prometheus-data:
name: boilerplate-prometheus-data
name: hivegrid-prometheus-data
@@ -1,4 +1,4 @@
# Prometheus scrape configuration for the boilerplate.
# Prometheus scrape configuration for HiveGrid.
#
# Scrapes the three application processes' /metrics endpoints with
# bearer-token authentication. METRICS_TOKEN is injected via the
@@ -17,22 +17,22 @@ global:
scrape_timeout: 10s
evaluation_interval: 15s
external_labels:
cluster: boilerplate
cluster: hivegrid
environment: staging
scrape_configs:
- job_name: boilerplate-api
- job_name: hivegrid-api
metrics_path: /metrics
scheme: https
authorization:
type: Bearer
credentials_file: /etc/prometheus/secrets/metrics_token
static_configs:
- targets: ["api.boiler.saas-ctrl.com"]
- targets: ["api.hivegrid.de"]
labels:
service: api
- job_name: boilerplate-worker
- job_name: hivegrid-worker
metrics_path: /metrics
scheme: http
authorization:
@@ -48,7 +48,7 @@ scrape_configs:
# and use the worker container's stable alias (in Coolify v4
# the UUID-based generated name changes on every redeploy
# so you must either add a docker-compose override that
# sets `networks.<net>.aliases: [boilerplate-worker]` or
# sets `networks.<net>.aliases: [hivegrid-worker]` or
# use Coolify's "Internal-only" service feature).
# (b) Expose the worker admin server through Traefik on a
# dedicated hostname (e.g. worker-admin.<domain>) with a
@@ -58,11 +58,11 @@ scrape_configs:
# Until (a) or (b) is wired the worker job stays DOWN; api+web
# cover the HTTP-side metrics, the per-queue queue_jobs_*
# counters from the worker registry are not yet visible.
- targets: ["worker.boilerplate.internal:9101"]
- targets: ["worker.hivegrid.internal:9101"]
labels:
service: worker
- job_name: boilerplate-web
- job_name: hivegrid-web
metrics_path: /api/metrics
scheme: https
authorization:
@@ -73,7 +73,7 @@ scrape_configs:
# the admin host. Scraping the tenant-facing host returns 404 by
# design. Use admin.<domain> for staging; in prod set to
# admin.<prod-domain>.
- targets: ["admin.boiler.saas-ctrl.com"]
- targets: ["admin.hivegrid.de"]
labels:
service: web
@@ -7,7 +7,7 @@
# -----------------------------------------------------------------------------
# Public URL — must match the Traefik-routed hostname in docker-compose.override.yml
# -----------------------------------------------------------------------------
SENTRY_BASE_URL=https://sentry.boiler.saas-ctrl.com
SENTRY_BASE_URL=https://sentry.saas-ctrl.com
# -----------------------------------------------------------------------------
# Admin / superuser bootstrap
@@ -40,9 +40,9 @@ SENTRY_DSN_WEB=
# Endpoint format examples:
# AWS: https://s3.eu-central-1.amazonaws.com
# Hetzner: https://nbg1.your-objectstorage.com
# MinIO local: https://minio.boiler.saas-ctrl.com
# MinIO local: https://minio.saas-ctrl.com
SENTRY_S3_BACKUP_ENDPOINT=https://nbg1.your-objectstorage.com
SENTRY_S3_BACKUP_BUCKET=boilerplate-sentry-backups
SENTRY_S3_BACKUP_BUCKET=hivegrid-sentry-backups
SENTRY_S3_BACKUP_ACCESS_KEY=replace-me
SENTRY_S3_BACKUP_SECRET_KEY=replace-me
@@ -1,7 +1,7 @@
# `infra/sentry` — Self-Hosted Sentry Stack
This directory contains the artifacts to deploy a **self-hosted Sentry** instance
on the staging Hetzner host (`boilerplate-staging`, see
on the staging Hetzner host (`hivegrid-staging`, see
`docs/improvement-plan/archive/STAGING-ACCESS.md`) via the existing Coolify-managed
Docker-Compose runtime.
@@ -25,7 +25,7 @@ it lacks full Performance / Replay / OTLP support.
- Docker Engine ≥ 24 + Docker Compose v2 (provided by Coolify)
- Traefik v3.6 reverse-proxy from `coolify-proxy` (Let's Encrypt ACME via
`letsencrypt` cert resolver)
- DNS-record `sentry.boiler.saas-ctrl.com` → `46.225.5.93`
- DNS-record `sentry.saas-ctrl.com` → `46.225.5.93`
## Layout
@@ -56,7 +56,7 @@ runbook). High-level:
1. Copy `.env.example` → `.env`, fill secrets (Coolify-Vault).
2. Run `./install.sh` (clones upstream at pinned tag, applies overlays,
installs).
3. In Coolify, attach this stack to the `sentry.boiler.saas-ctrl.com` domain.
3. In Coolify, attach this stack to the `sentry.saas-ctrl.com` domain.
4. Smoke-test per `## Test Requirements` in
`docs/improvement-plan/02-observability/sentry-selfhost-setup.md`.
@@ -40,13 +40,13 @@ services:
# its own labels; the ones below are the project-specific routing rules.
- "traefik.enable=true"
- "traefik.docker.network=coolify"
- "traefik.http.routers.sentry.rule=Host(`sentry.boiler.saas-ctrl.com`)"
- "traefik.http.routers.sentry.rule=Host(`sentry.saas-ctrl.com`)"
- "traefik.http.routers.sentry.entrypoints=https"
- "traefik.http.routers.sentry.tls=true"
- "traefik.http.routers.sentry.tls.certresolver=letsencrypt"
- "traefik.http.services.sentry.loadbalancer.server.port=80"
# HTTP -> HTTPS redirect
- "traefik.http.routers.sentry-http.rule=Host(`sentry.boiler.saas-ctrl.com`)"
- "traefik.http.routers.sentry-http.rule=Host(`sentry.saas-ctrl.com`)"
- "traefik.http.routers.sentry-http.entrypoints=http"
- "traefik.http.routers.sentry-http.middlewares=sentry-https-redirect"
- "traefik.http.middlewares.sentry-https-redirect.redirectscheme.scheme=https"
@@ -140,4 +140,4 @@ log "starting stack (detached)"
)
log "done. Web UI: ${SENTRY_BASE_URL}"
log "next: log in, enable 2FA on the admin account, then create projects 'boilerplate-api', 'boilerplate-worker', 'boilerplate-web' and paste their DSNs into Coolify secrets."
log "next: log in, enable 2FA on the admin account, then create projects 'hivegrid-api', 'hivegrid-worker', 'hivegrid-web' and paste their DSNs into Coolify secrets."
@@ -87,9 +87,9 @@ quotas:
# in the Sentry UI ("Project Settings -> Client Keys -> Allowed Domains").
# Listed here so operators have one searchable place to remember what should
# be set:
# - https://boiler.saas-ctrl.com
# - https://www.boiler.saas-ctrl.com
# - https://admin.boiler.saas-ctrl.com
# - https://api.boiler.saas-ctrl.com
# - https://hivegrid.de
# - https://www.hivegrid.de
# - https://admin.hivegrid.de
# - https://api.hivegrid.de
# Leave the array empty in the UI ONLY for the API and Worker projects (their
# DSNs are never used from a browser).
@@ -15,7 +15,7 @@ import os
# ---------------------------------------------------------------------------
# Single-organization mode
# ---------------------------------------------------------------------------
# We run one tenant ("Boilerplate"). This collapses the org-picker UI and lets
# We run one tenant ("HiveGrid"). This collapses the org-picker UI and lets
# the admin see the global dashboard directly.
SENTRY_SINGLE_ORGANIZATION = True
@@ -30,7 +30,7 @@ except NameError:
SENTRY_FEATURES = {}
# Public signup OFF — invites only. This is the single most important
# hardening setting; without it any visitor to sentry.boiler.saas-ctrl.com
# hardening setting; without it any visitor to sentry.saas-ctrl.com
# can create an account and start receiving events.
SENTRY_FEATURES["auth:register"] = False
@@ -61,7 +61,7 @@ SENTRY_ADMIN_EMAIL = os.environ.get("SENTRY_ADMIN_EMAIL", "")
# Upstream's compose ships its own smtp container; we keep that wiring. The
# admin email above is the From-address for system mails.
SENTRY_OPTIONS = globals().get("SENTRY_OPTIONS", {})
SENTRY_OPTIONS.setdefault("mail.from", SENTRY_ADMIN_EMAIL or "sentry@boiler.saas-ctrl.com")
SENTRY_OPTIONS.setdefault("mail.from", SENTRY_ADMIN_EMAIL or "sentry@hivegrid.de")
# ---------------------------------------------------------------------------
# URL prefix — must match the Traefik-routed hostname
@@ -69,7 +69,7 @@ SENTRY_OPTIONS.setdefault("mail.from", SENTRY_ADMIN_EMAIL or "sentry@boiler.saas
# Sentry uses this to build absolute URLs in emails, web-hooks, and the auth
# flow. Mismatch leads to broken magic-links in invite emails.
SENTRY_OPTIONS["system.url-prefix"] = os.environ.get(
"SENTRY_BASE_URL", "https://sentry.boiler.saas-ctrl.com"
"SENTRY_BASE_URL", "https://sentry.saas-ctrl.com"
)
# Trust the X-Forwarded-* headers from Traefik (TLS terminated upstream).
@@ -11,7 +11,7 @@
# and a default ACME cert-resolver named `letsencrypt`. The labels below match
# that convention.
#
# Host: sentry.boiler.saas-ctrl.com -> DNS A record points at 46.225.5.93.
# Host: sentry.saas-ctrl.com -> DNS A record points at 46.225.5.93.
# When a different staging domain is chosen, update both this file and the
# inline labels in docker-compose.override.yml (single source of truth lives
# in the override file; this fragment is a copy for review).
@@ -21,14 +21,14 @@
# Primary HTTPS router — terminates TLS, forwards to nginx:80.
traefik.enable: "true"
traefik.docker.network: "coolify"
traefik.http.routers.sentry.rule: "Host(`sentry.boiler.saas-ctrl.com`)"
traefik.http.routers.sentry.rule: "Host(`sentry.saas-ctrl.com`)"
traefik.http.routers.sentry.entrypoints: "https"
traefik.http.routers.sentry.tls: "true"
traefik.http.routers.sentry.tls.certresolver: "letsencrypt"
traefik.http.services.sentry.loadbalancer.server.port: "80"
# HTTP -> HTTPS redirect (port 80 catches plain-HTTP hits and 308s them to TLS).
traefik.http.routers.sentry-http.rule: "Host(`sentry.boiler.saas-ctrl.com`)"
traefik.http.routers.sentry-http.rule: "Host(`sentry.saas-ctrl.com`)"
traefik.http.routers.sentry-http.entrypoints: "http"
traefik.http.routers.sentry-http.middlewares: "sentry-https-redirect"
traefik.http.middlewares.sentry-https-redirect.redirectscheme.scheme: "https"