Update hivegrid-cloud reference to latest main
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
1 parent
e98d38e716
commit
7b21524c86
3781 files changed
+373484
-225607
No files matched your search
@@ -16,9 +16,9 @@ AUTHENTIK_TAG=2026.2.3
|
||||
# -----------------------------------------------------------------------------
|
||||
# Public URL — must match the Traefik-routed hostname (see traefik.labels.yml)
|
||||
# -----------------------------------------------------------------------------
|
||||
AUTHENTIK_BASE_URL=https://auth.boiler.saas-ctrl.com
|
||||
AUTHENTIK_BASE_URL=https://auth.tool-ctrl.de
|
||||
# Optional: cookie domain for cross-subdomain SSO (only set if you have
|
||||
# multiple boiler.saas-ctrl.com subdomains that should share the IdP cookie).
|
||||
# multiple app.hivegrid.de subdomains that should share the IdP cookie).
|
||||
AUTHENTIK_COOKIE_DOMAIN=
|
||||
|
||||
# -----------------------------------------------------------------------------
|
||||
@@ -32,7 +32,7 @@ AUTHENTIK_COOKIE_DOMAIN=
|
||||
AUTHENTIK_SECRET_KEY=replace-me-with-60-bytes-of-base64-entropy
|
||||
|
||||
# -----------------------------------------------------------------------------
|
||||
# Postgres (Authentik-internal, NOT shared with the boilerplate app)
|
||||
# Postgres (Authentik-internal, NOT shared with the HiveGrid app)
|
||||
# -----------------------------------------------------------------------------
|
||||
AUTHENTIK_POSTGRESQL__USER=authentik
|
||||
AUTHENTIK_POSTGRESQL__NAME=authentik
|
||||
@@ -65,8 +65,8 @@ AUTHENTIK_LOG_LEVEL=info
|
||||
|
||||
# -----------------------------------------------------------------------------
|
||||
# Sentry self-hosted DSN (Phase 02 #29). Optional — leave blank to disable.
|
||||
# DSN format: https://<key>@sentry.boiler.saas-ctrl.com/<project-id>
|
||||
# Create a "boilerplate-authentik" project in the Sentry UI and paste its DSN.
|
||||
# DSN format: https://<key>@sentry.saas-ctrl.com/<project-id>
|
||||
# Create a "hivegrid-authentik" project in the Sentry UI and paste its DSN.
|
||||
# -----------------------------------------------------------------------------
|
||||
AUTHENTIK_SENTRY__DSN=
|
||||
AUTHENTIK_SENTRY__ENVIRONMENT=staging
|
||||
@@ -83,7 +83,7 @@ AUTHENTIK_EMAIL__PASSWORD=
|
||||
AUTHENTIK_EMAIL__USE_TLS=true
|
||||
AUTHENTIK_EMAIL__USE_SSL=false
|
||||
AUTHENTIK_EMAIL__TIMEOUT=10
|
||||
AUTHENTIK_EMAIL__FROM=authentik@boiler.saas-ctrl.com
|
||||
AUTHENTIK_EMAIL__FROM=authentik@tool-ctrl.de
|
||||
|
||||
# -----------------------------------------------------------------------------
|
||||
# OIDC client secret for the Toolctrl admin-panel provider
|
||||
@@ -94,17 +94,17 @@ AUTHENTIK_EMAIL__FROM=authentik@boiler.saas-ctrl.com
|
||||
# -----------------------------------------------------------------------------
|
||||
TOOLCTRL_ADMIN_OIDC_CLIENT_ID=toolctrl-admin-panel
|
||||
TOOLCTRL_ADMIN_OIDC_CLIENT_SECRET=replace-me-64-hex-chars
|
||||
TOOLCTRL_ADMIN_OIDC_REDIRECT_URI=https://admin.boiler.saas-ctrl.com/auth/callback
|
||||
TOOLCTRL_ADMIN_OIDC_REDIRECT_URI=https://admin.hivegrid.de/auth/callback
|
||||
|
||||
# -----------------------------------------------------------------------------
|
||||
# S3 backup target (consumed by infra/authentik/backup/)
|
||||
# Endpoint format examples:
|
||||
# Hetzner: https://nbg1.your-objectstorage.com
|
||||
# AWS: https://s3.eu-central-1.amazonaws.com
|
||||
# MinIO local: https://minio.boiler.saas-ctrl.com
|
||||
# MinIO local: https://minio.saas-ctrl.com
|
||||
# -----------------------------------------------------------------------------
|
||||
AUTHENTIK_S3_BACKUP_ENDPOINT=https://nbg1.your-objectstorage.com
|
||||
AUTHENTIK_S3_BACKUP_BUCKET=boilerplate-authentik-backups
|
||||
AUTHENTIK_S3_BACKUP_BUCKET=hivegrid-authentik-backups
|
||||
AUTHENTIK_S3_BACKUP_ACCESS_KEY=replace-me
|
||||
AUTHENTIK_S3_BACKUP_SECRET_KEY=replace-me
|
||||
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
# `infra/authentik` — Self-Hosted Authentik IdP Stack
|
||||
|
||||
This directory contains the artifacts to deploy a **self-hosted Authentik**
|
||||
identity-provider instance on the staging Hetzner host (`boilerplate-staging`,
|
||||
identity-provider instance on the staging Hetzner host (`HiveGrid staging`,
|
||||
see `docs/improvement-plan/archive/STAGING-ACCESS.md`) via the existing
|
||||
Coolify-managed Docker-Compose runtime.
|
||||
|
||||
@@ -29,11 +29,11 @@ the default — it lacks OIDC, MFA-enforcement, brand-flows, and blueprints.
|
||||
- Docker Engine ≥ 24 + Docker Compose v2 (provided by Coolify)
|
||||
- Traefik v3.6 reverse-proxy from `coolify-proxy` (Let's Encrypt ACME via
|
||||
the `letsencrypt` cert-resolver)
|
||||
- DNS record `auth.boiler.saas-ctrl.com` → `46.225.5.93`
|
||||
- DNS record `auth.tool-ctrl.de` → `46.225.5.93`
|
||||
(Cloudflare-managed; see `docs/operations/authentik-setup.md` for the
|
||||
`cf-cli` snippet that creates it)
|
||||
- Sentry self-hosted stack from Phase 02 #29 reachable at
|
||||
`sentry.boiler.saas-ctrl.com` (Authentik containers report their own
|
||||
`sentry.saas-ctrl.com` (Authentik containers report their own
|
||||
errors to it via `AUTHENTIK_SENTRY__DSN`)
|
||||
|
||||
## Layout
|
||||
@@ -87,7 +87,7 @@ High-level:
|
||||
## Scope
|
||||
|
||||
- Authentik **server** + **worker** + dedicated **postgresql** + dedicated
|
||||
**redis** — these databases MUST NOT be shared with the boilerplate
|
||||
**redis** — these databases MUST NOT be shared with HiveGrid
|
||||
application's own Postgres/Redis (separate compromise blast-radius +
|
||||
separate backup/restore cycle).
|
||||
- **No Outposts** — pure OIDC use-case; the reverse-proxy outpost concept
|
||||
|
||||
@@ -10,7 +10,7 @@
|
||||
| Release line | `2026.2.x` (the latest stable line as of 2026-05-17 per `gh release list --repo goauthentik/authentik`) |
|
||||
| Why not `2026.5.0-rc2` | RC; we wait for the GA promotion of the `2026.5.x` line before upgrading. |
|
||||
| Why not `2025.12.5` | Older minor; `2026.2.3` is the latest stable and contains the security + flow-stage fixes we care about. |
|
||||
| Postgres image | `postgres:16-alpine` (Authentik supports 14/15/16; we standardise on 16 to match the rest of the boilerplate) |
|
||||
| Postgres image | `postgres:16-alpine` (Authentik supports 14/15/16; we standardise on 16 to match the rest of HiveGrid) |
|
||||
| Redis image | `redis:7-alpine` |
|
||||
|
||||
## Why we DO vendor the compose
|
||||
@@ -27,7 +27,7 @@ based on the upstream reference at
|
||||
disk plane so backups + log-rotation use the same path as every other
|
||||
Coolify stack.
|
||||
- The Postgres + Redis instances stay strictly **Authentik-internal** —
|
||||
we never share them with the boilerplate app's databases.
|
||||
we never share them with the HiveGrid app's databases.
|
||||
- No `ports:` mappings for Postgres/Redis → only reachable on the internal
|
||||
Docker network.
|
||||
|
||||
|
||||
@@ -116,7 +116,7 @@ REQUIRED for restore:
|
||||
OPTIONAL but recommended:
|
||||
- TOOLCTRL_ADMIN_OIDC_CLIENT_SECRET
|
||||
The OIDC provider's client_secret is stored encrypted in Postgres;
|
||||
the plaintext lives in Coolify-Vault and in the boilerplate app's
|
||||
the plaintext lives in Coolify-Vault and in the HiveGrid app's
|
||||
.env (consumed by Phase 09 #82).
|
||||
|
||||
- AUTHENTIK_SENTRY__DSN
|
||||
|
||||
+2
-2
@@ -13,7 +13,7 @@
|
||||
# * token TTLs: code 1m, access 10m, refresh 7d, id-token 10m
|
||||
# * include_claims_in_id_token=true
|
||||
# - Application `Toolctrl Admin Panel`
|
||||
# * launchURL points at https://admin.boiler.saas-ctrl.com/
|
||||
# * launchURL points at https://admin.hivegrid.de/
|
||||
# * Access binding: only members of toolctrl-admin OR toolctrl-staff
|
||||
#
|
||||
# Also creates a DISABLED placeholder provider `toolctrl-tenant-sso` reserved
|
||||
@@ -78,7 +78,7 @@ entries:
|
||||
attrs:
|
||||
name: "Toolctrl Admin Panel"
|
||||
provider: !Find [authentik_providers_oauth2.oauth2provider, [name, toolctrl-admin-panel]]
|
||||
meta_launch_url: "https://admin.boiler.saas-ctrl.com/"
|
||||
meta_launch_url: "https://admin.hivegrid.de/"
|
||||
meta_description: "Toolctrl admin panel — super-admin + staff console"
|
||||
policy_engine_mode: any
|
||||
open_in_new_tab: false
|
||||
|
||||
@@ -15,13 +15,13 @@
|
||||
#
|
||||
# Run this from the operator's machine OR inside a Coolify post-deploy hook:
|
||||
#
|
||||
# export AUTHENTIK_BASE_URL=https://auth.boiler.saas-ctrl.com
|
||||
# export AUTHENTIK_BASE_URL=https://auth.tool-ctrl.de
|
||||
# export AUTHENTIK_BOOTSTRAP_TOKEN=$(cat ~/.cli-tokens/authentik-bootstrap)
|
||||
# bash infra/authentik/bootstrap/bootstrap.sh
|
||||
|
||||
set -Eeuo pipefail
|
||||
|
||||
: "${AUTHENTIK_BASE_URL:?AUTHENTIK_BASE_URL must be set (e.g. https://auth.boiler.saas-ctrl.com)}"
|
||||
: "${AUTHENTIK_BASE_URL:?AUTHENTIK_BASE_URL must be set (e.g. https://auth.tool-ctrl.de)}"
|
||||
: "${AUTHENTIK_BOOTSTRAP_TOKEN:?AUTHENTIK_BOOTSTRAP_TOKEN must be set (see .env.example)}"
|
||||
|
||||
WAIT_MAX_SECONDS="${WAIT_MAX_SECONDS:-300}"
|
||||
@@ -83,7 +83,7 @@ cat <<'EONOTE'
|
||||
----------------------------------------------------------------------
|
||||
NEXT STEPS (operator)
|
||||
----------------------------------------------------------------------
|
||||
1. Log into https://auth.boiler.saas-ctrl.com using AUTHENTIK_BOOTSTRAP_EMAIL
|
||||
1. Log into https://auth.tool-ctrl.de using AUTHENTIK_BOOTSTRAP_EMAIL
|
||||
+ AUTHENTIK_BOOTSTRAP_PASSWORD. The first login forces a password change.
|
||||
|
||||
2. In the Authentik UI, enroll TOTP for your account
|
||||
@@ -100,6 +100,6 @@ NEXT STEPS (operator)
|
||||
on each.
|
||||
|
||||
5. Verify OIDC discovery from the admin-panel side once #82 is wired:
|
||||
curl -fsS https://auth.boiler.saas-ctrl.com/application/o/toolctrl-admin-panel/.well-known/openid-configuration | jq .issuer
|
||||
curl -fsS https://auth.tool-ctrl.de/application/o/toolctrl-admin-panel/.well-known/openid-configuration | jq .issuer
|
||||
----------------------------------------------------------------------
|
||||
EONOTE
|
||||
@@ -8,7 +8,7 @@
|
||||
# between staging and any future dedicated-host deployment).
|
||||
#
|
||||
# Postgres + Redis are strictly Authentik-internal — never share with the
|
||||
# boilerplate app's databases. No `ports:` mappings anywhere; ingress is
|
||||
# HiveGrid app's databases. No `ports:` mappings anywhere; ingress is
|
||||
# Traefik-only.
|
||||
#
|
||||
# Volume convention: named volumes are managed by Docker; on staging Coolify
|
||||
@@ -129,7 +129,7 @@ services:
|
||||
AUTHENTIK_EMAIL__USE_TLS: ${AUTHENTIK_EMAIL__USE_TLS:-true}
|
||||
AUTHENTIK_EMAIL__USE_SSL: ${AUTHENTIK_EMAIL__USE_SSL:-false}
|
||||
AUTHENTIK_EMAIL__TIMEOUT: ${AUTHENTIK_EMAIL__TIMEOUT:-10}
|
||||
AUTHENTIK_EMAIL__FROM: ${AUTHENTIK_EMAIL__FROM:-authentik@boiler.saas-ctrl.com}
|
||||
AUTHENTIK_EMAIL__FROM: ${AUTHENTIK_EMAIL__FROM:-authentik@tool-ctrl.de}
|
||||
volumes:
|
||||
- authentik-media:/media
|
||||
- authentik-certs:/certs
|
||||
@@ -185,7 +185,7 @@ services:
|
||||
AUTHENTIK_EMAIL__USE_TLS: ${AUTHENTIK_EMAIL__USE_TLS:-true}
|
||||
AUTHENTIK_EMAIL__USE_SSL: ${AUTHENTIK_EMAIL__USE_SSL:-false}
|
||||
AUTHENTIK_EMAIL__TIMEOUT: ${AUTHENTIK_EMAIL__TIMEOUT:-10}
|
||||
AUTHENTIK_EMAIL__FROM: ${AUTHENTIK_EMAIL__FROM:-authentik@boiler.saas-ctrl.com}
|
||||
AUTHENTIK_EMAIL__FROM: ${AUTHENTIK_EMAIL__FROM:-authentik@tool-ctrl.de}
|
||||
volumes:
|
||||
- authentik-media:/media
|
||||
- authentik-certs:/certs
|
||||
|
||||
@@ -13,7 +13,7 @@
|
||||
# 5. (optional) self-registration endpoint returns 404
|
||||
#
|
||||
# Usage:
|
||||
# export AUTHENTIK_BASE_URL=https://auth.boiler.saas-ctrl.com
|
||||
# export AUTHENTIK_BASE_URL=https://auth.tool-ctrl.de
|
||||
# export AUTHENTIK_APP_SLUG=toolctrl-admin-panel
|
||||
# bash infra/authentik/scripts/smoke.sh
|
||||
#
|
||||
@@ -23,7 +23,7 @@
|
||||
|
||||
set -Eeuo pipefail
|
||||
|
||||
: "${AUTHENTIK_BASE_URL:?AUTHENTIK_BASE_URL must be set (e.g. https://auth.boiler.saas-ctrl.com)}"
|
||||
: "${AUTHENTIK_BASE_URL:?AUTHENTIK_BASE_URL must be set (e.g. https://auth.tool-ctrl.de)}"
|
||||
: "${AUTHENTIK_APP_SLUG:=toolctrl-admin-panel}"
|
||||
|
||||
FAILED=0
|
||||
|
||||
@@ -10,7 +10,7 @@
|
||||
# and a default ACME cert-resolver named `letsencrypt`. The labels below match
|
||||
# that convention; they exactly mirror the shape used by infra/sentry/.
|
||||
#
|
||||
# Host: auth.boiler.saas-ctrl.com -> DNS A record points at 46.225.5.93
|
||||
# Host: auth.tool-ctrl.de -> DNS A record points at 46.225.5.93
|
||||
# (created via Cloudflare API — see docs/operations/authentik-setup.md).
|
||||
|
||||
# ---- ROUTERS ----------------------------------------------------------------
|
||||
@@ -18,14 +18,14 @@
|
||||
# Primary HTTPS router — terminates TLS, forwards to authentik-server:9000.
|
||||
traefik.enable: "true"
|
||||
traefik.docker.network: "coolify"
|
||||
traefik.http.routers.authentik.rule: "Host(`auth.boiler.saas-ctrl.com`)"
|
||||
traefik.http.routers.authentik.rule: "Host(`auth.tool-ctrl.de`)"
|
||||
traefik.http.routers.authentik.entrypoints: "https"
|
||||
traefik.http.routers.authentik.tls: "true"
|
||||
traefik.http.routers.authentik.tls.certresolver: "letsencrypt"
|
||||
traefik.http.services.authentik.loadbalancer.server.port: "9000"
|
||||
|
||||
# HTTP -> HTTPS redirect (port 80 catches plain-HTTP hits and 308s them to TLS).
|
||||
traefik.http.routers.authentik-http.rule: "Host(`auth.boiler.saas-ctrl.com`)"
|
||||
traefik.http.routers.authentik-http.rule: "Host(`auth.tool-ctrl.de`)"
|
||||
traefik.http.routers.authentik-http.entrypoints: "http"
|
||||
traefik.http.routers.authentik-http.middlewares: "authentik-https-redirect"
|
||||
traefik.http.middlewares.authentik-https-redirect.redirectscheme.scheme: "https"
|
||||
|
||||
@@ -0,0 +1,26 @@
|
||||
# Cap Standalone env for staging/prod. Copy to `.env` next to the compose file,
|
||||
# chmod 600, and replace every CHANGE_ME value.
|
||||
|
||||
CAP_IMAGE=tiago2/cap:3.1.5
|
||||
ADMIN_KEY=CHANGE_ME_RANDOM_MIN_32_CHARS
|
||||
REDIS_URL=redis://valkey:6379
|
||||
REDIS_PREFIX=hivegrid:cap:
|
||||
|
||||
# Serve widget.js and WASM from the self-hosted Cap instance instead of jsDelivr.
|
||||
ENABLE_ASSETS_SERVER=true
|
||||
WIDGET_VERSION=0.1.56
|
||||
WASM_VERSION=0.0.7
|
||||
|
||||
# Comma-separated browser origins allowed to generate/redeem challenges.
|
||||
# Staging example:
|
||||
# CORS_ORIGIN=https://staging-app.hivegrid.de,https://staging-api.hivegrid.de
|
||||
# Production example:
|
||||
# CORS_ORIGIN=https://app.hivegrid.de,https://admin.hivegrid.de,https://api.hivegrid.de
|
||||
CORS_ORIGIN=
|
||||
|
||||
# Cloudflare-proxied hosts should rate-limit by the real visitor IP header.
|
||||
RATELIMIT_IP_HEADER=cf-connecting-ip
|
||||
|
||||
# Optional diagnostics. Keep false in production unless actively debugging.
|
||||
SHOW_ERRORS=false
|
||||
DISABLE_ERROR_LOGGING=false
|
||||
@@ -0,0 +1,7 @@
|
||||
# Staging Cap on node1. DNS should point captcha-staging.hivegrid.de to node1
|
||||
# while the record is DNS-only until Caddy has issued the certificate.
|
||||
|
||||
captcha-staging.hivegrid.de {
|
||||
encode zstd gzip
|
||||
reverse_proxy 127.0.0.1:7300
|
||||
}
|
||||
@@ -0,0 +1,136 @@
|
||||
# Self-hosted Cap
|
||||
|
||||
Cap is the replacement candidate for mCaptcha. It keeps the same HiveGrid
|
||||
principle as mCaptcha: self-hosted, no Google/Cloudflare CAPTCHA dependency,
|
||||
and no third-party widget CDN. The app loads `/assets/widget.js` and WASM from
|
||||
the Cap instance itself.
|
||||
|
||||
## Domain requirement
|
||||
|
||||
Cap needs a public HTTPS URL. The browser calls the Cap instance to generate and
|
||||
solve challenges, and the app server calls the same instance for `siteverify`.
|
||||
A raw private IP is not a good production setup because browser TLS, CORS,
|
||||
Cloudflare proxying, and asset loading all depend on a stable public origin.
|
||||
|
||||
Recommended hostnames:
|
||||
|
||||
- Staging: `https://captcha-staging.hivegrid.de` on node1 (`176.9.111.179`)
|
||||
- Production: `https://captcha.hivegrid.de` on `hivegrid-prod-worker`
|
||||
|
||||
## Staging on node1
|
||||
|
||||
node1 runs Cap as a plain docker-compose stack and exposes it through the
|
||||
existing Coolify Traefik proxy (`coolify-proxy`). The compose service also
|
||||
joins the external `coolify` network with the stable alias `cap-staging`, so
|
||||
Traefik can reach it without publishing Cap directly to the public interface.
|
||||
|
||||
1. Create DNS `captcha-staging.hivegrid.de -> 176.9.111.179`, DNS-only until
|
||||
Traefik has issued the certificate.
|
||||
2. Copy `docker-compose.node1.yml` and `.env.example` to `/opt/cap/` on node1.
|
||||
3. Create `/opt/cap/.env` from `.env.example`, set `ADMIN_KEY`, and set:
|
||||
|
||||
```env
|
||||
CORS_ORIGIN=https://staging-app.hivegrid.de,https://staging-api.hivegrid.de,https://staging.hivegrid.de
|
||||
```
|
||||
|
||||
4. Start Cap:
|
||||
|
||||
```sh
|
||||
docker compose -f docker-compose.node1.yml up -d
|
||||
```
|
||||
|
||||
5. Install the Traefik dynamic config on node1, then restart the proxy:
|
||||
|
||||
```sh
|
||||
cp traefik-staging.yaml /data/coolify/proxy/dynamic/cap-staging.yaml
|
||||
docker restart coolify-proxy
|
||||
```
|
||||
|
||||
6. Open `https://captcha-staging.hivegrid.de`, log in with `ADMIN_KEY`, and
|
||||
create site keys for `login`, `forms`, and `booking`. Keep each public
|
||||
sitekey and its matching site secret.
|
||||
|
||||
7. Set the staging web app env:
|
||||
|
||||
```env
|
||||
CAPTCHA_PROVIDER=cap
|
||||
CAP_INSTANCE_URL=https://captcha-staging.hivegrid.de
|
||||
CAP_SITEKEY_LOGIN=<login sitekey>
|
||||
CAP_SITE_SECRET_LOGIN=<login secret>
|
||||
CAP_SITEKEY_FORMS=<forms sitekey>
|
||||
CAP_SITE_SECRET_FORMS=<forms secret>
|
||||
CAP_SITEKEY_BOOKING=<booking sitekey>
|
||||
CAP_SITE_SECRET_BOOKING=<booking secret>
|
||||
CAPTCHA_FAIL_OPEN=true
|
||||
```
|
||||
|
||||
## Production on worker/services VM
|
||||
|
||||
Production should run as a Coolify compose service on `hivegrid-prod-worker`
|
||||
(`49.13.24.190`, private `10.0.0.5`) with FQDN `captcha.hivegrid.de`.
|
||||
|
||||
1. In Coolify, create a Docker Compose service from
|
||||
`infra/cap/docker-compose.coolify.yml`.
|
||||
2. Set `SERVICE_FQDN_CAP_3000=captcha.hivegrid.de`.
|
||||
3. Create the service `.env` from `.env.example` and set:
|
||||
|
||||
```env
|
||||
CORS_ORIGIN=https://app.hivegrid.de,https://admin.hivegrid.de,https://api.hivegrid.de
|
||||
```
|
||||
|
||||
4. Deploy and verify `https://captcha.hivegrid.de` returns the Cap dashboard.
|
||||
5. Create the same three site keys (`login`, `forms`, `booking`).
|
||||
6. Set the production web env to the `CAP_*` values and redeploy web.
|
||||
7. Keep the old `MCAPTCHA_*` values available until the Cap rollout is verified.
|
||||
Rollback is `CAPTCHA_PROVIDER=mcaptcha` plus a web redeploy.
|
||||
|
||||
## App integration contract
|
||||
|
||||
The web app reads:
|
||||
|
||||
```env
|
||||
CAPTCHA_PROVIDER=cap
|
||||
CAP_INSTANCE_URL=https://...
|
||||
CAP_SITEKEY_LOGIN=...
|
||||
CAP_SITE_SECRET_LOGIN=...
|
||||
CAP_SITEKEY_FORMS=...
|
||||
CAP_SITE_SECRET_FORMS=...
|
||||
CAP_SITEKEY_BOOKING=...
|
||||
CAP_SITE_SECRET_BOOKING=...
|
||||
CAPTCHA_FAIL_OPEN=true
|
||||
```
|
||||
|
||||
When Cap is active, the UI renders:
|
||||
|
||||
```html
|
||||
<cap-widget data-cap-api-endpoint="https://<cap-host>/<sitekey>/"></cap-widget>
|
||||
```
|
||||
|
||||
The widget injects `cap-token` into the enclosing form. Server-side verification
|
||||
uses:
|
||||
|
||||
```http
|
||||
POST https://<cap-host>/<sitekey>/siteverify
|
||||
Content-Type: application/json
|
||||
|
||||
{ "secret": "<site secret>", "response": "<cap-token>" }
|
||||
```
|
||||
|
||||
The expected valid response is `{ "success": true }`.
|
||||
|
||||
`Caddyfile.staging.snippet` is kept as a fallback for a non-Coolify node1
|
||||
setup. The active staging setup uses `traefik-staging.yaml`.
|
||||
|
||||
## Cutover checklist
|
||||
|
||||
- Staging Cap dashboard reachable over HTTPS.
|
||||
- `/assets/widget.js` and `/assets/cap_wasm_bg.wasm` return 200 from the Cap
|
||||
host.
|
||||
- Staging web env uses `CAPTCHA_PROVIDER=cap`.
|
||||
- Register/login captcha path verifies with a real browser.
|
||||
- Public form with captcha enabled submits successfully once and rejects token
|
||||
replay.
|
||||
- Production Cap deploy is healthy behind Cloudflare.
|
||||
- Production web deploy completed after `CAP_*` env is set.
|
||||
- mCaptcha rollback env remains available until Cap has run through one live
|
||||
deploy cycle.
|
||||
@@ -0,0 +1,35 @@
|
||||
# Self-hosted Cap Standalone for HiveGrid production.
|
||||
#
|
||||
# Intended as a Coolify Docker Compose service on hivegrid-prod-worker
|
||||
# (worker/services VM). Coolify/Traefik terminates TLS for captcha.hivegrid.de.
|
||||
|
||||
name: hivegrid-prod-cap
|
||||
|
||||
volumes:
|
||||
cap-valkey-data:
|
||||
|
||||
services:
|
||||
cap:
|
||||
image: ${CAP_IMAGE:-tiago2/cap:3.1.5}
|
||||
restart: unless-stopped
|
||||
env_file: [.env]
|
||||
expose:
|
||||
- "3000"
|
||||
environment:
|
||||
# Coolify generates the Traefik route for captcha.hivegrid.de from this.
|
||||
- SERVICE_FQDN_CAP_3000=${SERVICE_FQDN_CAP_3000:-captcha.hivegrid.de}
|
||||
depends_on:
|
||||
valkey:
|
||||
condition: service_healthy
|
||||
|
||||
valkey:
|
||||
image: valkey/valkey:9-alpine
|
||||
restart: unless-stopped
|
||||
volumes:
|
||||
- cap-valkey-data:/data
|
||||
command: valkey-server --save 60 1 --loglevel warning --maxmemory-policy noeviction
|
||||
healthcheck:
|
||||
test: ["CMD", "valkey-cli", "ping"]
|
||||
interval: 5s
|
||||
timeout: 3s
|
||||
retries: 10
|
||||
@@ -0,0 +1,46 @@
|
||||
# Self-hosted Cap Standalone for staging/node1.
|
||||
#
|
||||
# node1 runs this as a plain docker-compose stack behind the existing
|
||||
# Coolify Traefik proxy. The Cap container joins the external coolify network
|
||||
# as cap-staging; the host port is kept on localhost for direct diagnostics.
|
||||
|
||||
name: cap
|
||||
|
||||
networks:
|
||||
cap-net:
|
||||
name: cap-net
|
||||
coolify:
|
||||
external: true
|
||||
|
||||
volumes:
|
||||
cap-valkey-data:
|
||||
name: cap-valkey-data
|
||||
|
||||
services:
|
||||
cap:
|
||||
image: ${CAP_IMAGE:-tiago2/cap:3.1.5}
|
||||
restart: unless-stopped
|
||||
networks:
|
||||
cap-net:
|
||||
coolify:
|
||||
aliases:
|
||||
- cap-staging
|
||||
env_file: [.env]
|
||||
ports:
|
||||
- "127.0.0.1:7300:3000"
|
||||
depends_on:
|
||||
valkey:
|
||||
condition: service_healthy
|
||||
|
||||
valkey:
|
||||
image: valkey/valkey:9-alpine
|
||||
restart: unless-stopped
|
||||
networks: [cap-net]
|
||||
volumes:
|
||||
- cap-valkey-data:/data
|
||||
command: valkey-server --save 60 1 --loglevel warning --maxmemory-policy noeviction
|
||||
healthcheck:
|
||||
test: ["CMD", "valkey-cli", "ping"]
|
||||
interval: 5s
|
||||
timeout: 3s
|
||||
retries: 10
|
||||
@@ -0,0 +1,33 @@
|
||||
http:
|
||||
routers:
|
||||
cap-staging-secure:
|
||||
rule: "Host(`captcha-staging.hivegrid.de`)"
|
||||
entryPoints:
|
||||
- https
|
||||
service: cap-staging
|
||||
middlewares:
|
||||
- cap-staging-security-headers
|
||||
tls:
|
||||
certResolver: letsencrypt
|
||||
cap-staging-web:
|
||||
rule: "Host(`captcha-staging.hivegrid.de`)"
|
||||
entryPoints:
|
||||
- http
|
||||
service: cap-staging
|
||||
middlewares:
|
||||
- cap-staging-https
|
||||
services:
|
||||
cap-staging:
|
||||
loadBalancer:
|
||||
servers:
|
||||
- url: "http://cap-staging:3000"
|
||||
middlewares:
|
||||
cap-staging-https:
|
||||
redirectScheme:
|
||||
scheme: https
|
||||
cap-staging-security-headers:
|
||||
headers:
|
||||
customResponseHeaders:
|
||||
X-Robots-Tag: "noindex, nofollow"
|
||||
contentTypeNosniff: true
|
||||
referrerPolicy: "no-referrer"
|
||||
+7
-7
@@ -1,7 +1,7 @@
|
||||
{
|
||||
"uid": "boilerplate-ai-cost",
|
||||
"title": "Boilerplate — AI Cost per Tenant",
|
||||
"tags": ["boilerplate", "ai"],
|
||||
"uid": "hivegrid-ai-cost",
|
||||
"title": "HiveGrid — AI Cost per Tenant",
|
||||
"tags": ["hivegrid", "ai"],
|
||||
"timezone": "browser",
|
||||
"schemaVersion": 39,
|
||||
"refresh": "1m",
|
||||
@@ -17,7 +17,7 @@
|
||||
"refId": "A",
|
||||
"expr": "topk(20, sum by (tenant_bucket) (increase(ai_tokens_consumed_total[24h])))",
|
||||
"legendFormat": "{{tenant_bucket}}",
|
||||
"datasource": { "uid": "boilerplate-prometheus", "type": "prometheus" }
|
||||
"datasource": { "uid": "hivegrid-prometheus", "type": "prometheus" }
|
||||
}
|
||||
]
|
||||
},
|
||||
@@ -27,7 +27,7 @@
|
||||
"title": "Tokens by provider (24h)",
|
||||
"gridPos": { "h": 10, "w": 12, "x": 12, "y": 0 },
|
||||
"targets": [
|
||||
{ "refId": "A", "expr": "sum by (provider) (increase(ai_tokens_consumed_total[24h]))", "legendFormat": "{{provider}}", "datasource": { "uid": "boilerplate-prometheus", "type": "prometheus" } }
|
||||
{ "refId": "A", "expr": "sum by (provider) (increase(ai_tokens_consumed_total[24h]))", "legendFormat": "{{provider}}", "datasource": { "uid": "hivegrid-prometheus", "type": "prometheus" } }
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -36,7 +36,7 @@
|
||||
"title": "Daily token-consumption trend (per model)",
|
||||
"gridPos": { "h": 10, "w": 24, "x": 0, "y": 10 },
|
||||
"targets": [
|
||||
{ "refId": "A", "expr": "sum by (model) (rate(ai_tokens_consumed_total[1h]))", "legendFormat": "{{model}}", "datasource": { "uid": "boilerplate-prometheus", "type": "prometheus" } }
|
||||
{ "refId": "A", "expr": "sum by (model) (rate(ai_tokens_consumed_total[1h]))", "legendFormat": "{{model}}", "datasource": { "uid": "hivegrid-prometheus", "type": "prometheus" } }
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -45,7 +45,7 @@
|
||||
"title": "Cardinality of tenant_bucket (should stay ≤ 101)",
|
||||
"gridPos": { "h": 4, "w": 6, "x": 0, "y": 20 },
|
||||
"targets": [
|
||||
{ "refId": "A", "expr": "count(count by (tenant_bucket) (ai_tokens_consumed_total))", "datasource": { "uid": "boilerplate-prometheus", "type": "prometheus" } }
|
||||
{ "refId": "A", "expr": "count(count by (tenant_bucket) (ai_tokens_consumed_total))", "datasource": { "uid": "hivegrid-prometheus", "type": "prometheus" } }
|
||||
],
|
||||
"fieldConfig": { "defaults": { "thresholds": { "mode": "absolute", "steps": [{ "color": "green", "value": null }, { "color": "yellow", "value": 80 }, { "color": "red", "value": 101 }] } } }
|
||||
}
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
{
|
||||
"uid": "boilerplate-api-latency",
|
||||
"title": "Boilerplate — API Latency",
|
||||
"tags": ["boilerplate", "api"],
|
||||
"uid": "hivegrid-api-latency",
|
||||
"title": "HiveGrid — API Latency",
|
||||
"tags": ["hivegrid", "api"],
|
||||
"timezone": "browser",
|
||||
"schemaVersion": 39,
|
||||
"refresh": "30s",
|
||||
@@ -13,9 +13,9 @@
|
||||
"title": "p50 / p95 / p99 (s) per route",
|
||||
"gridPos": { "h": 10, "w": 24, "x": 0, "y": 0 },
|
||||
"targets": [
|
||||
{ "refId": "A", "expr": "histogram_quantile(0.50, sum by (le, route) (rate(http_request_duration_seconds_bucket{service=\"api\"}[5m])))", "legendFormat": "p50 {{route}}", "datasource": { "uid": "boilerplate-prometheus", "type": "prometheus" } },
|
||||
{ "refId": "B", "expr": "histogram_quantile(0.95, sum by (le, route) (rate(http_request_duration_seconds_bucket{service=\"api\"}[5m])))", "legendFormat": "p95 {{route}}", "datasource": { "uid": "boilerplate-prometheus", "type": "prometheus" } },
|
||||
{ "refId": "C", "expr": "histogram_quantile(0.99, sum by (le, route) (rate(http_request_duration_seconds_bucket{service=\"api\"}[5m])))", "legendFormat": "p99 {{route}}", "datasource": { "uid": "boilerplate-prometheus", "type": "prometheus" } }
|
||||
{ "refId": "A", "expr": "histogram_quantile(0.50, sum by (le, route) (rate(http_request_duration_seconds_bucket{service=\"api\"}[5m])))", "legendFormat": "p50 {{route}}", "datasource": { "uid": "hivegrid-prometheus", "type": "prometheus" } },
|
||||
{ "refId": "B", "expr": "histogram_quantile(0.95, sum by (le, route) (rate(http_request_duration_seconds_bucket{service=\"api\"}[5m])))", "legendFormat": "p95 {{route}}", "datasource": { "uid": "hivegrid-prometheus", "type": "prometheus" } },
|
||||
{ "refId": "C", "expr": "histogram_quantile(0.99, sum by (le, route) (rate(http_request_duration_seconds_bucket{service=\"api\"}[5m])))", "legendFormat": "p99 {{route}}", "datasource": { "uid": "hivegrid-prometheus", "type": "prometheus" } }
|
||||
],
|
||||
"fieldConfig": { "defaults": { "unit": "s" } }
|
||||
},
|
||||
@@ -25,7 +25,7 @@
|
||||
"title": "Request rate per route",
|
||||
"gridPos": { "h": 8, "w": 12, "x": 0, "y": 10 },
|
||||
"targets": [
|
||||
{ "refId": "A", "expr": "sum by (route) (rate(http_requests_total{service=\"api\"}[1m]))", "legendFormat": "{{route}}", "datasource": { "uid": "boilerplate-prometheus", "type": "prometheus" } }
|
||||
{ "refId": "A", "expr": "sum by (route) (rate(http_requests_total{service=\"api\"}[1m]))", "legendFormat": "{{route}}", "datasource": { "uid": "hivegrid-prometheus", "type": "prometheus" } }
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -34,7 +34,7 @@
|
||||
"title": "Status-code distribution",
|
||||
"gridPos": { "h": 8, "w": 12, "x": 12, "y": 10 },
|
||||
"targets": [
|
||||
{ "refId": "A", "expr": "sum by (status) (rate(http_requests_total{service=\"api\"}[1m]))", "legendFormat": "{{status}}", "datasource": { "uid": "boilerplate-prometheus", "type": "prometheus" } }
|
||||
{ "refId": "A", "expr": "sum by (status) (rate(http_requests_total{service=\"api\"}[1m]))", "legendFormat": "{{status}}", "datasource": { "uid": "hivegrid-prometheus", "type": "prometheus" } }
|
||||
]
|
||||
}
|
||||
]
|
||||
|
||||
+10
-10
@@ -1,7 +1,7 @@
|
||||
{
|
||||
"uid": "boilerplate-business-kpi",
|
||||
"title": "Boilerplate — Business KPIs",
|
||||
"tags": ["boilerplate", "business"],
|
||||
"uid": "hivegrid-business-kpi",
|
||||
"title": "HiveGrid — Business KPIs",
|
||||
"tags": ["hivegrid", "business"],
|
||||
"timezone": "browser",
|
||||
"schemaVersion": 39,
|
||||
"refresh": "1m",
|
||||
@@ -13,7 +13,7 @@
|
||||
"title": "Active tenants",
|
||||
"gridPos": { "h": 4, "w": 6, "x": 0, "y": 0 },
|
||||
"targets": [
|
||||
{ "refId": "A", "expr": "active_tenants", "datasource": { "uid": "boilerplate-prometheus", "type": "prometheus" } }
|
||||
{ "refId": "A", "expr": "active_tenants", "datasource": { "uid": "hivegrid-prometheus", "type": "prometheus" } }
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -22,16 +22,16 @@
|
||||
"title": "Active users (24h)",
|
||||
"gridPos": { "h": 4, "w": 6, "x": 6, "y": 0 },
|
||||
"targets": [
|
||||
{ "refId": "A", "expr": "active_users_24h", "datasource": { "uid": "boilerplate-prometheus", "type": "prometheus" } }
|
||||
{ "refId": "A", "expr": "active_users_24h", "datasource": { "uid": "hivegrid-prometheus", "type": "prometheus" } }
|
||||
]
|
||||
},
|
||||
{
|
||||
"id": 3,
|
||||
"type": "timeseries",
|
||||
"title": "Stripe webhook events / min by type",
|
||||
"title": "Billing webhook events / min by type",
|
||||
"gridPos": { "h": 8, "w": 12, "x": 0, "y": 4 },
|
||||
"targets": [
|
||||
{ "refId": "A", "expr": "sum by (event_type) (rate(stripe_webhook_events_total[5m]))", "legendFormat": "{{event_type}}", "datasource": { "uid": "boilerplate-prometheus", "type": "prometheus" } }
|
||||
{ "refId": "A", "expr": "sum by (event_type) (rate(billing_provider_webhook_events_total[5m]))", "legendFormat": "{{event_type}}", "datasource": { "uid": "hivegrid-prometheus", "type": "prometheus" } }
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -40,7 +40,7 @@
|
||||
"title": "Mail delivery rate by status",
|
||||
"gridPos": { "h": 8, "w": 12, "x": 12, "y": 4 },
|
||||
"targets": [
|
||||
{ "refId": "A", "expr": "sum by (status) (rate(mail_sent_total[5m]))", "legendFormat": "{{status}}", "datasource": { "uid": "boilerplate-prometheus", "type": "prometheus" } }
|
||||
{ "refId": "A", "expr": "sum by (status) (rate(mail_sent_total[5m]))", "legendFormat": "{{status}}", "datasource": { "uid": "hivegrid-prometheus", "type": "prometheus" } }
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -49,7 +49,7 @@
|
||||
"title": "Outbound webhook attempts by status",
|
||||
"gridPos": { "h": 8, "w": 12, "x": 0, "y": 12 },
|
||||
"targets": [
|
||||
{ "refId": "A", "expr": "sum by (status) (rate(webhook_delivery_attempts_total[5m]))", "legendFormat": "{{status}}", "datasource": { "uid": "boilerplate-prometheus", "type": "prometheus" } }
|
||||
{ "refId": "A", "expr": "sum by (status) (rate(webhook_delivery_attempts_total[5m]))", "legendFormat": "{{status}}", "datasource": { "uid": "hivegrid-prometheus", "type": "prometheus" } }
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -58,7 +58,7 @@
|
||||
"title": "Active subscriptions by plan",
|
||||
"gridPos": { "h": 8, "w": 12, "x": 12, "y": 12 },
|
||||
"targets": [
|
||||
{ "refId": "A", "expr": "subscription_active", "legendFormat": "{{plan_id}}", "datasource": { "uid": "boilerplate-prometheus", "type": "prometheus" } }
|
||||
{ "refId": "A", "expr": "subscription_active", "legendFormat": "{{plan_id}}", "datasource": { "uid": "hivegrid-prometheus", "type": "prometheus" } }
|
||||
]
|
||||
}
|
||||
]
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
{
|
||||
"uid": "boilerplate-overview",
|
||||
"title": "Boilerplate — Overview",
|
||||
"tags": ["boilerplate", "overview"],
|
||||
"uid": "hivegrid-overview",
|
||||
"title": "HiveGrid — Overview",
|
||||
"tags": ["hivegrid", "overview"],
|
||||
"timezone": "browser",
|
||||
"schemaVersion": 39,
|
||||
"refresh": "30s",
|
||||
@@ -13,7 +13,7 @@
|
||||
"title": "Services up",
|
||||
"gridPos": { "h": 4, "w": 6, "x": 0, "y": 0 },
|
||||
"targets": [
|
||||
{ "refId": "A", "expr": "count(up{job=~\"boilerplate-.*\"} == 1)", "datasource": { "uid": "boilerplate-prometheus", "type": "prometheus" } }
|
||||
{ "refId": "A", "expr": "count(up{job=~\"hivegrid-.*\"} == 1)", "datasource": { "uid": "hivegrid-prometheus", "type": "prometheus" } }
|
||||
],
|
||||
"fieldConfig": { "defaults": { "mappings": [], "thresholds": { "mode": "absolute", "steps": [{ "color": "red", "value": null }, { "color": "green", "value": 3 }] } } }
|
||||
},
|
||||
@@ -23,7 +23,7 @@
|
||||
"title": "HTTP request rate (per service)",
|
||||
"gridPos": { "h": 8, "w": 12, "x": 6, "y": 0 },
|
||||
"targets": [
|
||||
{ "refId": "A", "expr": "sum by (service) (rate(http_requests_total[1m]))", "legendFormat": "{{service}}", "datasource": { "uid": "boilerplate-prometheus", "type": "prometheus" } }
|
||||
{ "refId": "A", "expr": "sum by (service) (rate(http_requests_total[1m]))", "legendFormat": "{{service}}", "datasource": { "uid": "hivegrid-prometheus", "type": "prometheus" } }
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -36,7 +36,7 @@
|
||||
"refId": "A",
|
||||
"expr": "sum by (service) (rate(http_requests_total{status=~\"5..\"}[5m])) / clamp_min(sum by (service) (rate(http_requests_total[5m])), 0.001)",
|
||||
"legendFormat": "{{service}}",
|
||||
"datasource": { "uid": "boilerplate-prometheus", "type": "prometheus" }
|
||||
"datasource": { "uid": "hivegrid-prometheus", "type": "prometheus" }
|
||||
}
|
||||
],
|
||||
"fieldConfig": { "defaults": { "unit": "percentunit", "max": 0.5 } }
|
||||
@@ -51,7 +51,7 @@
|
||||
"refId": "A",
|
||||
"expr": "histogram_quantile(0.95, sum by (le, service) (rate(http_request_duration_seconds_bucket[5m])))",
|
||||
"legendFormat": "{{service}}",
|
||||
"datasource": { "uid": "boilerplate-prometheus", "type": "prometheus" }
|
||||
"datasource": { "uid": "hivegrid-prometheus", "type": "prometheus" }
|
||||
}
|
||||
],
|
||||
"fieldConfig": { "defaults": { "unit": "s" } }
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
{
|
||||
"uid": "boilerplate-queue-health",
|
||||
"title": "Boilerplate — Queue Health",
|
||||
"tags": ["boilerplate", "queue"],
|
||||
"uid": "hivegrid-queue-health",
|
||||
"title": "HiveGrid — Queue Health",
|
||||
"tags": ["hivegrid", "queue"],
|
||||
"timezone": "browser",
|
||||
"schemaVersion": 39,
|
||||
"refresh": "30s",
|
||||
@@ -13,7 +13,7 @@
|
||||
"title": "Active jobs (gauge) per queue",
|
||||
"gridPos": { "h": 8, "w": 12, "x": 0, "y": 0 },
|
||||
"targets": [
|
||||
{ "refId": "A", "expr": "queue_jobs_active", "legendFormat": "{{queue}}", "datasource": { "uid": "boilerplate-prometheus", "type": "prometheus" } }
|
||||
{ "refId": "A", "expr": "queue_jobs_active", "legendFormat": "{{queue}}", "datasource": { "uid": "hivegrid-prometheus", "type": "prometheus" } }
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -22,7 +22,7 @@
|
||||
"title": "Waiting jobs (gauge) per queue",
|
||||
"gridPos": { "h": 8, "w": 12, "x": 12, "y": 0 },
|
||||
"targets": [
|
||||
{ "refId": "A", "expr": "queue_jobs_waiting", "legendFormat": "{{queue}}", "datasource": { "uid": "boilerplate-prometheus", "type": "prometheus" } }
|
||||
{ "refId": "A", "expr": "queue_jobs_waiting", "legendFormat": "{{queue}}", "datasource": { "uid": "hivegrid-prometheus", "type": "prometheus" } }
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -31,7 +31,7 @@
|
||||
"title": "Completed rate per queue",
|
||||
"gridPos": { "h": 8, "w": 12, "x": 0, "y": 8 },
|
||||
"targets": [
|
||||
{ "refId": "A", "expr": "sum by (queue) (rate(queue_jobs_completed_total[5m]))", "legendFormat": "{{queue}}", "datasource": { "uid": "boilerplate-prometheus", "type": "prometheus" } }
|
||||
{ "refId": "A", "expr": "sum by (queue) (rate(queue_jobs_completed_total[5m]))", "legendFormat": "{{queue}}", "datasource": { "uid": "hivegrid-prometheus", "type": "prometheus" } }
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -40,7 +40,7 @@
|
||||
"title": "Failed rate per queue (by error_type)",
|
||||
"gridPos": { "h": 8, "w": 12, "x": 12, "y": 8 },
|
||||
"targets": [
|
||||
{ "refId": "A", "expr": "sum by (queue, error_type) (rate(queue_jobs_failed_total[5m]))", "legendFormat": "{{queue}} / {{error_type}}", "datasource": { "uid": "boilerplate-prometheus", "type": "prometheus" } }
|
||||
{ "refId": "A", "expr": "sum by (queue, error_type) (rate(queue_jobs_failed_total[5m]))", "legendFormat": "{{queue}} / {{error_type}}", "datasource": { "uid": "hivegrid-prometheus", "type": "prometheus" } }
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -49,7 +49,7 @@
|
||||
"title": "Job duration heatmap",
|
||||
"gridPos": { "h": 10, "w": 24, "x": 0, "y": 16 },
|
||||
"targets": [
|
||||
{ "refId": "A", "expr": "sum by (le) (rate(queue_job_duration_seconds_bucket[5m]))", "format": "heatmap", "datasource": { "uid": "boilerplate-prometheus", "type": "prometheus" } }
|
||||
{ "refId": "A", "expr": "sum by (le) (rate(queue_job_duration_seconds_bucket[5m]))", "format": "heatmap", "datasource": { "uid": "hivegrid-prometheus", "type": "prometheus" } }
|
||||
]
|
||||
}
|
||||
]
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
# Grafana stack for the boilerplate.
|
||||
# Grafana stack for HiveGrid.
|
||||
#
|
||||
# Deploys alongside Prometheus on the Hetzner observability box. Joins the
|
||||
# same docker network so Grafana can reach Prometheus via DNS
|
||||
@@ -7,7 +7,7 @@
|
||||
services:
|
||||
grafana:
|
||||
image: grafana/grafana-oss:11.4.0
|
||||
container_name: boilerplate-grafana
|
||||
container_name: hivegrid-grafana
|
||||
restart: unless-stopped
|
||||
user: "472:472"
|
||||
environment:
|
||||
@@ -17,7 +17,7 @@ services:
|
||||
# Disable anonymous access — Business / per-tenant data lives in here.
|
||||
GF_AUTH_ANONYMOUS_ENABLED: "false"
|
||||
GF_AUTH_DISABLE_LOGIN_FORM: "false"
|
||||
GF_SERVER_ROOT_URL: ${GRAFANA_ROOT_URL:-https://grafana.boiler.saas-ctrl.com}
|
||||
GF_SERVER_ROOT_URL: ${GRAFANA_ROOT_URL:-https://grafana.saas-ctrl.com}
|
||||
# Prometheus token used by the provisioned datasource. The yaml
|
||||
# references it via env interpolation.
|
||||
GRAFANA_PROMETHEUS_TOKEN: ${GRAFANA_PROMETHEUS_TOKEN:?GRAFANA_PROMETHEUS_TOKEN must be set}
|
||||
@@ -42,9 +42,9 @@ services:
|
||||
|
||||
networks:
|
||||
observability:
|
||||
name: boilerplate-observability
|
||||
name: hivegrid-observability
|
||||
external: true
|
||||
|
||||
volumes:
|
||||
grafana-data:
|
||||
name: boilerplate-grafana-data
|
||||
name: hivegrid-grafana-data
|
||||
+2
-2
@@ -1,9 +1,9 @@
|
||||
apiVersion: 1
|
||||
|
||||
providers:
|
||||
- name: boilerplate
|
||||
- name: hivegrid
|
||||
orgId: 1
|
||||
folder: Boilerplate
|
||||
folder: HiveGrid
|
||||
type: file
|
||||
disableDeletion: true
|
||||
updateIntervalSeconds: 30
|
||||
|
||||
+1
-1
@@ -3,7 +3,7 @@ apiVersion: 1
|
||||
datasources:
|
||||
- name: Prometheus
|
||||
type: prometheus
|
||||
uid: boilerplate-prometheus
|
||||
uid: hivegrid-prometheus
|
||||
access: proxy
|
||||
url: http://prometheus:9090
|
||||
isDefault: true
|
||||
|
||||
@@ -54,7 +54,7 @@ CAPTCHA_FAIL_OPEN=true
|
||||
|
||||
Verify flow: client solves the PoW widget → token → app server calls
|
||||
`POST {MCAPTCHA_INSTANCE_URL}/api/v1/pow/siteverify` with `{ secret, token, key }`
|
||||
(handled by `@toolctrl/captcha`).
|
||||
(handled by `@hivegrid/captcha`).
|
||||
|
||||
## Fail policy
|
||||
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
; PgBouncer config for boilerplate.
|
||||
; PgBouncer config for HiveGrid.
|
||||
;
|
||||
; Mounted at /etc/pgbouncer/pgbouncer.ini inside the edoburu/pgbouncer container.
|
||||
; See docs/ops/pgbouncer.md for tuning rationale and rollback.
|
||||
@@ -10,7 +10,7 @@
|
||||
; to Postgres) — see schema.prisma `directUrl`.
|
||||
|
||||
[databases]
|
||||
boilerplate = host=postgres port=5432 dbname=boilerplate_validation
|
||||
hivegrid = host=postgres port=5432 dbname=hivegrid_validation
|
||||
|
||||
[pgbouncer]
|
||||
listen_addr = 0.0.0.0
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# Prometheus — Boilerplate
|
||||
# Prometheus — HiveGrid
|
||||
|
||||
Scrapes `/metrics` from the three boilerplate apps every 15s.
|
||||
Scrapes `/metrics` from the three HiveGrid apps every 15s.
|
||||
|
||||
## Files
|
||||
|
||||
@@ -17,13 +17,13 @@ openssl rand -hex 32 > secrets/metrics_token
|
||||
chmod 600 secrets/metrics_token
|
||||
|
||||
# 2. Create the shared docker network (also used by Grafana)
|
||||
docker network create boilerplate-observability
|
||||
docker network create hivegrid-observability
|
||||
|
||||
# 3. Start the stack
|
||||
docker compose up -d
|
||||
```
|
||||
|
||||
Then expose `prometheus.boiler.saas-ctrl.com` via Coolify's proxy with
|
||||
Then expose `prometheus.saas-ctrl.com` via Coolify's proxy with
|
||||
**basic-auth or Cloudflare-Access** in front — Prometheus has no
|
||||
built-in auth and ships internal state.
|
||||
|
||||
@@ -31,10 +31,10 @@ built-in auth and ships internal state.
|
||||
|
||||
```sh
|
||||
curl -s -H "Authorization: Bearer $(cat secrets/metrics_token)" \
|
||||
https://api.boiler.saas-ctrl.com/metrics | head -20
|
||||
https://api.hivegrid.de/metrics | head -20
|
||||
```
|
||||
|
||||
The Prometheus UI at `https://prometheus.boiler.saas-ctrl.com/targets`
|
||||
The Prometheus UI at `https://prometheus.saas-ctrl.com/targets`
|
||||
should list all three jobs as **UP**.
|
||||
|
||||
## Cardinality budget
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
# Prometheus stack for the boilerplate.
|
||||
# Prometheus stack for HiveGrid.
|
||||
#
|
||||
# Deploy as a Coolify "Docker Compose" service or with vanilla
|
||||
# `docker compose up -d` on the Hetzner observability box. The bind mount
|
||||
@@ -15,7 +15,7 @@
|
||||
services:
|
||||
prometheus:
|
||||
image: prom/prometheus:v3.6.0
|
||||
container_name: boilerplate-prometheus
|
||||
container_name: hivegrid-prometheus
|
||||
restart: unless-stopped
|
||||
user: "65534:65534"
|
||||
command:
|
||||
@@ -25,7 +25,7 @@ services:
|
||||
- --web.console.libraries=/usr/share/prometheus/console_libraries
|
||||
- --web.console.templates=/usr/share/prometheus/consoles
|
||||
- --web.enable-lifecycle
|
||||
- --web.external-url=https://prometheus.boiler.saas-ctrl.com
|
||||
- --web.external-url=https://prometheus.saas-ctrl.com
|
||||
volumes:
|
||||
- ./prometheus.yml:/etc/prometheus/prometheus.yml:ro
|
||||
- ./secrets:/etc/prometheus/secrets:ro
|
||||
@@ -37,9 +37,9 @@ services:
|
||||
|
||||
networks:
|
||||
observability:
|
||||
name: boilerplate-observability
|
||||
name: hivegrid-observability
|
||||
external: true
|
||||
|
||||
volumes:
|
||||
prometheus-data:
|
||||
name: boilerplate-prometheus-data
|
||||
name: hivegrid-prometheus-data
|
||||
@@ -1,4 +1,4 @@
|
||||
# Prometheus scrape configuration for the boilerplate.
|
||||
# Prometheus scrape configuration for HiveGrid.
|
||||
#
|
||||
# Scrapes the three application processes' /metrics endpoints with
|
||||
# bearer-token authentication. METRICS_TOKEN is injected via the
|
||||
@@ -17,22 +17,22 @@ global:
|
||||
scrape_timeout: 10s
|
||||
evaluation_interval: 15s
|
||||
external_labels:
|
||||
cluster: boilerplate
|
||||
cluster: hivegrid
|
||||
environment: staging
|
||||
|
||||
scrape_configs:
|
||||
- job_name: boilerplate-api
|
||||
- job_name: hivegrid-api
|
||||
metrics_path: /metrics
|
||||
scheme: https
|
||||
authorization:
|
||||
type: Bearer
|
||||
credentials_file: /etc/prometheus/secrets/metrics_token
|
||||
static_configs:
|
||||
- targets: ["api.boiler.saas-ctrl.com"]
|
||||
- targets: ["api.hivegrid.de"]
|
||||
labels:
|
||||
service: api
|
||||
|
||||
- job_name: boilerplate-worker
|
||||
- job_name: hivegrid-worker
|
||||
metrics_path: /metrics
|
||||
scheme: http
|
||||
authorization:
|
||||
@@ -48,7 +48,7 @@ scrape_configs:
|
||||
# and use the worker container's stable alias (in Coolify v4
|
||||
# the UUID-based generated name changes on every redeploy
|
||||
# so you must either add a docker-compose override that
|
||||
# sets `networks.<net>.aliases: [boilerplate-worker]` or
|
||||
# sets `networks.<net>.aliases: [hivegrid-worker]` or
|
||||
# use Coolify's "Internal-only" service feature).
|
||||
# (b) Expose the worker admin server through Traefik on a
|
||||
# dedicated hostname (e.g. worker-admin.<domain>) with a
|
||||
@@ -58,11 +58,11 @@ scrape_configs:
|
||||
# Until (a) or (b) is wired the worker job stays DOWN; api+web
|
||||
# cover the HTTP-side metrics, the per-queue queue_jobs_*
|
||||
# counters from the worker registry are not yet visible.
|
||||
- targets: ["worker.boilerplate.internal:9101"]
|
||||
- targets: ["worker.hivegrid.internal:9101"]
|
||||
labels:
|
||||
service: worker
|
||||
|
||||
- job_name: boilerplate-web
|
||||
- job_name: hivegrid-web
|
||||
metrics_path: /api/metrics
|
||||
scheme: https
|
||||
authorization:
|
||||
@@ -73,7 +73,7 @@ scrape_configs:
|
||||
# the admin host. Scraping the tenant-facing host returns 404 by
|
||||
# design. Use admin.<domain> for staging; in prod set to
|
||||
# admin.<prod-domain>.
|
||||
- targets: ["admin.boiler.saas-ctrl.com"]
|
||||
- targets: ["admin.hivegrid.de"]
|
||||
labels:
|
||||
service: web
|
||||
|
||||
|
||||
@@ -7,7 +7,7 @@
|
||||
# -----------------------------------------------------------------------------
|
||||
# Public URL — must match the Traefik-routed hostname in docker-compose.override.yml
|
||||
# -----------------------------------------------------------------------------
|
||||
SENTRY_BASE_URL=https://sentry.boiler.saas-ctrl.com
|
||||
SENTRY_BASE_URL=https://sentry.saas-ctrl.com
|
||||
|
||||
# -----------------------------------------------------------------------------
|
||||
# Admin / superuser bootstrap
|
||||
@@ -40,9 +40,9 @@ SENTRY_DSN_WEB=
|
||||
# Endpoint format examples:
|
||||
# AWS: https://s3.eu-central-1.amazonaws.com
|
||||
# Hetzner: https://nbg1.your-objectstorage.com
|
||||
# MinIO local: https://minio.boiler.saas-ctrl.com
|
||||
# MinIO local: https://minio.saas-ctrl.com
|
||||
SENTRY_S3_BACKUP_ENDPOINT=https://nbg1.your-objectstorage.com
|
||||
SENTRY_S3_BACKUP_BUCKET=boilerplate-sentry-backups
|
||||
SENTRY_S3_BACKUP_BUCKET=hivegrid-sentry-backups
|
||||
SENTRY_S3_BACKUP_ACCESS_KEY=replace-me
|
||||
SENTRY_S3_BACKUP_SECRET_KEY=replace-me
|
||||
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
# `infra/sentry` — Self-Hosted Sentry Stack
|
||||
|
||||
This directory contains the artifacts to deploy a **self-hosted Sentry** instance
|
||||
on the staging Hetzner host (`boilerplate-staging`, see
|
||||
on the staging Hetzner host (`hivegrid-staging`, see
|
||||
`docs/improvement-plan/archive/STAGING-ACCESS.md`) via the existing Coolify-managed
|
||||
Docker-Compose runtime.
|
||||
|
||||
@@ -25,7 +25,7 @@ it lacks full Performance / Replay / OTLP support.
|
||||
- Docker Engine ≥ 24 + Docker Compose v2 (provided by Coolify)
|
||||
- Traefik v3.6 reverse-proxy from `coolify-proxy` (Let's Encrypt ACME via
|
||||
`letsencrypt` cert resolver)
|
||||
- DNS-record `sentry.boiler.saas-ctrl.com` → `46.225.5.93`
|
||||
- DNS-record `sentry.saas-ctrl.com` → `46.225.5.93`
|
||||
|
||||
## Layout
|
||||
|
||||
@@ -56,7 +56,7 @@ runbook). High-level:
|
||||
1. Copy `.env.example` → `.env`, fill secrets (Coolify-Vault).
|
||||
2. Run `./install.sh` (clones upstream at pinned tag, applies overlays,
|
||||
installs).
|
||||
3. In Coolify, attach this stack to the `sentry.boiler.saas-ctrl.com` domain.
|
||||
3. In Coolify, attach this stack to the `sentry.saas-ctrl.com` domain.
|
||||
4. Smoke-test per `## Test Requirements` in
|
||||
`docs/improvement-plan/02-observability/sentry-selfhost-setup.md`.
|
||||
|
||||
|
||||
@@ -40,13 +40,13 @@ services:
|
||||
# its own labels; the ones below are the project-specific routing rules.
|
||||
- "traefik.enable=true"
|
||||
- "traefik.docker.network=coolify"
|
||||
- "traefik.http.routers.sentry.rule=Host(`sentry.boiler.saas-ctrl.com`)"
|
||||
- "traefik.http.routers.sentry.rule=Host(`sentry.saas-ctrl.com`)"
|
||||
- "traefik.http.routers.sentry.entrypoints=https"
|
||||
- "traefik.http.routers.sentry.tls=true"
|
||||
- "traefik.http.routers.sentry.tls.certresolver=letsencrypt"
|
||||
- "traefik.http.services.sentry.loadbalancer.server.port=80"
|
||||
# HTTP -> HTTPS redirect
|
||||
- "traefik.http.routers.sentry-http.rule=Host(`sentry.boiler.saas-ctrl.com`)"
|
||||
- "traefik.http.routers.sentry-http.rule=Host(`sentry.saas-ctrl.com`)"
|
||||
- "traefik.http.routers.sentry-http.entrypoints=http"
|
||||
- "traefik.http.routers.sentry-http.middlewares=sentry-https-redirect"
|
||||
- "traefik.http.middlewares.sentry-https-redirect.redirectscheme.scheme=https"
|
||||
|
||||
@@ -140,4 +140,4 @@ log "starting stack (detached)"
|
||||
)
|
||||
|
||||
log "done. Web UI: ${SENTRY_BASE_URL}"
|
||||
log "next: log in, enable 2FA on the admin account, then create projects 'boilerplate-api', 'boilerplate-worker', 'boilerplate-web' and paste their DSNs into Coolify secrets."
|
||||
log "next: log in, enable 2FA on the admin account, then create projects 'hivegrid-api', 'hivegrid-worker', 'hivegrid-web' and paste their DSNs into Coolify secrets."
|
||||
@@ -87,9 +87,9 @@ quotas:
|
||||
# in the Sentry UI ("Project Settings -> Client Keys -> Allowed Domains").
|
||||
# Listed here so operators have one searchable place to remember what should
|
||||
# be set:
|
||||
# - https://boiler.saas-ctrl.com
|
||||
# - https://www.boiler.saas-ctrl.com
|
||||
# - https://admin.boiler.saas-ctrl.com
|
||||
# - https://api.boiler.saas-ctrl.com
|
||||
# - https://hivegrid.de
|
||||
# - https://www.hivegrid.de
|
||||
# - https://admin.hivegrid.de
|
||||
# - https://api.hivegrid.de
|
||||
# Leave the array empty in the UI ONLY for the API and Worker projects (their
|
||||
# DSNs are never used from a browser).
|
||||
@@ -15,7 +15,7 @@ import os
|
||||
# ---------------------------------------------------------------------------
|
||||
# Single-organization mode
|
||||
# ---------------------------------------------------------------------------
|
||||
# We run one tenant ("Boilerplate"). This collapses the org-picker UI and lets
|
||||
# We run one tenant ("HiveGrid"). This collapses the org-picker UI and lets
|
||||
# the admin see the global dashboard directly.
|
||||
SENTRY_SINGLE_ORGANIZATION = True
|
||||
|
||||
@@ -30,7 +30,7 @@ except NameError:
|
||||
SENTRY_FEATURES = {}
|
||||
|
||||
# Public signup OFF — invites only. This is the single most important
|
||||
# hardening setting; without it any visitor to sentry.boiler.saas-ctrl.com
|
||||
# hardening setting; without it any visitor to sentry.saas-ctrl.com
|
||||
# can create an account and start receiving events.
|
||||
SENTRY_FEATURES["auth:register"] = False
|
||||
|
||||
@@ -61,7 +61,7 @@ SENTRY_ADMIN_EMAIL = os.environ.get("SENTRY_ADMIN_EMAIL", "")
|
||||
# Upstream's compose ships its own smtp container; we keep that wiring. The
|
||||
# admin email above is the From-address for system mails.
|
||||
SENTRY_OPTIONS = globals().get("SENTRY_OPTIONS", {})
|
||||
SENTRY_OPTIONS.setdefault("mail.from", SENTRY_ADMIN_EMAIL or "sentry@boiler.saas-ctrl.com")
|
||||
SENTRY_OPTIONS.setdefault("mail.from", SENTRY_ADMIN_EMAIL or "sentry@hivegrid.de")
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# URL prefix — must match the Traefik-routed hostname
|
||||
@@ -69,7 +69,7 @@ SENTRY_OPTIONS.setdefault("mail.from", SENTRY_ADMIN_EMAIL or "sentry@boiler.saas
|
||||
# Sentry uses this to build absolute URLs in emails, web-hooks, and the auth
|
||||
# flow. Mismatch leads to broken magic-links in invite emails.
|
||||
SENTRY_OPTIONS["system.url-prefix"] = os.environ.get(
|
||||
"SENTRY_BASE_URL", "https://sentry.boiler.saas-ctrl.com"
|
||||
"SENTRY_BASE_URL", "https://sentry.saas-ctrl.com"
|
||||
)
|
||||
|
||||
# Trust the X-Forwarded-* headers from Traefik (TLS terminated upstream).
|
||||
|
||||
@@ -11,7 +11,7 @@
|
||||
# and a default ACME cert-resolver named `letsencrypt`. The labels below match
|
||||
# that convention.
|
||||
#
|
||||
# Host: sentry.boiler.saas-ctrl.com -> DNS A record points at 46.225.5.93.
|
||||
# Host: sentry.saas-ctrl.com -> DNS A record points at 46.225.5.93.
|
||||
# When a different staging domain is chosen, update both this file and the
|
||||
# inline labels in docker-compose.override.yml (single source of truth lives
|
||||
# in the override file; this fragment is a copy for review).
|
||||
@@ -21,14 +21,14 @@
|
||||
# Primary HTTPS router — terminates TLS, forwards to nginx:80.
|
||||
traefik.enable: "true"
|
||||
traefik.docker.network: "coolify"
|
||||
traefik.http.routers.sentry.rule: "Host(`sentry.boiler.saas-ctrl.com`)"
|
||||
traefik.http.routers.sentry.rule: "Host(`sentry.saas-ctrl.com`)"
|
||||
traefik.http.routers.sentry.entrypoints: "https"
|
||||
traefik.http.routers.sentry.tls: "true"
|
||||
traefik.http.routers.sentry.tls.certresolver: "letsencrypt"
|
||||
traefik.http.services.sentry.loadbalancer.server.port: "80"
|
||||
|
||||
# HTTP -> HTTPS redirect (port 80 catches plain-HTTP hits and 308s them to TLS).
|
||||
traefik.http.routers.sentry-http.rule: "Host(`sentry.boiler.saas-ctrl.com`)"
|
||||
traefik.http.routers.sentry-http.rule: "Host(`sentry.saas-ctrl.com`)"
|
||||
traefik.http.routers.sentry-http.entrypoints: "http"
|
||||
traefik.http.routers.sentry-http.middlewares: "sentry-https-redirect"
|
||||
traefik.http.middlewares.sentry-https-redirect.redirectscheme.scheme: "https"
|
||||
|
||||
Reference in new issue
Block a user