Track HiveGrid Cloud planning reference

2026-06-26 20:48:01 +02:00
1 parent b3d5ea6764
commit e341c5938a
@@ -0,0 +1,201 @@
+# Read / Write Split (Prisma `$extends`)
+
+> Phase 03 task #41. Status: code-complete; replica provisioning is operator-owned.
+
+## TL;DR
+
+```ts
+import { db, dbRead } from "~/utils/db.server";
+
+// Default — primary (write target). Use for all writes, auth, read-after-write.
+await db.user.update({ where: { id }, data: { lastSeenAt: new Date() } });
+
+// Explicit primary — same client, communicates intent.
+import { getWriteDbClient } from "@toolctrl/db/client.server";
+
+// Replica — eventually consistent. Use for listings, dashboards, public reads.
+const users = await dbRead.user.findMany({ where: { tenantId } });
+```
+
+`dbRead` is wired to `DATABASE_READ_URL` when set. When unset (single-DB dev,
+self-hosted setups) it returns the same client as `db`, so the code path is
+identical and lag = 0.
+
+## Convention: when to use which
+
+| Use `dbRead` (replica) | Use `db` / `getWriteDbClient()` (primary) |
+| ---------------------- | ----------------------------------------- |
+| List endpoints / paginated grids | All mutations (`create`, `update`, `delete`, `upsert`) |
+| Public marketing pages (sites, portals) | Auth-resolution paths (session → user → tenant) |
+| Knowledge-base public reads + search | Stripe webhook handlers reading state they just wrote |
+| Analytics dashboards & aggregations | Outbox dispatcher reading `PENDING` events (see #42) |
+| Admin tenant / role enumerations (post-auth) | Anything inside a `db.$transaction(...)` callback |
+| Dashboard counts (`count()` / `groupBy()`) | Idempotency-key lookups (request-scoped consistency) |
+| Search-result rendering | Read-after-write inside the same request |
+
+When in doubt: leave on `db` (= primary). The replica is an optimisation,
+not a correctness requirement. **Auto-routing is intentionally not
+implemented** — the spec rejects it because it makes read-after-write
+hazards invisible.
+
+## Read-after-write hazard
+
+Streaming replication is **eventually consistent**:
+
+- Typical lag: < 1 s
+- Burst writes (bulk imports, k6 load tests): up to 5–10 s
+- A query issued on the replica milliseconds after the write hits the
+  primary will read the **pre-write** state.
+
+**Mitigation:**
+
+1. Always use `db` inside transactions and for writes.
+2. Never re-read on `dbRead` what you just wrote on `db` in the same
+   request. If you must, route the read through `db` too.
+3. Auth-critical paths (`getUserAndTenants`, session resolution) stay on
+   `db` even though they're "reads" — they gate the rest of the
+   request, and a stale read can mis-route a tenant boundary check.
+
+## Fallback semantics
+
+`getReadDbClient()`:
+
+```ts
+export function getReadDbClient(): DbClient {
+  const readUrl = process.env.DATABASE_READ_URL;
+  if (!readUrl || readUrl.length === 0) {
+    return getDbClient(); // single instance, no double connection
+  }
+  // memoised replica singleton
+  if (!global.__dbRead) {
+    global.__dbRead = createDbClient({ connectionString: readUrl });
+  }
+  return global.__dbRead;
+}
+```
+
+| State of `DATABASE_READ_URL` | `dbRead` returns |
+| ---------------------------- | --------------------------------- |
+| unset / empty                | the write client (`db === dbRead`) |
+| valid `postgres://...`       | a distinct read-only client       |
+
+This means CI, local dev, and any deploy without a replica continue to
+work without code changes. All migrations are zero-risk to roll out.
+
+## Telemetry
+
+`@toolctrl/db/telemetry.server` exposes `attachPrismaTelemetry()` — a
+`$extends({ query })` wrapper that records every operation with
+`{ client: 'read' | 'write', model, operation, durationMs, ok }`. Wire it
+in the app bootstrap by adapting the recorder to your metrics sink
+(prom-client `Histogram` / `Counter` or an OpenTelemetry `Meter`). The
+canonical counter shape is `prisma.queries.total{client,model,operation}`.
+
+## Coolify / staging setup (operator-owned)
+
+Replica provisioning is **out of scope** for the application repo — these
+steps live with the platform operator.
+
+1. **Primary Postgres** (Coolify service `postgres`). `postgresql.conf`:
+
+   ```
+   wal_level = replica
+   max_wal_senders = 3
+   wal_keep_size = 256MB
+   ```
+
+   Then:
+
+   ```sql
+   CREATE USER replicator WITH REPLICATION ENCRYPTED PASSWORD '...';
+   CREATE USER read_user WITH ENCRYPTED PASSWORD '...';
+   GRANT CONNECT ON DATABASE toolctrl TO read_user;
+   GRANT USAGE ON SCHEMA public TO read_user;
+   GRANT SELECT ON ALL TABLES IN SCHEMA public TO read_user;
+   ALTER DEFAULT PRIVILEGES IN SCHEMA public
+     GRANT SELECT ON TABLES TO read_user;
+   ```
+
+2. **Replica Postgres** (Coolify service `postgres-replica`,
+   `postgres:16-alpine`):
+
+   ```bash
+   # Init from primary
+   until pg_basebackup -h postgres -D /var/lib/postgresql/data -U replicator -P -R; do
+     sleep 5
+   done
+   ```
+
+   `postgresql.replica.conf`:
+
+   ```
+   hot_standby = on
+   primary_conninfo = 'host=postgres port=5432 user=replicator password=...'
+   ```
+
+3. **App env**:
+
+   ```sh
+   DATABASE_URL=postgres://app_user:...@pgbouncer:6432/toolctrl?pgbouncer=true&connection_limit=10
+   DATABASE_READ_URL=postgres://read_user:...@postgres-replica:5432/toolctrl?connection_limit=10
+   ```
+
+   Optionally front the replica with its own PgBouncer for connection pooling.
+
+4. **Verification:**
+
+   ```sql
+   -- On primary
+   SELECT * FROM pg_stat_replication;
+
+   -- On replica
+   SELECT pg_last_xact_replay_timestamp(), now() - pg_last_xact_replay_timestamp() AS lag;
+   ```
+
+   Grafana panel `prisma.queries.total{client="read"}` should rise on
+   dashboard / KB / portal traffic.
+
+## Rollback
+
+- **Per-call-site:** `git revert <migration-commit>` flips that module
+  back to `db` (primary). No data change.
+- **Global:** unset `DATABASE_READ_URL` and restart the app. `dbRead === db`,
+  all traffic hits the primary.
+- **Replica down:** Prisma's read client will throw connection errors.
+  Either unset `DATABASE_READ_URL` (degraded performance, no errors) or
+  add a circuit-breaker (future work, out of scope here).
+
+## Migration recipe (for future modules)
+
+1. **Find candidate read-only functions** — pure `findMany`, `findUnique`,
+   `findFirst`, `count`, `groupBy`, `aggregate` with no surrounding write.
+2. **Check the caller graph** — is this function called immediately
+   after a write that updates the same rows? If yes, leave on `db`.
+3. **Replace `db.<model>.<op>` with `dbRead.<model>.<op>`** and add a
+   one-line comment explaining the choice.
+4. **Add `dbRead` to the file's imports** alongside `db` (keep `db` for
+   the mutations still in the file).
+5. **Run** `pnpm lint --max-warnings=0` and `pnpm typecheck`.
+6. **Tests** stay green because of the fallback semantics — single-DB
+   CI doesn't see a difference.
+7. **Commit per-module** (`03-scaling/rw-split: migrate <module> ...`)
+   so a single revert can roll back one module.
+
+## What was migrated in #41
+
+- `apps/web/app/modules/crm/services/CrmService.ts` — user enumeration
+  for sync display.
+- `apps/web/app/modules/knowledgeBase/db/*.db.server.ts` — public KB
+  reads, listings, search, featured-article queries, dashboard counts.
+- `apps/web/app/modules/analytics/db/analyticsUniqueVisitors.db.server.ts`
+  — `groupBy` aggregations.
+- `apps/web/app/utils/db/tenants.db.server.ts` — admin tenant enumerations
+  (post-auth).
+- `apps/web/app/modules/sites/db/sites.db.server.ts` — read-only site
+  loaders (auth-gating `getSiteByTypeAndSiteId` left on primary).
+- `apps/web/app/modules/portals/db/portals.db.server.ts` and
+  `portalPages.db.server.ts` — listings, page lookups, dashboard counts.
+
+`appData.ts` was **not** migrated wholesale; only the secondary tenant
+listings it consumes were moved. The auth-resolution path stays on
+primary by design.