160 lines
10 KiB
Markdown
160 lines
10 KiB
Markdown
# Realtime Collaboration (Hocuspocus + Y.js)
|
|
|
|
**Status:** Phase 05 Task #59 — in_progress (Novel-Editor integration shipped; Rows + Workflow editors deferred to follow-up tasks).
|
|
|
|
## What ships in this phase
|
|
|
|
- A new `apps/hocuspocus/` workspace app — Node service running [Hocuspocus 4.0.0](https://github.com/ueberdosis/hocuspocus) with a Postgres persistence adapter on top of the `CollabDoc` Prisma model.
|
|
- A JWT auth hook (`onAuthenticate`) that ties Y.js documents to the toolctrl session: tokens are signed by the web app's `/api/collab/token` endpoint and verified by the hocuspocus container with the same `HOCUSPOCUS_AUTH_SECRET`.
|
|
- Y.js + Tiptap collaboration extensions wired into the Novel-Editor (`apps/web/app/modules/novel/`). When `VITE_HOCUSPOCUS_URL` is set _and_ the per-tenant `realtime-collab-enabled` feature flag evaluates true, the editor opens a WebSocket session and renders live cursors + presence. Otherwise it mounts exactly as before — single-user REST save.
|
|
- A new `CollabDoc` table (one row per `tenantId:resourceType:resourceId`) storing the binary Y.Doc state, version counter and `lastUpdatedAt`. See `prisma/schema/collabDocs.prisma` + migration `20260517145000_05_collab_docs`.
|
|
|
|
## Architecture
|
|
|
|
```
|
|
┌──────────────┐
|
|
│ Browser │
|
|
│ Novel-Edit │
|
|
└─────┬────────┘
|
|
│ 1. POST /api/collab/token (session-auth)
|
|
▼
|
|
┌──────────────┐
|
|
│ apps/web │
|
|
│ token issuer │ signs JWT { userId, tenantId, resourceType, resourceId }
|
|
└─────┬────────┘
|
|
│ 2. wss://collab.../?<docName> token=<jwt>
|
|
▼
|
|
┌──────────────┐ ┌───────────────┐
|
|
│ Hocuspocus │ ─── reads ───▶ │ Postgres │
|
|
│ container │ ◀── writes ─── │ CollabDoc[] │
|
|
└──────────────┘ └───────────────┘
|
|
```
|
|
|
|
- **DocumentName** is the triple `tenantId:resourceType:resourceId` (e.g. `tenant_abc:kb-article:art_xyz`). Both the JWT and the persistence layer key on this triple, so cross-tenant access is impossible by construction.
|
|
- **JWT TTL** is 15 minutes. `/api/collab/token` returns `expiresIn` and `refreshAfter`; embedding routes should refetch around `refreshAfter` seconds after issuance so the Hocuspocus connection can be recreated before expiry. The shipped `NovelEditor` consumes the `collab.token` it is given; automatic browser-side refresh orchestration is still a follow-up owned by the embedding route/hook.
|
|
- **Tenant scope** is enforced at issuance time. `/api/collab/token` refuses users who are not active members of the requested tenant before it evaluates the feature flag or signs a JWT. Hocuspocus then checks that the token claims match the requested document name.
|
|
- **Persistence cadence** is Hocuspocus' default debounce (2s / max 10s). The store-callback in `apps/hocuspocus/src/extensions/database.ts` upserts the row + bumps `version`. Snapshot/compaction (`Y.encodeStateAsUpdate`) is a deferred follow-up.
|
|
|
|
## Activation
|
|
|
|
Realtime collaboration is **off by default** to keep the boilerplate friendly to self-hosted installs that don't want the extra container. Activation requires three things:
|
|
|
|
1. **Container running.** Deploy `apps/hocuspocus/` (Dockerfile in the repo). Enable the `realtime` profile in `docker-compose`:
|
|
|
|
```bash
|
|
docker compose --profile realtime up -d hocuspocus
|
|
```
|
|
|
|
The image listens on port 1234 internally and is meant to sit behind a TLS-terminating reverse-proxy (Traefik / Caddy).
|
|
|
|
2. **Shared secret set.** `HOCUSPOCUS_AUTH_SECRET` must be configured on both the web app and the hocuspocus container with the same value:
|
|
|
|
```
|
|
HOCUSPOCUS_AUTH_SECRET=$(openssl rand -hex 32)
|
|
```
|
|
|
|
The hocuspocus container refuses to boot without this secret. The `/api/collab/token` endpoint returns `503 collab_disabled` when the web app cannot see it.
|
|
|
|
3. **`realtime-collab-enabled` feature flag turned on for the tenant.** Without it the token endpoint returns `403 feature_disabled` and the Novel-Editor stays single-user. Use the admin Engine v2 page (`/admin/feature-flags/engine`) or the CLI:
|
|
|
|
```bash
|
|
toolctrl-cli feature-flags enable realtime-collab-enabled --tenant <tenantId>
|
|
```
|
|
|
|
## Browser opt-in
|
|
|
|
The Novel-Editor accepts an optional `collab` prop. When the embedding route knows which `tenantId / resourceType / resourceId` is being edited, it should:
|
|
|
|
```tsx
|
|
const tokenRes = await fetch("/api/collab/token", {
|
|
method: "POST",
|
|
headers: { "content-type": "application/json" },
|
|
body: JSON.stringify({ tenantId, resourceType: "kb-article", resourceId }),
|
|
});
|
|
if (tokenRes.ok) {
|
|
const { token, documentName, expiresIn, refreshAfter } = await tokenRes.json();
|
|
return (
|
|
<NovelEditor
|
|
content={initial}
|
|
onChange={onSave}
|
|
collab={{
|
|
url: import.meta.env.VITE_HOCUSPOCUS_URL,
|
|
token,
|
|
documentName,
|
|
user: { name: rootData.user?.email ?? "Anonymous", color: pickColorFromUserId(rootData.user?.id) },
|
|
}}
|
|
/>
|
|
);
|
|
// Store refreshAfter/expiresIn alongside token/documentName and refetch
|
|
// before expiry in the embedding route/hook. NovelEditor does not
|
|
// currently refetch tokens by itself.
|
|
}
|
|
// Fall through to the legacy single-user editor on 403/503.
|
|
return <NovelEditor content={initial} onChange={onSave} />;
|
|
```
|
|
|
|
The `useHocuspocusProvider` hook inside the editor returns `null` whenever `url` or `token` are blank, so the same component handles "feature off" and "feature on" without any caller branching beyond the token fetch above.
|
|
|
|
## Graceful degradation
|
|
|
|
- `VITE_HOCUSPOCUS_URL` unset → editor mounts without Collaboration extensions. REST-save path identical to the pre-task behaviour.
|
|
- `realtime-collab-enabled` flag off → `/api/collab/token` returns 403; caller falls back to single-user mode.
|
|
- Hocuspocus container unreachable mid-edit → the editor renders a yellow banner ("Offline — changes will sync when reconnected"); the existing `onChange` callback continues firing local saves.
|
|
- Token expires → Hocuspocus disconnects with a 4-series close code. Today the editor can only use a newly supplied `collab.token`; the embedding route/hook must refetch and re-render, preferably using the endpoint's `refreshAfter` hint. A built-in refresh hook remains deferred.
|
|
- User is not an active member of the requested tenant → `/api/collab/token` returns `403 tenant-membership-required` and no feature-flag state is exposed.
|
|
|
|
## Database model
|
|
|
|
```
|
|
model CollabDoc {
|
|
id String @id @default(cuid())
|
|
tenantId String
|
|
resourceType String // "kb-article" | "crm-note" | "workflow" | …
|
|
resourceId String // primary key of the upstream entity
|
|
ydocState Bytes // Y.encodeStateAsUpdate(doc)
|
|
version Int @default(0)
|
|
lastUpdatedAt DateTime @default(now())
|
|
lastSnapshotAt DateTime?
|
|
createdAt DateTime @default(now())
|
|
updatedAt DateTime @updatedAt
|
|
|
|
@@unique([tenantId, resourceType, resourceId])
|
|
@@index([tenantId, resourceType, lastUpdatedAt(sort: Desc)])
|
|
@@index([lastUpdatedAt])
|
|
}
|
|
```
|
|
|
|
Rollback: dropping `CollabDoc` is safe — no other code path reads from it.
|
|
|
|
## Environment variables
|
|
|
|
| Name | Required for collab | Notes |
|
|
| ------------------------------ | :-----------------: | -------------------------------------------------------------- |
|
|
| `HOCUSPOCUS_AUTH_SECRET` | ✓ | HS256 secret. ≥ 32 bytes entropy. Web + hocuspocus must agree. |
|
|
| `VITE_HOCUSPOCUS_URL` | ✓ | Browser-visible `wss://…` URL. |
|
|
| `HOCUSPOCUS_URL` | | Server-internal endpoint (defaults to `ws://hocuspocus:1234`). |
|
|
| `HOCUSPOCUS_PORT` | | Internal listen port (default `1234`). |
|
|
| `HOCUSPOCUS_HOST` | | Bind address inside the container (default `0.0.0.0`). |
|
|
| `HOCUSPOCUS_MAX_PAYLOAD_BYTES` | | Per-doc state cap (default `10485760` / 10 MiB). |
|
|
|
|
All entries are also documented in `packages/config/src/catalog.ts` so the env-drift verifier accepts them.
|
|
|
|
## Out of scope / deferred
|
|
|
|
- **Snapshot + cleanup cron** — a planned follow-up will land `apps/worker/src/cron/collab-cleanup.ts` plus the `CollabDocArchive` table referenced in the spec.
|
|
- **Presence-bar UI** — the live cursor labels ship via `CollaborationCursor`; a sidebar avatar-stack of currently connected users is a follow-up.
|
|
- **Rows + Workflow editor integrations** — the Novel-Editor lands the pattern; other editors will reuse `useHocuspocusProvider` + `/api/collab/token`, schedule token refetch using `refreshAfter`, and add their own row/workflow permission checks before opting into collab.
|
|
- **Connection quota per tenant** — the spec's `onConnect` plan-tier quota check is a follow-up.
|
|
|
|
## Scaling considerations
|
|
|
|
- Hocuspocus uses long-lived WebSocket connections. Run at least two replicas behind a sticky-session balancer if the deployment serves > a few hundred concurrent collaborators per region.
|
|
- The Postgres write-amplification is bounded by Hocuspocus' debounce (2-10s). For very high update-rates, the spec calls for moving persistence to Redis with periodic Postgres snapshotting — that is a follow-up infra step, not a code change.
|
|
- The 10 MiB per-doc limit (`HOCUSPOCUS_MAX_PAYLOAD_BYTES`) acts as the worst-case row size in Postgres; for normal Novel-Editor content it stays well under 1 MiB.
|
|
|
|
## References
|
|
|
|
- Improvement plan: [`docs/improvement-plan/05-business-features/realtime-collab.md`](../improvement-plan/05-business-features/realtime-collab.md)
|
|
- Hocuspocus docs: https://tiptap.dev/docs/hocuspocus
|
|
- Y.js docs: https://docs.yjs.dev/
|