Realtime Collaboration (Hocuspocus + Y.js)

Status: Phase 05 Task #59 — in_progress (Novel-Editor integration shipped; Rows + Workflow editors deferred to follow-up tasks).

What ships in this phase

A new apps/hocuspocus/ workspace app — Node service running Hocuspocus 4.0.0 with a Postgres persistence adapter on top of the CollabDoc Prisma model.
A JWT auth hook (onAuthenticate) that ties Y.js documents to the toolctrl session: tokens are signed by the web app's /api/collab/token endpoint and verified by the hocuspocus container with the same HOCUSPOCUS_AUTH_SECRET.
Y.js + Tiptap collaboration extensions wired into the Novel-Editor (apps/web/app/modules/novel/). When VITE_HOCUSPOCUS_URL is set and the per-tenant realtime-collab-enabled feature flag evaluates true, the editor opens a WebSocket session and renders live cursors + presence. Otherwise it mounts exactly as before — single-user REST save.
A new CollabDoc table (one row per tenantId:resourceType:resourceId) storing the binary Y.Doc state, version counter and lastUpdatedAt. See prisma/schema/collabDocs.prisma + migration 20260517145000_05_collab_docs.

Architecture

            ┌──────────────┐
            │   Browser    │
            │  Novel-Edit  │
            └─────┬────────┘
                  │  1. POST /api/collab/token  (session-auth)
                  ▼
            ┌──────────────┐
            │ apps/web     │
            │ token issuer │  signs JWT { userId, tenantId, resourceType, resourceId }
            └─────┬────────┘
                  │  2. wss://collab.../?<docName>  token=<jwt>
                  ▼
            ┌──────────────┐                ┌───────────────┐
            │ Hocuspocus   │ ─── reads ───▶ │ Postgres      │
            │  container   │ ◀── writes ─── │ CollabDoc[]   │
            └──────────────┘                └───────────────┘

DocumentName is the triple tenantId:resourceType:resourceId (e.g. tenant_abc:kb-article:art_xyz). Both the JWT and the persistence layer key on this triple, so cross-tenant access is impossible by construction.
JWT TTL is 15 minutes. /api/collab/token returns expiresIn and refreshAfter; embedding routes should refetch around refreshAfter seconds after issuance so the Hocuspocus connection can be recreated before expiry. The shipped NovelEditor consumes the collab.token it is given; automatic browser-side refresh orchestration is still a follow-up owned by the embedding route/hook.
Tenant scope is enforced at issuance time. /api/collab/token refuses users who are not active members of the requested tenant before it evaluates the feature flag or signs a JWT. Hocuspocus then checks that the token claims match the requested document name.
Persistence cadence is Hocuspocus' default debounce (2s / max 10s). The store-callback in apps/hocuspocus/src/extensions/database.ts upserts the row + bumps version. Snapshot/compaction (Y.encodeStateAsUpdate) is a deferred follow-up.

Activation

Realtime collaboration is off by default to keep the boilerplate friendly to self-hosted installs that don't want the extra container. Activation requires three things:

Container running. Deploy apps/hocuspocus/ (Dockerfile in the repo). Enable the realtime profile in docker-compose:
```
docker compose --profile realtime up -d hocuspocus
```
The image listens on port 1234 internally and is meant to sit behind a TLS-terminating reverse-proxy (Traefik / Caddy).
Shared secret set. HOCUSPOCUS_AUTH_SECRET must be configured on both the web app and the hocuspocus container with the same value:
```
HOCUSPOCUS_AUTH_SECRET=$(openssl rand -hex 32)
```
The hocuspocus container refuses to boot without this secret. The /api/collab/token endpoint returns 503 collab_disabled when the web app cannot see it.
realtime-collab-enabled feature flag turned on for the tenant. Without it the token endpoint returns 403 feature_disabled and the Novel-Editor stays single-user. Use the admin Engine v2 page (/admin/feature-flags/engine) or the CLI:
```
toolctrl-cli feature-flags enable realtime-collab-enabled --tenant <tenantId>
```

Browser opt-in

The Novel-Editor accepts an optional collab prop. When the embedding route knows which tenantId / resourceType / resourceId is being edited, it should:

const tokenRes = await fetch("/api/collab/token", {
  method: "POST",
  headers: { "content-type": "application/json" },
  body: JSON.stringify({ tenantId, resourceType: "kb-article", resourceId }),
});
if (tokenRes.ok) {
  const { token, documentName, expiresIn, refreshAfter } = await tokenRes.json();
  return (
    <NovelEditor
      content={initial}
      onChange={onSave}
      collab={{
        url: import.meta.env.VITE_HOCUSPOCUS_URL,
        token,
        documentName,
        user: { name: rootData.user?.email ?? "Anonymous", color: pickColorFromUserId(rootData.user?.id) },
      }}
    />
  );
  // Store refreshAfter/expiresIn alongside token/documentName and refetch
  // before expiry in the embedding route/hook. NovelEditor does not
  // currently refetch tokens by itself.
}
// Fall through to the legacy single-user editor on 403/503.
return <NovelEditor content={initial} onChange={onSave} />;

The useHocuspocusProvider hook inside the editor returns null whenever url or token are blank, so the same component handles "feature off" and "feature on" without any caller branching beyond the token fetch above.

Graceful degradation

VITE_HOCUSPOCUS_URL unset → editor mounts without Collaboration extensions. REST-save path identical to the pre-task behaviour.
realtime-collab-enabled flag off → /api/collab/token returns 403; caller falls back to single-user mode.
Hocuspocus container unreachable mid-edit → the editor renders a yellow banner ("Offline — changes will sync when reconnected"); the existing onChange callback continues firing local saves.
Token expires → Hocuspocus disconnects with a 4-series close code. Today the editor can only use a newly supplied collab.token; the embedding route/hook must refetch and re-render, preferably using the endpoint's refreshAfter hint. A built-in refresh hook remains deferred.
User is not an active member of the requested tenant → /api/collab/token returns 403 tenant-membership-required and no feature-flag state is exposed.

Database model

model CollabDoc {
  id              String   @id @default(cuid())
  tenantId        String
  resourceType    String          // "kb-article" | "crm-note" | "workflow" | …
  resourceId      String          // primary key of the upstream entity
  ydocState       Bytes           // Y.encodeStateAsUpdate(doc)
  version         Int      @default(0)
  lastUpdatedAt   DateTime @default(now())
  lastSnapshotAt  DateTime?
  createdAt       DateTime @default(now())
  updatedAt       DateTime @updatedAt

  @@unique([tenantId, resourceType, resourceId])
  @@index([tenantId, resourceType, lastUpdatedAt(sort: Desc)])
  @@index([lastUpdatedAt])
}

Rollback: dropping CollabDoc is safe — no other code path reads from it.

Environment variables

Name	Required for collab	Notes
`HOCUSPOCUS_AUTH_SECRET`	✓	HS256 secret. ≥ 32 bytes entropy. Web + hocuspocus must agree.
`VITE_HOCUSPOCUS_URL`	✓	Browser-visible `wss://…` URL.
`HOCUSPOCUS_URL`		Server-internal endpoint (defaults to `ws://hocuspocus:1234`).
`HOCUSPOCUS_PORT`		Internal listen port (default `1234`).
`HOCUSPOCUS_HOST`		Bind address inside the container (default `0.0.0.0`).
`HOCUSPOCUS_MAX_PAYLOAD_BYTES`		Per-doc state cap (default `10485760` / 10 MiB).

All entries are also documented in packages/config/src/catalog.ts so the env-drift verifier accepts them.

Out of scope / deferred

Snapshot + cleanup cron — a planned follow-up will land apps/worker/src/cron/collab-cleanup.ts plus the CollabDocArchive table referenced in the spec.
Presence-bar UI — the live cursor labels ship via CollaborationCursor; a sidebar avatar-stack of currently connected users is a follow-up.
Rows + Workflow editor integrations — the Novel-Editor lands the pattern; other editors will reuse useHocuspocusProvider + /api/collab/token, schedule token refetch using refreshAfter, and add their own row/workflow permission checks before opting into collab.
Connection quota per tenant — the spec's onConnect plan-tier quota check is a follow-up.

Scaling considerations

Hocuspocus uses long-lived WebSocket connections. Run at least two replicas behind a sticky-session balancer if the deployment serves > a few hundred concurrent collaborators per region.
The Postgres write-amplification is bounded by Hocuspocus' debounce (2-10s). For very high update-rates, the spec calls for moving persistence to Redis with periodic Postgres snapshotting — that is a follow-up infra step, not a code change.
The 10 MiB per-doc limit (HOCUSPOCUS_MAX_PAYLOAD_BYTES) acts as the worst-case row size in Postgres; for normal Novel-Editor content it stays well under 1 MiB.

References

Improvement plan: docs/improvement-plan/05-business-features/realtime-collab.md
Hocuspocus docs: https://tiptap.dev/docs/hocuspocus
Y.js docs: https://docs.yjs.dev/

10 KiB Raw Blame History