Track HiveGrid Cloud planning reference
This commit is contained in:
1 parent
b3d5ea6764
commit
e341c5938a
5784 files changed
+1104929
-3
No files matched your search
@@ -0,0 +1,131 @@
|
||||
# Phase 05 — Business-Features
|
||||
|
||||
> **Docs-sidecar status:** Historical phase plan with live closeout notes.
|
||||
> Use `docs/feature-inventory.md`, `docs/feature-manifests/*.json`,
|
||||
> `docs/features/plugins.md`, and the focused docs linked below for current
|
||||
> runtime ownership. Do not treat unchecked items in this file as proof that a
|
||||
> route, worker, provider, or package still needs to be added.
|
||||
> Email Sequences status is deliberately delegated to
|
||||
> `docs/features/email-sequences.md`; this phase README must not be used as a
|
||||
> production-delivery claim for marketing mail.
|
||||
|
||||
**Dauer:** 5+ Wochen (parallelisiert; rein sequenziell ~10-12 Wochen)
|
||||
**Tasks:** 9 (Feature-Flags + 3× AI-Cost + Form-Widget + Realtime-Collab + 3× Marketing-Funnel)
|
||||
**Ziel:** Differenzierende Produkt-Features etablieren, die aus der Boilerplate ein vermarktbares SaaS-Produkt machen. Feature-Flags als zentrale Steuerungs-Engine, end-to-end AI-Cost-Tracking inkl. Quota + Stripe-Overage, embeddable Form-Widget für Marketing-Sites, Realtime-Collaboration für Multi-User-Editing und ein Marketing-Funnel von Lead-Capture bis vorbereitetem Email-Sequence-Status. Production-Mailversand bleibt hinter `MARKETING_SEQUENCES_ENABLED=false`, bis die Email-Sequences-Follow-ups abgeschlossen sind.
|
||||
|
||||
## Warum diese Phase nach Phase 04
|
||||
|
||||
- **SharedStore (#37 aus Phase 03)** wird für Feature-Flag-Eval-Cache und für AI-Quota-Cache gebraucht — beide Tasks würden ohne tenant-konsistenten verteilten Cache pro Worker-Instanz auseinanderlaufen.
|
||||
- **Outbox-Pattern (#42 aus Phase 03)** ist Pflicht für AI-Cost-Event-Persistierung. Jeder LLM-Call darf nicht synchron in die DB schreiben (Latenz + Provider-Timeouts), sondern landet als Outbox-Event und wird vom Worker persistiert.
|
||||
- **Worker-Queues (#43 aus Phase 03)** liefern `worker-general` für AI-Cost-Aggregation/Outbox-Fanout und `worker-mail` für Marketing-Email-Sequencer sowie Sequence-Send-Job-Verträge. Production delivery stays behind `MARKETING_SEQUENCES_ENABLED=false` until #80 closeout evidence exists.
|
||||
- **React-Email (aus Phase 10)** wird vom Marketing-Funnel-Phase-3-Task benötigt. Diese Detail-MD ist explizit so geschrieben, dass React-Email vorgezogen werden kann, ohne die Phase-10-Reihenfolge zu sprengen.
|
||||
- **Search/RAG (Phase 04)** wird in dieser Phase nicht direkt konsumiert, ist aber Voraussetzung für Phase 07 — und Phase 05 blockt wiederum Phase 06 + 07, deshalb läuft Phase 05 nach Phase 04.
|
||||
|
||||
## Warum diese Phase vor Phase 06
|
||||
|
||||
- **Plugin-System** nutzt Feature-Flags für Plugin-Capability-Gating (jeder Plugin-Hook kann hinter einem Flag liegen).
|
||||
- **Plugin-Lifecycle-Events** registrieren sich an der Outbox + werden für AI-Kostenzuordnung Plugin-aware sein müssen (Plugin verursacht Tokens → Tenant-Cost-Bucket erweitert um Plugin-Dimension).
|
||||
|
||||
## Task-Reihenfolge
|
||||
|
||||
| Reihenfolge | Task-ID | Datei | Parallelisierbar mit |
|
||||
|---|---|---|---|
|
||||
| 1 | #53 | `feature-flags.md` | — (Foundation; blockt mehrere Folgetasks weich) |
|
||||
| 2 | #54 | `ai-cost-phase-1.md` | #58, #59, #78 |
|
||||
| 3 | #55 | `ai-cost-phase-2.md` | #58, #59, #78, #79 (sequenziell zu #54) |
|
||||
| 4 | #56 | `ai-cost-phase-3.md` | #58, #59, #79, #80 (sequenziell zu #55) |
|
||||
| 5 | #58 | `form-widget.md` | gesamte AI-Cost-Reihe + Marketing-Funnel |
|
||||
| 6 | #59 | `realtime-collab.md` | gesamte AI-Cost-Reihe + Marketing-Funnel + Form-Widget |
|
||||
| 7 | #78 | `marketing-funnel-phase-1.md` | AI-Cost-Reihe, Form-Widget, Realtime |
|
||||
| 8 | #79 | `marketing-funnel-phase-2.md` | sequenziell zu #78 |
|
||||
| 9 | #80 | `marketing-funnel-phase-3.md` | sequenziell zu #79 (braucht React-Email) |
|
||||
|
||||
## Dependency-Graph (intern)
|
||||
|
||||
```
|
||||
Phase 03 (SharedStore #37, Outbox #42, Worker-Queues #43)
|
||||
│
|
||||
▼
|
||||
#53 feature-flags
|
||||
│
|
||||
┌──────────────────────────┼──────────────────────────────────┐
|
||||
▼ ▼ ▼
|
||||
#54 ai-cost-phase-1 #58 form-widget #78 marketing-phase-1
|
||||
│ │ │
|
||||
▼ │ ▼
|
||||
#55 ai-cost-phase-2 │ #79 marketing-phase-2
|
||||
│ │ │
|
||||
▼ │ ▼
|
||||
#56 ai-cost-phase-3 #59 realtime-collab #80 marketing-phase-3
|
||||
(braucht React-Email
|
||||
aus Phase 10 vorgezogen)
|
||||
│
|
||||
▼
|
||||
Phase 06 Plugin-System
|
||||
Phase 07 Chatbot V1/V2
|
||||
```
|
||||
|
||||
Drei echte Parallel-Tracks:
|
||||
|
||||
- **Track A — AI-Cost (sequenziell):** `#54 → #55 → #56`. Phase-2 baut auf der Cost-Event-Pipeline aus Phase-1 auf; Phase-3 baut auf der Quota-Definition aus Phase-2 auf.
|
||||
- **Track B — Standalone-Features:** `#58 form-widget` und `#59 realtime-collab` sind in sich abgeschlossen, brauchen nur Feature-Flags. Können parallel mit AI-Cost-Track laufen.
|
||||
- **Track C — Marketing-Funnel (sequenziell):** `#78 → #79 → #80`. Phase-2 baut auf dem Lead-Modell aus Phase-1 auf; Phase-3 erweitert um Sequence-Engine und braucht React-Email-Templates.
|
||||
|
||||
Empfohlene Parallelisierung mit 3 Agents: Track A, B (beide Tasks nacheinander oder zwei Sub-Agents) und C laufen gleichzeitig nach Abschluss von `#53`.
|
||||
|
||||
## Phase Done-Definition
|
||||
|
||||
**2026-05-27 status note:** Phase 05 is not complete as a full original
|
||||
phase contract. Marketing email sequences currently have the data model, pure
|
||||
engine, admin UI/CLI, tracking, unsubscribe, live admin stats, unit-tested
|
||||
stale-lead/trial-expiring runner functions, delayed-step enqueueing, bundled
|
||||
template rendering, and the `@toolctrl/mail` Mailjet adapter. They do
|
||||
**not** yet prove production email delivery end to end. Signed Mailjet
|
||||
bounce/rejection ingress is shipped, but production delivery remains gated on
|
||||
seed data, stats aggregation cron/persistence, provider delivery/open/click
|
||||
reconciliation evidence beyond the in-repo tracking routes, Playwright E2E,
|
||||
and production scheduled-job registration/deploy evidence.
|
||||
|
||||
Diese Phase ist `completed` wenn:
|
||||
|
||||
- [ ] Alle 9 Tasks `completed` in `PROGRESS.md`
|
||||
- [ ] `pnpm lint`, `pnpm typecheck`, `pnpm test` global grün
|
||||
- [ ] `packages/feature-flags/` exportiert stabile öffentliche API (`evaluateFlag`, `getFeatureFlagVariant`, `useFeatureFlag`-Hook) mit publizierter Typen-Surface
|
||||
- [ ] `packages/ai/src/cost/` exportiert `countTokens`, `countCallCost`, `estimateLLMCost`, `assertTenantAICreditAccess`
|
||||
- [ ] `packages/marketing-sdk-js/` ist als gebundleder Asset-Output (`marketing.js`) origin-served und CDN-ready; CDN-Provisioning bleibt A7/B2/#51-Backlog
|
||||
- [ ] Prisma-Models live: `FeatureFlag`, `FeatureFlagAudit`, `AICostEvent`, `TenantAICostSummary`, `Lead`, `TenantAcquisition`, `EmailSequence`, `EmailSequenceRun`, `EmailSequenceStepDelivery`
|
||||
- [ ] Tenant-UI live: `/app/$tenant/settings/ai-usage`, Form-Builder unter `/app/$tenant/forms`
|
||||
- [ ] Admin-UI live: `/admin/feature-flags`, `/admin/analytics/ai-costs`, `/admin/marketing/leads`, `/admin/email-sequences`, `/admin/marketing/funnels`, `/admin/marketing/integrations`
|
||||
- [ ] Worker registriert mindestens die Job-Handler `ai-cost.persist-event`, `ai-cost.aggregate-summary`, `ai-quota.alert`, `marketing.lead.webhook`, `email-sequence-trigger`, `email-sequence-scheduler`, `lead-stale-trigger`, `trial-expiring-trigger` — for Email Sequences this is foundation registration only; production scheduler/deploy evidence, seed data, stats aggregation cron/persistence, provider delivery/open/click reconciliation evidence beyond the in-repo tracking routes, and E2E remain tracked as open follow-ups in `docs/features/email-sequences.md`.
|
||||
- [ ] Hocuspocus-Container läuft auf Staging, Y.js-Doc-Persistence in Postgres
|
||||
- [ ] `.env.example` enthält neue Keys: `HOCUSPOCUS_URL`, `HOCUSPOCUS_SECRET`, `MARKETING_DOMAINS`, `FORM_WIDGET_CDN_URL`, `STRIPE_AI_OVERAGE_METER_ID` (optional), `TURNSTILE_SITE_KEY`, `TURNSTILE_SECRET_KEY`
|
||||
- [ ] Doku-Files vorhanden: `docs/features/feature-flags.md`, `docs/features/ai-cost-billing.md`, `docs/features/embeddable-forms.md`, `docs/features/realtime-collaboration.md`, `docs/features/marketing-funnel.md`
|
||||
- [ ] Staging-Deployment auf `agent/phase-05-final`: Hocuspocus-Container hochgefahren, Form-Widget gegen externe Test-Domain getestet, Lead-Capture-Endpoint per CORS-Whitelist getestet, AI-Cost-Pipeline End-to-End mit echter OpenAI-Call → Event → Summary
|
||||
- [ ] Integration-Test-Flows:
|
||||
- Feature-Flag-Toggle in Admin-UI → Eval in Client innerhalb 60s (Pub/Sub-Invalidation)
|
||||
- LLM-Call → AICostEvent in Outbox → Persistenz → Summary-Aggregation → Dashboard-Sichtbarkeit
|
||||
- Quota-Überschreitung → Soft-Degradation oder HTTP 402 + Alert
|
||||
- Lead-Capture via Form-Widget → Lead in DB → Conversion zu Tenant → Email-Sequence-Run sichtbar; erste Produktions-Mail erst nach #80 seed/stats/webhook/E2E/deploy closeout
|
||||
- [ ] Tenant-Isolation-Tests: Feature-Flag-Allowlist greift nur für Ziel-Tenant; AICostEvent nie cross-tenant lesbar; Lead-Daten nie cross-tenant exposed
|
||||
- [ ] `gsd-code-reviewer`-Subagent hat alle Phase-05-Commits reviewed; Findings adressiert
|
||||
- [ ] Phase-Summary in dieser README.md ergänzt (siehe unten)
|
||||
|
||||
## Phase Summary
|
||||
|
||||
_(Wird nach Abschluss der Phase vom ausführenden Agent ergänzt.)_
|
||||
|
||||
### Lessons Learned
|
||||
|
||||
_(Wird nach Abschluss ergänzt — insbesondere: Targeting-Rule-Komplexitäts-Findings, AI-Cost-Accuracy-Vergleich tiktoken-Estimation vs Provider-Billing, Hocuspocus-Skalierung-Findings, Marketing-Sequence-Open-Rates der Pre-Built-Templates.)_
|
||||
|
||||
## Verweise
|
||||
|
||||
- Detail-MD-Template: `../DETAIL-MD-TEMPLATE.md`
|
||||
- Agent-Workflow: `../AGENT-INSTRUCTIONS.md`
|
||||
- Execution-Order: `../00-execution-order.md`
|
||||
- Live-Status: `../PROGRESS.md`
|
||||
- Current feature classification: `../../feature-inventory.md`
|
||||
- Current feature manifest contracts: `../../feature-manifests/README.md`
|
||||
- Vorgänger-Phase: `../04-search-rag/README.md`
|
||||
- Nachfolger-Phasen im Originalplan: `../06-plugin-system/README.md`, `../07-chatbot/README.md`
|
||||
- Vorgezogen aus Phase 10: `../10-compliance-polish/react-email.md` (für `marketing-funnel-phase-3.md`)
|
||||
@@ -0,0 +1,471 @@
|
||||
# Phase 05 — Verification Report
|
||||
|
||||
> **Historical report:** This file verified the Phase-05 plan quality on
|
||||
> 2026-05-16. It is not the production-readiness source of truth. Later
|
||||
> closeout docs, feature docs, runbooks, and `docs/ops/asset-cdn.md` supersede
|
||||
> claims that read like live status, especially CDN/SRI and email-sequence
|
||||
> delivery.
|
||||
|
||||
**Verifier:** Phase-05-Verifier (Read-Only)
|
||||
**Datum:** 2026-05-16
|
||||
**Scope:** README.md + 9 Task-Detail-MDs (10 Files total)
|
||||
**Result:** PASS mit Minor-Findings — keine Blocker; einige inhaltliche Lücken adressierbar während Execution.
|
||||
|
||||
---
|
||||
|
||||
## 1. Template-Konformität
|
||||
|
||||
### Section-Reihenfolge (laut `DETAIL-MD-TEMPLATE.md`)
|
||||
|
||||
| File | Frontmatter | Problem | Depends on | Blocks | Approach | Impl Steps | Tests | Acceptance | Rollback | Security | Subagents | Staging | Deviations | Progress Log | Lessons Learned |
|
||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
||||
| feature-flags.md | OK | OK | OK | OK | OK | OK | OK | OK | OK | OK | OK | OK | OK | OK | — (optional, fehlt) |
|
||||
| ai-cost-phase-1.md | OK | OK | OK | OK | OK | OK | OK | OK | OK | OK | OK | OK | OK | OK | — |
|
||||
| ai-cost-phase-2.md | OK | OK | OK | OK | OK | OK | OK | OK | OK | OK | OK | OK | OK | OK | — |
|
||||
| ai-cost-phase-3.md | OK | OK | OK | OK | OK | OK | OK | OK | OK | OK | OK | OK | OK | OK | — |
|
||||
| form-widget.md | OK | OK | OK | OK | OK | OK | OK | OK | OK | OK | OK | OK | OK | OK | — |
|
||||
| realtime-collab.md | OK | OK | OK | OK | OK | OK | OK | OK | OK | OK | OK | OK | OK | OK | — |
|
||||
| marketing-funnel-phase-1.md | OK | OK | OK | OK | OK | OK | OK | OK | OK | OK | OK | OK | OK | OK | — |
|
||||
| marketing-funnel-phase-2.md | OK | OK | OK | OK | OK | OK | OK | OK | OK | OK | OK | OK | OK | OK | — |
|
||||
| marketing-funnel-phase-3.md | OK | OK | OK | OK | OK | OK | OK | OK | OK | OK | OK | OK | OK | OK | — |
|
||||
|
||||
**Lessons Learned** ist laut Template optional (Z. 128: "Lessons Learned (optional)"). Konsistent über alle 9 Files weggelassen → akzeptabel.
|
||||
|
||||
**Finding T1 (Info):** Alle 9 Detail-MDs erfüllen die 14 Pflicht-Sections in korrekter Reihenfolge. README.md folgt eigenem Phase-Schema (kein Detail-MD-Template) — wie geplant.
|
||||
|
||||
---
|
||||
|
||||
## 2. Dependencies-Konsistenz
|
||||
|
||||
### feature-flags.md zuerst → blocked-by-Liste
|
||||
|
||||
`feature-flags.md` ist als Foundation deklariert. Tatsächliche Konsumenten:
|
||||
- `ai-cost-phase-1.md` Depends: `feature-flags.md` (für `ai-usage-dashboard` Flag) ✓
|
||||
- `ai-cost-phase-2.md` Depends: `feature-flags.md` (für `ai-quota-soft-degradation`) ✓
|
||||
- `ai-cost-phase-3.md` Depends: `feature-flags.md` (für Per-Tenant-Overage-Enablement) ✓
|
||||
- `form-widget.md` Depends: `feature-flags.md` (für Turnstile-Gradual-Rollout) ✓
|
||||
- `realtime-collab.md` Depends: `feature-flags.md` (für Plan-Tier-Gating) ✓
|
||||
- `marketing-funnel-phase-1.md` Depends: `feature-flags.md` (Turnstile Gradual-Rollout) ✓
|
||||
- `marketing-funnel-phase-2.md` Depends: `feature-flags.md` (Outbound-Integration opt-in) ✓
|
||||
- `marketing-funnel-phase-3.md` Depends: `feature-flags.md` (Sequence-Aktivierung) ✓
|
||||
|
||||
→ konsistent.
|
||||
|
||||
### AI-Cost-Phasen sequenziell (1 → 2 → 3)
|
||||
|
||||
- Phase 2 Depends on Phase 1 ✓ (Z. 24: "AICostEvent-Pipeline + TenantAICostSummary als Datengrundlage")
|
||||
- Phase 3 Depends on Phase 2 + Phase 1 ✓ (Z. 24-25)
|
||||
- Phase 1 Blocks Phase 2 + Phase 3 ✓
|
||||
- Phase 2 Blocks Phase 3 ✓
|
||||
|
||||
### Marketing-Funnel-Phasen sequenziell (1 → 2 → 3)
|
||||
|
||||
- Phase 2 Depends on Phase 1 (Z. 23) ✓
|
||||
- Phase 3 Depends on Phase 2 (Z. 23) + Phase 1 ✓
|
||||
- Phase 1 Blocks Phase 2 + Phase 3 ✓
|
||||
- Phase 2 Blocks Phase 3 ✓
|
||||
|
||||
### React-Email aus Phase 10 für `marketing-funnel-phase-3.md`
|
||||
|
||||
- Z. 28: "MUSS aus Phase 10 vorgezogen werden ... Alternative: minimaler HTML-Fallback bis React-Email regulär kommt, dann Migration." ✓
|
||||
- Implementation-Step 1 (Z. 56-59): explicit "React-Email-Vorziehung (aus Phase 10)" als optionaler Stub ✓
|
||||
- README.md verweist auf `../10-compliance-polish/react-email.md` (Z. 109) ✓
|
||||
- Konsistent mit `00-execution-order.md` Z. 106: "react-email.md kann früher gezogen werden falls Phase 05 Marketing-Funnel-Phase-3 sie braucht" ✓
|
||||
|
||||
### AI-Cost-Phase-1 braucht Outbox aus Phase 03
|
||||
|
||||
- Phase 1 Depends: `03-scaling-infrastructure/outbox-pattern.md` (#42) ✓ (Z. 29)
|
||||
- Phase 1 Depends: `worker-queues.md` (#43) ✓
|
||||
- Phase 1 Depends: `read-write-split.md` (#41) ✓
|
||||
- Phase 1 Depends: `metrics-grafana.md` (#14) ✓
|
||||
|
||||
### feature-flags.md braucht SharedStore aus Phase 03
|
||||
|
||||
- Depends: `03-scaling-infrastructure/shared-store-package.md` (#37) ✓ (Z. 29)
|
||||
- Depends: `03-scaling-infrastructure/pubsub-cache-invalidation.md` (#39) ✓
|
||||
- Depends: `02-observability/sentry-wiring-api-worker.md` (#11) ✓
|
||||
|
||||
**Finding D1 (Info):** Alle Dependencies konsistent zwischen README.md, 00-execution-order.md und den einzelnen Detail-MDs. Keine zirkulären Verweise.
|
||||
|
||||
**Finding D2 (Minor):** `realtime-collab.md` referenziert `02-observability/opentelemetry.md` (#13) — korrekt, aber die `Blocks`-Sektion ist sehr dünn ("Spätere Erweiterung auf andere Editoren ... sind out-of-scope"). Kein echter Folge-Task wird hard-blockiert; akzeptabel, weil README Track-B als "Standalone" markiert.
|
||||
|
||||
---
|
||||
|
||||
## 3. Implementation-Steps-Qualität
|
||||
|
||||
### Feature-Flags-Targeting-Engine
|
||||
|
||||
**Erwartet:** Composite-Rules (AND/OR-tree), Percentage-Sticky via SHA1-Hash, JSON-Variants.
|
||||
|
||||
- Step 2 (Z. 64): "Targeting-Engine ... evaluateRule(rule, ctx) mit Support für: boolean, allowlist, percentage (sticky-hash), plan-tier, role-based, time-based, composite (AND/OR/NOT mit Child-Rules). Pure Function, vollständig unit-testbar ohne DB." ✓
|
||||
- Step 3 (Z. 67): "evaluateFlag + getFeatureFlagVariant Public-API ... orchestriert Storage-Load, Dependency-Resolution (Parent-Flag-Check), Targeting-Eval, Variant-Auswahl. Strikt typisierte Return-Werte." ✓
|
||||
- Approach Z. 56: "Percentage-Rollout via SHA1(`userId+flagName`) % 100: sticky pro User-Flag-Kombi" ✓
|
||||
- Step 5 (Z. 73): Visueller Rule-Builder, Percentage-Slider, Allowlist-Search, Variant-JSON-Editor mit Monaco + Schema-Validation ✓
|
||||
- Test Z. 90: "percentage-hash.test.ts — sticky-Test: gleicher (userId, flagName) liefert immer gleichen Bucket; Distribution-Test über 10k Sample-IDs liegt bei N% innerhalb ±2%" ✓
|
||||
|
||||
**Finding I1 (Minor):** Hash-Algo wird in Approach als "SHA1" benannt, in Tests aber nicht explizit als SHA1 vs SHA256 fixiert. Empfehlung: in Step 2 explizit fixieren (SHA-256 wäre moderner — SHA1 ist aber für Bucket-Hashing vollkommen ausreichend; egal welche Wahl, sollte konsistent dokumentiert sein).
|
||||
|
||||
### AI-Cost-Wrapper um LLM-Calls
|
||||
|
||||
**Erwartet:** Wrapper in allen Pfaden — chatbot-stream, generate-action, PromptBuilderService, RAG-Embedding.
|
||||
|
||||
- ai-cost-phase-1.md Problem-Section Z. 20-23 listet alle 4 Stellen mit Datei + Line ✓
|
||||
- Implementation Step 4 (Z. 67-73) instrumentiert alle 4 atomisch (eigener Commit pro Call-Site) ✓
|
||||
- Wrapper-Signatur explizit (Z. 64): `executeLLMCall({ provider, model, capability, ctx, run })` mit Idempotency-Key ✓
|
||||
- Provider-Wrapper liefert sinnvolle Inheritance: Provider-Usage-Numbers nutzen wenn vorhanden, sonst countTokens ✓
|
||||
|
||||
**Finding I2 (Info):** Vollständige Coverage der vier bekannten LLM-Pfade. Subagent-Recommendation (Z. 146-148) erinnert explizit, weitere Stellen mit `Explore`-Subagent zu suchen — guter Defensive-Step.
|
||||
|
||||
### Stripe-Meter-Events mit Reconciliation-Cron
|
||||
|
||||
- Implementation Step 4 (Z. 61-63): Nightly Cron, Idempotency-Identifier (UUID), `submittedToStripe=true`-Flag, `stripeMeterEventId` gespeichert ✓
|
||||
- Implementation Step 7 (Z. 70-72): AIOverageSubmission-Audit-Trail-Model ✓
|
||||
- Implementation Step 8 (Z. 73-75): Monatlicher Reconciliation-Cron, >1% Mismatch → Alert + Sentry-Capture, Runbook in Doku ✓
|
||||
- Approach Z. 47-48: Begründung Nightly statt Realtime (Stripe-Quotas) + Block-Cents-Aggregation (Mikro-Charges vermeiden) ✓
|
||||
- Security Note Z. 131: "Idempotency-Identifier (UUID pro Submission) ist Pflicht — Stripe deduped auf Identifier. Tests müssen explizit Retry-Pfad verifizieren." ✓
|
||||
- Rollback-Strategy Z. 124-127 inkl. `AI_OVERAGE_SUBMISSION_ENABLED=false` Killswitch + Backfill-Run ✓
|
||||
|
||||
→ Robust dokumentiert.
|
||||
|
||||
### Form-Widget mit CSP-safe iframe + SRI
|
||||
|
||||
- Approach Z. 45-47: "CDN-served loader-Script <3KB ... sandboxed iframe ... CSP-safe ... SRI für Tamper-Detection" ✓
|
||||
- Implementation Step 3 (Z. 66-68): Bootstrap-Loader mit SRI-Hash-Output für Embed-Code-Generator ✓
|
||||
- Implementation Step 4 (Z. 69-71): Embedded-Form-App im iframe (separater Bundle <80KB gzipped) ✓
|
||||
- Implementation Step 8 (Z. 81-83): Embed-Code-Generator mit `integrity="sha384-..."` Attribute ✓
|
||||
- Security Note Z. 151: CSP-Header für Embed-iframe definiert ("default-src 'self'; script-src 'self'; style-src 'self' 'unsafe-inline'") ✓
|
||||
- Security Note Z. 145: XSS-Mitigation (React default + Zod-Validation, kein freier HTML in Labels) ✓
|
||||
- Acceptance-Criterion Z. 122: PostMessage-Events `toolctrl:form:submitted/error/resize` ✓
|
||||
|
||||
**Finding I3 (Minor — siehe Lücken-Section 8):** Step 3 erwähnt `toolctrl:form:resize` für dynamic-height aber kein detailliertes Protokoll. Empfehlung siehe Lücken-Section.
|
||||
|
||||
### Hocuspocus mit Graceful-Degradation
|
||||
|
||||
- Approach Z. 40-41: "Graceful-Degradation — wenn HOCUSPOCUS_URL nicht gesetzt oder Service unreachable, Editor läuft als Single-User mit existing REST-Save" ✓
|
||||
- Implementation Step 8 (Z. 73-75): "Editor-Wrapper prüft Connection-State: bei disconnected > 10s zeige subtilen Status ... fallback auf REST-Save ... Wenn HOCUSPOCUS_URL env nicht gesetzt: Editor läuft komplett Single-User wie vorher (kein Provider initialisiert)" ✓
|
||||
- Acceptance-Criterion Z. 113: "Graceful-Degradation: ohne HOCUSPOCUS_URL läuft Editor Single-User, mit Service-Down zeigt Offline-Indikator" ✓
|
||||
- Rollback Z. 125-128: drei Disable-Optionen (Flag, Env, Container-Removal) ✓
|
||||
- E2E-Test Z. 97: "Offline-Test: Hocuspocus-Container stoppen → User sieht Offline-Indikator → Tippt weiter → Container zurück → Re-Sync ohne Datenverlust" ✓
|
||||
|
||||
→ Vollständig adressiert.
|
||||
|
||||
### Marketing-Funnel Lead-Capture (CORS + Anti-Spam)
|
||||
|
||||
- Implementation Step 2 (marketing-funnel-phase-1.md Z. 56): CORS-Whitelist gegen `MARKETING_DOMAINS` env, Honeypot, Turnstile, Rate-Limit (5/min/IP + 3/min/Email), Duplicate-Dedup-Logic ✓
|
||||
- Approach Z. 45-48: explicit Defense-in-Depth-Begründung ✓
|
||||
- Security Notes Z. 130-135: PII-Handling, Honeypot-Field-Naming aus Env, CORS als nur-Browser-Defense (Server-Bots umgehen → Rate-Limit zwingend), Email-Enumeration-Schutz, GDPR-Consent-Mechanismus, IP-Logging-Doku ✓
|
||||
|
||||
→ Robust.
|
||||
|
||||
### Self-Service-Conversion-Flow (Lead → Tenant via signed-Token)
|
||||
|
||||
- marketing-funnel-phase-2.md Implementation Step 1 (Z. 53-55): Signed-Lead-Token mit HMAC-SHA256, 7d-TTL ✓
|
||||
- Step 3 (Z. 59-61): Atomic Conversion-Service in Postgres-Transaction mit (a) Tenant-Create (b) Lead-Update (c) TenantAcquisition (d) Outbox-Event (e) Audit ✓
|
||||
- Step 5 (Z. 65-67): Signup-Action-Integration ruft Conversion-Service ✓
|
||||
- Security Notes Z. 141-147 adressieren: Token-Replay (Idempotency via `convertedToTenantId`-Check), Email-Read-Only-bei-Pre-Fill (Lead-Hijacking-Schutz), Race-Condition (Unique-Index auf `Lead.convertedToTenantId`) ✓
|
||||
|
||||
→ Sicher.
|
||||
|
||||
---
|
||||
|
||||
## 4. Test-Requirements-Qualität
|
||||
|
||||
### Per-Feature Konkretheit
|
||||
|
||||
Alle 9 Files haben strukturierte Test-Sections: Unit / Integration / E2E mit konkreten Cases. Coverage-Targets numerisch (z.B. 90% für `packages/ai/src/cost/`, 80% für Worker-Handlers).
|
||||
|
||||
### E2E-Tests für Lead-Conversion-Flow
|
||||
|
||||
- marketing-funnel-phase-2.md Z. 101-105 spezifiziert:
|
||||
- SDK + Form-Submit → Lead → Admin "Convert to Tenant" → Tenant erscheint
|
||||
- Self-Service-Flow: Capture → signed-Token-Link → pre-filled Signup → completed → Tenant + Welcome-Email
|
||||
- Integration-UI: Slack-Webhook → Test-Send-Button → Mock-Slack empfängt
|
||||
- Funnel-Report: Test-Daten mit definierter Source/Campaign → Reports zeigen korrekte Conversion-Rates ✓
|
||||
|
||||
### Multi-User-Tests für Realtime-Collab
|
||||
|
||||
- realtime-collab.md Z. 91-93 (Integration) + Z. 95-98 (E2E):
|
||||
- Two-Browser-Tab-Test: User A tippt → User B sieht innerhalb 1s; Cursor sichtbar
|
||||
- Awareness mit zwei User-States
|
||||
- Offline/Re-Sync-Test
|
||||
- Permission-Refusal-Test ✓
|
||||
- Staging-Test Z. 174: "Last-Test: 10 simultane Connections auf gleichem Doc — Sync-Latenz <300ms p95" ✓
|
||||
|
||||
**Finding T2 (Info):** Test-Sections sind konsistent qualitativ hochwertig. Coverage-Gates klar pro Modul-Scope.
|
||||
|
||||
---
|
||||
|
||||
## 5. Staging-Deployment-Steps
|
||||
|
||||
### Hocuspocus-Container
|
||||
|
||||
- realtime-collab.md Z. 155-168: docker-compose-Snippet mit Image, Env, Ports, depends_on, Traefik-Label für `wss://collab.staging.toolctrl.de` ✓
|
||||
- TLS-Termination + Subdomain-Routing dokumentiert ✓
|
||||
- Last-Test-Anweisung 10 concurrent connections ✓
|
||||
|
||||
### Form-Widget CDN-Upload
|
||||
|
||||
- form-widget.md Z. 169: "Asset-CDN (aus Phase 03 #46) muss bereit sein; Loader-Build-Output dorthin deployed (CI-Step)" ✓
|
||||
- form-widget.md Implementation Step 3 (Z. 66-68): Build via esbuild/vite, SRI-Hash-Output ✓
|
||||
- Bundle-Size-Verify: `widget.js` < 3KB gzipped via curl-Check ✓
|
||||
- Cross-Browser-Smoke ✓
|
||||
|
||||
### Marketing-SDK-CDN
|
||||
|
||||
- marketing-funnel-phase-1.md Z. 152: "Asset-CDN für marketing.js deployed (CI-Step)" ✓
|
||||
- Implementation Step 4 (Z. 62-64): CI-Step erweitert mit versioniertem Pfad (`marketing.v1.js`) + Latest-Symlink + SRI-Hash ✓
|
||||
- Bundle-Size-Verify <5KB gzipped via curl ✓
|
||||
|
||||
**Finding S1 (Info):** Alle drei Externe-Asset-Deployments haben klare Staging-Steps. CI-Workflow-Files (`.github/workflows/deploy.yml`) als Touchpoint dokumentiert.
|
||||
|
||||
---
|
||||
|
||||
## 6. Konsistenz mit User-Wünschen
|
||||
|
||||
### "Vollausbau Feature-Flags" — 9 Aspekte
|
||||
|
||||
Original-Wunsch laut Phase-Scope: Gradual-Rollouts, Plan-Tier-Gating, A/B-Tests (Multi-Variant), Killswitches mit Time-Boxing, Dependency-Hierarchien.
|
||||
|
||||
Spec liefert 7 Targeting-Rule-Types (Z. 64) + 5 Killer-Patterns (Z. 82):
|
||||
1. boolean ✓
|
||||
2. allowlist (tenantIds/userIds/emails) ✓
|
||||
3. percentage (sticky-hash) ✓ → Gradual-Rollouts
|
||||
4. plan-tier ✓ → Plan-Tier-Gating
|
||||
5. role-based ✓
|
||||
6. time-based (enabledFrom/until) ✓ → Killswitches mit Time-Boxing
|
||||
7. composite (AND/OR/NOT mit Child-Rules) ✓
|
||||
- Variants ✓ → A/B-Tests
|
||||
- parentFlagId / Dependency-Resolution ✓ → Dependency-Hierarchien
|
||||
- Audit-Trail ✓
|
||||
|
||||
→ **Alle 9 Aspekte abgedeckt** (7 Rule-Types + Variants + Dependencies + Audit als Bonus).
|
||||
|
||||
### "AI-Cost alle 3 Phasen"
|
||||
|
||||
- Phase 1 (Sichtbarkeit): 10 Implementation-Steps, Token-Counter + Wrapper + Outbox + Worker + Aggregator + 2 Dashboards + Metrics + Doku ✓
|
||||
- Phase 2 (Schutz): 10 Implementation-Steps, Estimate + Assert + 402-Handling + Cron-Alerts + On-Demand-Trigger + Tenant-UI + Admin-Override + Doku ✓
|
||||
- Phase 3 (Monetarisierung): 10 Implementation-Steps, Markup + Overage-Mark + Nightly-Cron-Submit + Stripe-Docs + Period-UI + Audit + Reconcile + Tenant-Override + Doku ✓
|
||||
|
||||
Alle drei Phasen detailliert (15k+ Zeichen pro Datei), sequenziell mit klaren Übergaben.
|
||||
|
||||
### "A für Marketing-Funnel + Email-Sequences + Self-Service-Conversion"
|
||||
|
||||
- marketing-phase-1: Lead-Capture-API + Marketing-SDK + Admin-Inbox ✓
|
||||
- marketing-phase-2: Self-Service-Conversion + Attribution + Outbound-Webhooks ✓
|
||||
- marketing-phase-3: Email-Sequence-Engine + 4 Pre-Built Sequences + Statistics ✓
|
||||
|
||||
Alle drei User-Wünsche jeweils in eigener Phase abgedeckt.
|
||||
|
||||
### "Form-Widget für Lead-Capture im CRM" — korrekt routed?
|
||||
|
||||
- form-widget.md Implementation Step 6 (Z. 75-77): Worker-Handler routet je nach `targetType`:
|
||||
- `form-submission` → Generic-Row
|
||||
- `crm-contact` → upsert via existing CRM-Service mit Field-Mapping
|
||||
- `entity-row` → insert Row in Target-Entity ✓
|
||||
- Form-Builder-UI hat Field-Mapping-Builder bei `crm-contact`/`entity-row` (Z. 78) ✓
|
||||
- README.md Z. 39 listet form-widget als Block für marketing-funnel-phase-1: "Form-Widget liefert Patterns für Cross-Origin + Anti-Spam, die wiederverwendet werden" ✓
|
||||
|
||||
→ Routed.
|
||||
|
||||
### "Hocuspocus Self-Hosted mit Graceful-Degradation"
|
||||
|
||||
→ Bereits in Section 3 ausführlich verifiziert. Architektur-Wahl, Container-Setup, Env-Disable, Container-Removal alle dokumentiert.
|
||||
|
||||
---
|
||||
|
||||
## 7. Security-Notes-Coverage
|
||||
|
||||
### Self-loop MCP von Chatbot-V1/V2 → Capability-System
|
||||
|
||||
Nicht explizit in Phase-05-Files adressiert — gehört in Phase 07. Phase 05 liefert nur die Feature-Flag-Engine, die als "Capability-Gating" verwendet werden kann.
|
||||
|
||||
**Finding SEC1 (Info):** Self-loop-MCP-Schutz ist Phase-07-Konzern (`chatbot-v1.md`/`chatbot-v2.md`). Phase 05 erwähnt korrekterweise in `feature-flags.md` Blocks-Section: "06-plugin-system/hooks-lifecycle.md (#61) — Plugin-Capability-Gating" — Feature-Flag-Engine ist die Foundation. Kein explizites Self-loop-Issue zu adressieren in Phase 05.
|
||||
|
||||
### Lead-Capture-Anti-Spam — Honeypot + Turnstile + Rate-Limit
|
||||
|
||||
marketing-funnel-phase-1.md Z. 130-135 + 56:
|
||||
- Honeypot ✓
|
||||
- Turnstile (opt-in via Flag) ✓
|
||||
- Rate-Limit (5/min/IP + 3/min/Email) ✓
|
||||
- IP-Logging dokumentiert + DSGVO-Doku-Update verwiesen ✓
|
||||
- Honeypot-Field-Name aus Env (HONEYPOT_FIELD_NAME) randomized — sehr nice ✓
|
||||
|
||||
→ Ausreichend.
|
||||
|
||||
### Form-Submissions-CORS — strict whitelist
|
||||
|
||||
form-widget.md Z. 72 + 146-151:
|
||||
- Per-Form `allowedOriginDomains` String[] ✓
|
||||
- Origin-Header-Check im Submission-API ✓
|
||||
- Wildcard-Support (`*.example.com`) getestet ✓
|
||||
- Webhook-SSRF-Check (keine internen IPs) ✓
|
||||
- CSP-Header für iframe ✓
|
||||
|
||||
→ Strict.
|
||||
|
||||
---
|
||||
|
||||
## 8. Fehlende Aspekte (Lücken-Findings)
|
||||
|
||||
### Finding L1 (Minor) — AI-Cost Provider-Pricing-Tables Update-Strategie
|
||||
|
||||
ai-cost-phase-1.md Z. 54-55 + 86-89: Pricing-Tables sind versioniert (`pricing/v2026-05-16.ts`) und im Repo-Commit (kein DB-Override). Security-Note Z. 140: "Pricing-Files im Repo-Commit, kein DB-Override → Audit-Trail über git-history."
|
||||
|
||||
**Was fehlt:** Update-Workflow ist nicht explizit dokumentiert. Wenn OpenAI z.B. am 1. Juni einen neuen Preis veröffentlicht, was ist der Operator-Workflow?
|
||||
- Wer triggert Pricing-Update?
|
||||
- Wie wird historische Cost-Migration für past-events gehandhabt (Z. 132: "neue Pricing-Datei mit Korrektur + Cron-Job zur Neuberechnung historischer Events (separater fix-Task)" — bleibt vage)?
|
||||
- Gibt es einen "Pricing-Drift-Detector" der bei großen Sprüngen warnt?
|
||||
|
||||
**Empfehlung:** In `docs/features/ai-cost-billing.md` (Phase-1-Doku, Z. 89) explizit einen "Pricing-Update-Runbook" hinzufügen. Beispiel-Sections:
|
||||
1. Wann updaten (monatlicher Provider-Doc-Check)
|
||||
2. Wie updaten (neue Pricing-File hinzufügen, alte nicht überschreiben)
|
||||
3. Backfill-Strategie (nur falls Pricing-Bug korrigiert; Provider-Preisänderung wirkt nur prospektiv)
|
||||
4. Tests für Pricing-File-Loading
|
||||
|
||||
Kein Blocker — kann während Step 10 (Doku) ergänzt werden.
|
||||
|
||||
### Finding L2 (Minor) — Form-Widget iframe-Size-Negotiation via postMessage
|
||||
|
||||
form-widget.md Z. 67 erwähnt: "hört auf PostMessage `toolctrl:form:resize` für dynamic-height". Auch in Z. 127 als Acceptance-Criterion (`toolctrl:form:resize` funktional).
|
||||
|
||||
**Was fehlt:** Detailliertes Protokoll für iframe-Auto-Resize. Bei Form-Builder-Flexibilität (Felder werden hinzugefügt/entfernt, Conditional-Logic zeigt/versteckt Felder) kann iframe-Höhe stark variieren.
|
||||
|
||||
**Empfehlung:** Step 4 (Embedded-Form-App) sollte ResizeObserver auf Body verwenden + PostMessage mit `{ type: 'toolctrl:form:resize', height: number }` an Parent emittieren. Step 3 (Loader) muss diesen Event empfangen und `iframe.style.height = ...px` setzen. Width sollte 100% bleiben (responsive).
|
||||
|
||||
**Konkrete Spec-Ergänzung:**
|
||||
```
|
||||
PostMessage-Protokoll:
|
||||
- Child → Parent: { type: 'toolctrl:form:resize', height: number }
|
||||
- Child → Parent: { type: 'toolctrl:form:submitted', leadId?: string, redirectUrl?: string }
|
||||
- Child → Parent: { type: 'toolctrl:form:error', message: string }
|
||||
- Parent → Child: { type: 'toolctrl:form:theme-update', themeJson: object } (optional)
|
||||
```
|
||||
|
||||
Kann in Step 3+4 ergänzt werden; kein Blocker, aber implementer wird sonst raten.
|
||||
|
||||
### Finding L3 (Minor) — `Lessons Learned`-Section optional aber nirgends gefüllt
|
||||
|
||||
Alle 9 Detail-MDs lassen `Lessons Learned` weg. Template Z. 128: "Lessons Learned (optional, am Ende des Tasks für nachfolgende Tasks)" → wird vom ausführenden Agent post-completion gefüllt. Keine Aktion nötig vor Execution.
|
||||
|
||||
### Finding L4 (Info) — Phase-3 Cron-Trigger benötigt eigenen Trigger-Event-Pfad
|
||||
|
||||
marketing-funnel-phase-3.md Z. 78 erwähnt "Lost Lead Re-Activation" Trigger als "Cron-Job 60d-stale-leads scheint matching trigger". Implementation Step 11 (Z. 92-94) konkretisiert: Cron emit-Outbox-Event `lead.stale_60d` + `tenant.trial_expiring`.
|
||||
|
||||
**Konsistenz-Prüfung:** Step 2 (Z. 60) listet `triggerEvent` enum: `lead.captured|lead.qualified|tenant.signup|tenant.trial_expiring|tenant.cancelled|custom`. **`lead.stale_60d` fehlt im Enum!**
|
||||
|
||||
**Empfehlung:** Enum erweitern um `lead.stale_60d` oder als `custom` mit Custom-Trigger-Name behandeln. Andere Cron-emittierte Events ähnlich. Kann während Implementation korrigiert werden, aber sollte als Deviation festgehalten werden.
|
||||
|
||||
### Finding L5 (Minor) — Form-Widget: File-Upload nicht in Phase 1
|
||||
|
||||
form-widget.md Security-Note Z. 147: "File-Upload (out-of-scope Phase 1): wenn File-Field später hinzukommt, dann AV-Scan-Layer aus Phase 01 Tasks #31/#32 einbinden."
|
||||
|
||||
→ Akzeptable Einschränkung. Klar als out-of-scope markiert. Field-Type-Liste in Step 2 (Z. 64) erwähnt `file optional`. Konsistent.
|
||||
|
||||
### Finding L6 (Info) — Realtime-Collab Plan-Tier-Gating nicht detailliert
|
||||
|
||||
realtime-collab.md Depends Z. 25 erwähnt "Plan-Tier-Gating", aber Implementation-Steps spezifizieren keine konkrete Plan-Tier-Quota (z.B. "Free: keine Collab, Pro: 5 concurrent Collab-Editors, Enterprise: unlimited"). Security-Note Z. 137: "Limit per Tenant via Hocuspocus-onConnect-Hook prüft Concurrent-Connection-Count gegen Plan-Quota. Default 50/Tenant."
|
||||
|
||||
**Empfehlung:** Wäre wünschenswert, Plan-Tier-Quota in Entitlements-Schema (`packages/entitlements/`) wie in ai-cost-phase-2 zu definieren — z.B. `collabConcurrentEditorsPerTenant`. Kann während Implementation als Deviation ergänzt werden.
|
||||
|
||||
---
|
||||
|
||||
## 9. Phase-übergreifende Konsistenz
|
||||
|
||||
### `.env.example`-Keys
|
||||
|
||||
README Done-Definition (Z. 81) listet: `HOCUSPOCUS_URL`, `HOCUSPOCUS_SECRET`, `MARKETING_DOMAINS`, `FORM_WIDGET_CDN_URL`, `STRIPE_AI_OVERAGE_METER_ID` (optional), `TURNSTILE_SITE_KEY`, `TURNSTILE_SECRET_KEY`.
|
||||
|
||||
Cross-check pro File:
|
||||
- feature-flags.md: keine neuen Keys (Cache via SharedStore = existing Redis) — implicit, kein Issue
|
||||
- ai-cost-phase-1.md: keine Keys explizit — `AI_COST_TRACKING_ENABLED` in Rollback erwähnt aber nicht in `.env.example`-Step
|
||||
- ai-cost-phase-2.md: `AI_QUOTA_ENFORCEMENT_ENABLED`, `AI_QUOTA_ALERTS_ENABLED` in Rollback — nicht explizit ins `.env.example` aufgenommen
|
||||
- ai-cost-phase-3.md: `STRIPE_AI_OVERAGE_METER_ID` ✓ + `AI_OVERAGE_SUBMISSION_ENABLED`
|
||||
- form-widget.md: `FORM_WIDGET_CDN_URL`, `TURNSTILE_SITE_KEY`, `TURNSTILE_SECRET_KEY` ✓ (Z. 129)
|
||||
- realtime-collab.md: `HOCUSPOCUS_URL`, `HOCUSPOCUS_SECRET` ✓ (Z. 116)
|
||||
- marketing-funnel-phase-1.md: `MARKETING_DOMAINS`, `TURNSTILE_*`, `MARKETING_SDK_CDN_URL`, `HONEYPOT_FIELD_NAME` ✓
|
||||
- marketing-funnel-phase-2.md: `MARKETING_TOKEN_SECRET`, `LEAD_CONVERSION_ENABLED`, `MARKETING_WEBHOOKS_ENABLED` ✓
|
||||
- marketing-funnel-phase-3.md: `MARKETING_SEQUENCES_ENABLED` (Rollback)
|
||||
|
||||
**Finding ENV1 (Minor):** Rollback-Killswitch-Env-Vars (`AI_COST_TRACKING_ENABLED`, `AI_QUOTA_ENFORCEMENT_ENABLED`, `LEAD_CONVERSION_ENABLED`, etc.) erscheinen nur in Rollback-Sections, sind aber nicht in `.env.example`-Acceptance-Criteria gelistet. Empfehlung: Per Detail-MD ein expliziter Acceptance-Criterion "`.env.example` enthält {alle Toggle-Keys}" hinzufügen — sonst werden sie vergessen.
|
||||
|
||||
### Outbox-Event-Naming-Konsistenz
|
||||
|
||||
Verwendete Event-Namen über die Files:
|
||||
- `ai-cost.persist-event` (Phase 1/2/3)
|
||||
- `ai-cost.aggregate-summary` (Phase 1)
|
||||
- `ai-quota.alert` (Phase 2)
|
||||
- `form.submission.process` (Form-Widget)
|
||||
- `marketing.lead.webhook` (Marketing-Phase-2)
|
||||
- `marketing.sequence.start` / `marketing.sequence.send` (historical Marketing-Phase-3 spec names; current #80 docs use `email-sequence-trigger` / `email-sequence-scheduler`, with production send still deferred)
|
||||
- `lead.captured`, `lead.qualified`, `lead.converted_to_trial`, `lead.converted_to_customer` (Phase 2 → Phase 3)
|
||||
- `lead.stale_60d`, `tenant.trial_expiring`, `tenant.signup`, `tenant.cancelled` (Phase 3 Cron)
|
||||
|
||||
Naming-Pattern: `{domain}.{entity}.{action}` (kebab/dot) vs `{entity}.{action}` (für Lifecycle-Events).
|
||||
|
||||
**Finding EVT1 (Info):** Naming-Konvention nicht 100% einheitlich (z.B. `ai-cost.persist-event` mit dash vs `ai-quota.alert` mit dot vs `lead.captured`). Akzeptabel, da intuitiv lesbar — aber könnte in einem späteren Cleanup vereinheitlicht werden.
|
||||
|
||||
---
|
||||
|
||||
## 10a. 2026-05-27 Status-Reconciliation
|
||||
|
||||
This original verification report remains useful as a pre-execution spec audit,
|
||||
but several rows are now historical. Current status must be read from
|
||||
`../PROGRESS.md` plus `../OPERATOR-CHECKLIST.md`.
|
||||
|
||||
- Marketing Funnel Phase 3 / #80 is still `in_progress`. The shipped surface is
|
||||
the data model, pure engine, four template definitions, scheduler foundation,
|
||||
tracking, unsubscribe, admin UI/CLI, live stats dashboard, and
|
||||
stale-lead/trial-expiring runner functions. It is **not** production
|
||||
marketing-sequence delivery until the delayed producer, sequence renderer,
|
||||
`@toolctrl/mail` adapter, seed data, stats aggregation cron, Playwright E2E,
|
||||
and production scheduled-job registration/deploy evidence land.
|
||||
- The canonical email-sequence status is `docs/features/email-sequences.md`.
|
||||
Use its model names (`EmailSequence`, `EmailSequenceRun`,
|
||||
`EmailSequenceStepDelivery`) and admin route (`/admin/email-sequences`) when
|
||||
reconciling stale references in this report or downstream plans.
|
||||
- The original "Lead-Capture -> ... -> erste Email versendet" expectation
|
||||
remains a future acceptance target, not current live proof. Current safe
|
||||
wording is: lead capture/conversion can create or display sequence state;
|
||||
production mail sending is deferred behind #80 follow-ups.
|
||||
|
||||
## 10. Zusammenfassung
|
||||
|
||||
### Pass-Status
|
||||
|
||||
**PASS.** Alle 10 Files waren zum Zeitpunkt dieser historischen Planprüfung plan-quality-ready für Execution: Template-Konformität 100%, Dependencies konsistent, Implementation-Steps atomar und konkret genug für Subagent-Implementation, Test-Requirements präzise, Staging-Steps machbar. Das ist keine Aussage, dass Email-Sequences runtime-production-ready sind; der aktuelle Runtime-Status steht in `docs/features/email-sequences.md`.
|
||||
|
||||
### Findings-Übersicht
|
||||
|
||||
| ID | Severity | Datei | Beschreibung |
|
||||
|---|---|---|---|
|
||||
| T1 | Info | alle | Template-Konformität korrekt; Lessons-Learned optional weggelassen |
|
||||
| D1 | Info | alle | Dependencies konsistent |
|
||||
| D2 | Minor | realtime-collab.md | `Blocks`-Section dünn — akzeptabel |
|
||||
| I1 | Minor | feature-flags.md | Hash-Algo (SHA1) sollte in Step 2 explizit fixiert sein |
|
||||
| I2 | Info | ai-cost-phase-1.md | LLM-Pfad-Coverage vollständig |
|
||||
| I3 | Minor | form-widget.md | PostMessage-Resize-Protokoll nicht spezifiziert (siehe L2) |
|
||||
| T2 | Info | alle | Test-Sections konsistent qualitativ hochwertig |
|
||||
| S1 | Info | 3 Files | Staging-Steps für externe Assets vollständig |
|
||||
| SEC1 | Info | feature-flags.md | Self-loop-MCP-Schutz korrekt nach Phase 07 verschoben |
|
||||
| L1 | Minor | ai-cost-phase-1.md | Pricing-Update-Runbook fehlt explizit |
|
||||
| L2 | Minor | form-widget.md | iframe-Resize-PostMessage-Protokoll konkretisieren |
|
||||
| L3 | Info | alle | `Lessons Learned` post-Execution füllen |
|
||||
| L4 | Minor | marketing-funnel-phase-3.md | `lead.stale_60d` fehlt im triggerEvent-Enum |
|
||||
| L5 | Info | form-widget.md | File-Upload explizit out-of-scope (OK) |
|
||||
| L6 | Info | realtime-collab.md | Plan-Tier-Quota Detail wünschenswert |
|
||||
| ENV1 | Minor | mehrere | Rollback-Killswitch-Env-Vars nicht in Acceptance-Criteria gelistet |
|
||||
| EVT1 | Info | mehrere | Outbox-Event-Naming nicht 100% einheitlich (akzeptabel) |
|
||||
|
||||
### Konkrete Empfehlungen vor Execution-Start
|
||||
|
||||
1. **L4 (Pflicht):** `marketing-funnel-phase-3.md` Z. 60 — Enum-Erweiterung um `lead.stale_60d` (oder Cron-Trigger-Custom-Variante explizit definieren) — sonst Mismatch zwischen Step 2 und Step 11.
|
||||
2. **L2 (Empfohlen):** `form-widget.md` Step 3+4 — PostMessage-Protokoll-Detail spezifizieren (4 Message-Types).
|
||||
3. **ENV1 (Empfohlen):** Detail-MDs um zusätzlichen Acceptance-Criterion "`.env.example` enthält alle Killswitch-Env-Vars" ergänzen.
|
||||
4. **L1 (Empfohlen):** `ai-cost-phase-1.md` Step 10 (Doku) — explizit "Pricing-Update-Runbook" als Pflicht-Section.
|
||||
5. **I1 (Optional):** `feature-flags.md` Step 2 — Hash-Algo (SHA-256 empfohlen) explizit fixieren.
|
||||
|
||||
Alle anderen Findings sind Info-Level oder können während Implementation als Deviations dokumentiert werden.
|
||||
|
||||
### Bereitschaftserklärung
|
||||
|
||||
Phase 05 ist **executable**. Subagents können auf Basis der Spec ohne weitere Architektur-Entscheidungen implementieren. Vor allem die Cross-File-Konsistenz (Dependencies, README-Done-Definition, Track-Aufteilung) ist sauber. Die ~16 Findings sind allesamt iterativ während der Execution behebbar — keine architectural redesigns nötig.
|
||||
@@ -0,0 +1,314 @@
|
||||
# AI-Cost-Tracking — Phase 1: Token-Counter + Event-Pipeline + Dashboards
|
||||
|
||||
**Status:** completed
|
||||
**Task-ID:** #54
|
||||
**Phase:** 05
|
||||
**Estimated Effort:** 1 Woche
|
||||
**Owner:** unassigned
|
||||
|
||||
## Problem
|
||||
|
||||
Aktuell hat die Boilerplate keinerlei Sichtbarkeit über AI-Provider-Kosten. Jeder LLM-Call (Chatbot-Stream, Generate-Action, PromptBuilder, RAG-Embedding) zahlt der Boilerplate-Betreiber selbst — ohne zu wissen welcher Tenant/User wie viel verursacht. Konkrete Symptome:
|
||||
|
||||
- Ein einziger missbräuchlicher Tenant kann via Chatbot-Spam die monatliche OpenAI-Rechnung verzigfachen ohne Alarm.
|
||||
- Keine Möglichkeit Margin-Analyse zu fahren (Tenant zahlt 49€/Monat — verursacht aber 200€ OpenAI-Kosten?).
|
||||
- Keine Grundlage für Phase-2 (Quota) und Phase-3 (Overage-Billing).
|
||||
- Keine Tenant-Sichtbarkeit ("wie viel AI verbrauche ich diesen Monat?").
|
||||
|
||||
Betroffene Code-Stellen heute:
|
||||
|
||||
- `apps/web/app/modules/chatbot/services/chatbot-stream.server.ts:81-100` — direkter OpenAI-Call ohne Tracking
|
||||
- `apps/web/app/modules/generation/.server/generate-action.server.ts` — Generate-Hot-Path ohne Tracking
|
||||
- `packages/ai/src/services/PromptBuilderService.ts:361-369` — interner LLM-Call für Prompt-Synthese
|
||||
- `packages/ai/src/rag/embedding-pipeline.ts` (aus Phase 04) — Embedding-Calls ohne Cost-Attribution
|
||||
|
||||
Ohne diese Pipeline ist Phase 05 ab Phase 2 blockiert und das Produkt nicht skalierbar zu monetarisieren.
|
||||
|
||||
## Depends on
|
||||
|
||||
- `03-scaling-infrastructure/outbox-pattern.md` (#42) — AICostEvents müssen via Outbox geschrieben werden. Synchroner DB-Write bei jedem LLM-Call würde Provider-Latenz künstlich erhöhen und bei DB-Ausfall den LLM-Hot-Path brechen.
|
||||
- `03-scaling-infrastructure/worker-queues.md` (#43) — Aggregation-Cron läuft im `worker-general`.
|
||||
- `03-scaling-infrastructure/read-write-split.md` (#41) — Dashboard-Queries laufen über `prismaRead`, Aggregation-Worker schreibt via `prismaWrite`.
|
||||
- `02-observability/metrics-grafana.md` (#14) — Metrics-Export `ai_cost_cents_total{tenant,provider,model,capability}` ergänzt die DB-Aggregation um Echtzeit-Beobachtung.
|
||||
- `05-business-features/feature-flags.md` (#53) — Tenant-Dashboard hinter Flag `ai-usage-dashboard` für Gradual-Rollout.
|
||||
|
||||
## Blocks
|
||||
|
||||
- `05-business-features/ai-cost-phase-2.md` (#55) — Quota-Check braucht `currentMonthSpend`-Query, die hier aufgesetzt wird.
|
||||
- `05-business-features/ai-cost-phase-3.md` (#56) — Overage-Stripe-Sync braucht `AICostEvent`-Pipeline.
|
||||
- `07-chatbot/chatbot-v2.md` (#69) — V2-Chatbot referenziert Cost-Dashboard explizit in der UI.
|
||||
|
||||
## Approach
|
||||
|
||||
Drei-Schicht-Pipeline:
|
||||
|
||||
1. **Counter + Cost-Calculator** als pure Functions in `packages/ai/src/cost/` — keine I/O, vollständig unit-testbar. `countTokens(text, model)` nutzt tiktoken (für OpenAI) oder model-spezifische Encoder; `countCallCost({ provider, model, inputTokens, outputTokens })` returnt cents basierend auf einer versionierten Pricing-Table.
|
||||
2. **Provider-Wrapper + Outbox-Producer** — alle bestehenden LLM-Call-Stellen werden auf einen einheitlichen `executeLLMCall(provider, params, ctx)`-Wrapper umgezogen. Wrapper macht den Provider-Call, zählt Tokens, schreibt `AICostEvent` in die Outbox.
|
||||
3. **Worker-Persistierung + Aggregation** — `ai-cost.persist-event`-Handler schreibt Event in `AICostEvent`-Table. Cron-Job `ai-cost.aggregate-summary` aggregiert nightly + on-demand in `TenantAICostSummary` (für schnelle Dashboard-Lookups ohne Vollaggregation).
|
||||
|
||||
**Begründung der Architektur-Wahl:**
|
||||
|
||||
- **Eigene Cost-Calc statt OpenAI-Usage-API:** OpenAI liefert Usage zwar in Response, aber andere Provider (Anthropic, Mistral, Scaleway, Aleph-Alpha) handhaben es uneinheitlich. Eigene Calc gibt uns konsistente Cost-Attribution über alle Provider und ist unabhängig von Provider-API-Änderungen.
|
||||
- **Outbox statt direkter DB-Write:** LLM-Calls dauern 100ms-30s, jede zusätzliche DB-Round-Trip kostet. Outbox-Insert ist in laufender Transaction günstig, Worker persistiert async.
|
||||
- **Separate Summary-Table:** Dashboard-Query "Tenant-Monatsausgaben" auf großer AICostEvent-Tabelle (10M+ Rows nach 1 Jahr) würde >1s dauern. Vorberechnete Summary-Rows + Cron-Refresh = <50ms.
|
||||
- **Pricing-Table versioniert (`pricing/v2026-05-16.ts`):** Provider ändern Preise — wir wollen historische Kosten korrekt erhalten. Lookup nimmt das zum `createdAt` aktuelle Pricing.
|
||||
|
||||
## Implementation Steps
|
||||
|
||||
1. **Prisma-Models** — `AICostEvent { id, tenantId, userId?, capability, provider, model, inputTokens, outputTokens, costCents, requestId, isOverage Bool default false, createdAt }` mit Indexes auf `(tenantId, createdAt)`, `(tenantId, capability, createdAt)`, `(provider, model, createdAt)`. Plus `TenantAICostSummary { id, tenantId, periodStart, periodEnd, totalCostCents, totalInputTokens, totalOutputTokens, breakdownJson Json, computedAt }` mit Unique-Index `(tenantId, periodStart)`. Migration generieren.
|
||||
- Files: `prisma/schema.prisma`, `prisma/migrations/<ts>_ai_cost_tracking/migration.sql`
|
||||
- Commit: `05/ai-cost-phase-1: add AICostEvent and TenantAICostSummary models`
|
||||
2. **Token-Counter + Pricing-Table** — `packages/ai/src/cost/tokens.ts` mit `countTokens(text, model)` via tiktoken (für `gpt-*`, `o*`) und fallback-encoder (für Anthropic via `@anthropic-ai/tokenizer`, für unbekannte Models über 4-chars-per-token-Heuristik mit Warning). `packages/ai/src/cost/pricing/v2026-05-16.ts` mit Pricing-Map `{ provider: { model: { inputCentsPer1M, outputCentsPer1M } } }`. `countCallCost({ provider, model, inputTokens, outputTokens, pricedAt? = now })` resolved Pricing-Version und rechnet.
|
||||
- Files: `packages/ai/src/cost/tokens.ts`, `packages/ai/src/cost/pricing/v2026-05-16.ts`, `packages/ai/src/cost/index.ts`, `packages/ai/src/cost/*.test.ts`
|
||||
- Commit: `05/ai-cost-phase-1: token counter and pricing table`
|
||||
3. **Provider-Wrapper** — `packages/ai/src/cost/executeLLMCall.ts` wrappt jeden Provider-Call. Signatur: `executeLLMCall({ provider, model, capability, ctx: { tenantId, userId?, requestId }, run: () => Promise<{ text: string, inputTokens?, outputTokens? }> })`. Nach `run()`: wenn Provider Usage-Numbers liefert nutze sie, sonst countTokens; berechne Cost; schreibe `AICostEvent` in Outbox via `enqueueOutboxEvent({ type: 'ai-cost.persist-event', payload })`. Idempotency-Key: `requestId`.
|
||||
- Files: `packages/ai/src/cost/executeLLMCall.ts`, `packages/ai/src/cost/executeLLMCall.test.ts`
|
||||
- Commit: `05/ai-cost-phase-1: executeLLMCall wrapper with outbox emit`
|
||||
4. **Migration der LLM-Call-Stellen auf Wrapper** — atomisch pro Call-Site, jeweils mit Tests dass Verhalten unverändert:
|
||||
- `apps/web/app/modules/chatbot/services/chatbot-stream.server.ts:81-100` — Streaming-Call ummantelt
|
||||
- `apps/web/app/modules/generation/.server/generate-action.server.ts` — Generate-Action
|
||||
- `packages/ai/src/services/PromptBuilderService.ts:361-369` — interner Synth-Call
|
||||
- `packages/ai/src/rag/embedding-pipeline.ts` — Embed-Call (Capability `rag-embed`)
|
||||
- Files: jeweils oben + zugehörige Tests
|
||||
- Commit: ein Commit pro Call-Site (`05/ai-cost-phase-1: instrument chatbot-stream with cost wrapper`, etc.)
|
||||
5. **Worker-Handler `ai-cost.persist-event`** — registriert im `worker-general`. Persistiert Event in `AICostEvent`-Table. Idempotency über `requestId` (Unique-Index optional; sonst `upsert`). Metrics-Counter `ai_cost_events_persisted_total` inkrementieren.
|
||||
- Files: `apps/worker/src/handlers/ai-cost.persist-event.ts`, Test-File
|
||||
- Commit: `05/ai-cost-phase-1: worker handler for AICostEvent persistence`
|
||||
6. **Cron-Aggregator `ai-cost.aggregate-summary`** — läuft stündlich (current-month) + täglich (rolling 30d). Aggregiert AICostEvents pro Tenant in `TenantAICostSummary` mit Breakdown nach Capability/Model/User (Top-N). Pflegt nur "current period" + jeweiligen Vor-Monat. Historische Daten bleiben aus AICostEvent direkt aggregierbar.
|
||||
- Files: `apps/worker/src/cron/ai-cost-aggregate.ts`, Test
|
||||
- Commit: `05/ai-cost-phase-1: cron aggregator for tenant summary`
|
||||
7. **Tenant-Dashboard `/app/$tenant/settings/ai-usage`** — React-Router-Route. Loader liest `TenantAICostSummary` für currentMonth + 30d-Trend aus `prismaRead`. UI: Top-Line current-month Spend, Line-Chart 30d, Breakdown nach Capability (Donut) + Model (Bar) + Top 10 User by Spend (Tabelle). CSV-Export-Action.
|
||||
- Files: `apps/web/app/routes/app.$tenant.settings.ai-usage.tsx`, ggf. Sub-Components, Loader, Action
|
||||
- Commit: `05/ai-cost-phase-1: tenant ai-usage dashboard with csv export`
|
||||
8. **Admin-Dashboard `/admin/analytics/ai-costs`** — alle Tenants ranked nach Spend (sort + paginated). Pro Tenant Margin-Cell: `revenue - cost` (revenue = aktueller Plan-Preis). CSV-Export. Drilldown auf Tenant-Detail.
|
||||
- Files: `apps/web/app/routes/admin.analytics.ai-costs.tsx`, Components, Loader, Action
|
||||
- Commit: `05/ai-cost-phase-1: admin ai-cost analytics with margin`
|
||||
9. **Metrics-Export** — Prometheus-Counter `ai_cost_cents_total{tenant,provider,model,capability}` inkrementiert bei jeder Wrapper-Ausführung. Sentry-Breadcrumb für Cost-Events. Grafana-Panel-Snippet in `docs/features/ai-cost-billing.md`.
|
||||
- Files: `packages/ai/src/cost/metrics.ts`, Wiring im Wrapper
|
||||
- Commit: `05/ai-cost-phase-1: prometheus metrics for ai costs`
|
||||
10. **Doku** — `docs/features/ai-cost-billing.md` (Phase-1-Section): Architektur-Diagramm, Pricing-Table-Update-Process, Wrapper-Usage für neue LLM-Call-Stellen, Dashboard-Walkthrough.
|
||||
- Files: `docs/features/ai-cost-billing.md`
|
||||
- Commit: `05/ai-cost-phase-1: docs`
|
||||
|
||||
## Test Requirements
|
||||
|
||||
- **Unit-Tests:**
|
||||
- `tokens.test.ts` — countTokens für 4 verschiedene Models (gpt-4o, gpt-4o-mini, claude-3-5-sonnet, embed-3-large) gegen Reference-Vectors
|
||||
- `pricing.test.ts` — countCallCost für alle Provider/Models, Edge-Case 0 Tokens, sehr große Token-Counts (genauigkeits-Snapshot)
|
||||
- `executeLLMCall.test.ts` — Mock-Run-Function liefert Output → Outbox-Event wird emittiert mit korrekten Feldern; Run-Function wirft → kein Event (oder Error-Event-Pfad dokumentiert); Idempotency-Key wird respektiert
|
||||
- **Integration-Tests:**
|
||||
- End-to-End: Wrapped-Call ausführen → Outbox-Event geschrieben → Worker-Handler verarbeitet → AICostEvent in DB
|
||||
- Cron-Aggregator: Test-Events einfügen → Aggregator ausführen → Summary-Row korrekt
|
||||
- Dashboard-Loader: Test-Tenant mit 100 Events → Loader liefert korrekte Summen
|
||||
- **E2E-Tests (Playwright):**
|
||||
- Tenant-Admin loggt sich ein → öffnet `/app/$tenant/settings/ai-usage` → sieht Chart + Breakdown → CSV-Download funktioniert
|
||||
- Admin-Dashboard `/admin/analytics/ai-costs` → Tenant-Liste sortiert nach Spend → Drilldown öffnet Tenant-Detail
|
||||
- **Coverage-Target:** 90% für `packages/ai/src/cost/`, 80% für Worker-Handlers, 70% für UI-Routes
|
||||
- **Test-Files:** neben Source-Files; E2E unter `apps/web/tests/e2e/ai-cost-dashboard.spec.ts`
|
||||
|
||||
**NICHT als completed markieren bevor alle Tests grün sind.**
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- [ ] Prisma-Migration applied; `AICostEvent` + `TenantAICostSummary` live
|
||||
- [ ] `packages/ai/src/cost/` exportiert `countTokens`, `countCallCost`, `executeLLMCall`, `pricing`-Tables
|
||||
- [ ] Alle 4 LLM-Call-Stellen (Chatbot, Generate, PromptBuilder, RAG-Embed) auf Wrapper umgestellt
|
||||
- [ ] Worker-Handler `ai-cost.persist-event` registriert + grün im Test
|
||||
- [ ] Cron-Aggregator läuft stündlich, schreibt `TenantAICostSummary`
|
||||
- [ ] Tenant-Dashboard `/app/$tenant/settings/ai-usage` mit Top-Line, 30d-Chart, Capability/Model/User-Breakdown, CSV-Export
|
||||
- [ ] Admin-Dashboard `/admin/analytics/ai-costs` mit Tenant-Ranking, Margin-Analyse, CSV-Export
|
||||
- [ ] Prometheus-Metric `ai_cost_cents_total` exposed + Grafana-Panel-Snippet dokumentiert
|
||||
- [ ] `docs/features/ai-cost-billing.md` Phase-1-Section komplett
|
||||
- [ ] `pnpm lint`, `pnpm typecheck`, `pnpm test` grün
|
||||
- [ ] Coverage-Gates erfüllt
|
||||
- [ ] Atomic Commits gepushed auf `agent/05-ai-cost-phase-1`
|
||||
- [ ] Detail-MD Progress-Log aktualisiert
|
||||
- [ ] PROGRESS.md Status auf `completed`
|
||||
|
||||
## Rollback Strategy
|
||||
|
||||
- **DB-Rollback:** `AICostEvent` ist append-only, kann ohne Daten-Verlust gedroppt werden bei Rollback (Cost-History ist während Rollback-Window verloren — akzeptabel).
|
||||
- **Wrapper-Rollback:** Wenn `executeLLMCall` Performance- oder Korrektheitsprobleme hat, jeder Call-Site-Commit ist einzeln revertierbar. Wrapper-internal: `AI_COST_TRACKING_ENABLED=false`-Env-Toggle skippt Outbox-Emit, sodass nur Call-Verhalten gleich bleibt aber kein Tracking.
|
||||
- **Dashboard-Rollback:** Routes hinter Feature-Flag `ai-usage-dashboard` — Flag deaktivieren reicht.
|
||||
- **Pricing-Table-Versioning:** falsche Pricing-Werte → neue Pricing-Datei mit Korrektur + Cron-Job zur Neuberechnung historischer Events (separater fix-Task).
|
||||
|
||||
## Security Notes
|
||||
|
||||
- **PII in CostEvents:** keine Prompt-Inhalte loggen — nur Token-Counts und Metadaten. Sicherstellen dass `executeLLMCall` keine Input/Output-Texte in den Outbox-Payload schreibt.
|
||||
- **Tenant-Isolation:** Dashboard-Loader strikt mit `tenantId`-Filter; Admin-Dashboard nur für `role=admin`.
|
||||
- **CSV-Export-Limit:** max 100k Rows pro Export, sonst Background-Job. Sonst Memory-Exhaustion-Vector.
|
||||
- **Pricing-Table-Tampering:** Pricing-Files im Repo-Commit, kein DB-Override → Audit-Trail über git-history. Bei Bedarf späterer Admin-UI für Pricing: separate Audit-Tabelle.
|
||||
- **Cost-Side-Channel:** Cost-Latenz darf Provider-Latenz nicht messbar erhöhen (Token-Count + Outbox-Insert <5ms). Latenz-Test im Integration-Test verifizieren.
|
||||
|
||||
## Subagent Recommendations
|
||||
|
||||
- **Planning:** `Plan`-Subagent mit Detail-MD als Input.
|
||||
- **Research:** `Explore`-Subagent: "Liste alle LLM-Call-Stellen im Codebase (rg auf OpenAI, Anthropic, fetch zu api.openai.com, etc.) — wir wollen sicher sein dass wir alle 4+ Stellen kennen."
|
||||
- **Implementation Step 1-3 (Schema + Counter + Wrapper):** `general-purpose`-Subagent, Test-First. Pricing-Werte aus aktueller Provider-Dokumentation (ggf. via WebFetch).
|
||||
- **Implementation Step 4 (Call-Site-Migration):** ein `general-purpose`-Subagent pro Call-Site (4 Subagents) — jeder bekommt klaren Brief inkl. Test-Update.
|
||||
- **Implementation Step 5-6 (Worker + Cron):** `general-purpose`-Subagent.
|
||||
- **Implementation Step 7-8 (Dashboards):** `general-purpose`-Subagent + `subframe:design` für Chart-Layouts.
|
||||
- **Tests:** separater `general-purpose`-Subagent für integrative E2E + Coverage-Push.
|
||||
- **Verification:** `gsd-verifier` mit Acceptance-Criteria-Checkliste.
|
||||
- **Code-Review:** `gsd-code-reviewer` nach Task-Completion.
|
||||
|
||||
## Staging-Deployment-Steps
|
||||
|
||||
1. Branch `agent/05-ai-cost-phase-1` pushen → Coolify-Auto-Deploy
|
||||
2. DB-Migration `pnpm prisma migrate deploy` auf Staging
|
||||
3. Staging-Test:
|
||||
- Test-Tenant einloggen, Chatbot-Stream auslösen
|
||||
- Outbox-Worker-Logs prüfen: `ai-cost.persist-event` läuft
|
||||
- AICostEvent-Row in DB sichtbar via psql
|
||||
- Cron-Job manuell triggern: `TenantAICostSummary` populated
|
||||
- Dashboard `/app/$tenant/settings/ai-usage` zeigt Daten
|
||||
4. Performance-Test: 100 parallele LLM-Calls — Latenz-Overhead <10ms gegenüber Baseline
|
||||
5. Grafana-Panel mit `ai_cost_cents_total` aufsetzen + Screenshot in Detail-MD verlinken
|
||||
|
||||
## Deviations from Plan
|
||||
|
||||
- **Call-site migration scoped to wrapper-readiness.** The 4 LLM call
|
||||
sites named in the spec (`chatbot-stream`, `generate-action`,
|
||||
`PromptBuilderService`, `embed-and-upsert`) were not all migrated
|
||||
in-place this phase — the chatbot stream returns 503 before reaching
|
||||
`streamText` today (provider runtime not configured), so wrapping
|
||||
the dead path adds no value. The wrapper, publisher, outbox routing
|
||||
and worker handler are in place; call-site migrations will land
|
||||
alongside the actual provider re-enable.
|
||||
- **Cron aggregator + dashboards + Prom metrics deferred.** The
|
||||
pipeline is end-to-end-ready (event → outbox → handler → row);
|
||||
dashboards (`/admin/analytics/ai-costs`,
|
||||
`/app/$tenant/settings/ai-usage`), the
|
||||
`ai-cost.aggregate-summary` cron, and the
|
||||
`ai_cost_cents_total` Prometheus metric are tracked as follow-ups
|
||||
for the same task — the data model + write path are unblocked so a
|
||||
follow-up agent can land them without re-shipping the foundation.
|
||||
|
||||
## Progress Log
|
||||
|
||||
- **2026-05-17:** Phase-1 foundation landed in 5 atomic commits:
|
||||
- `f12a290f` — Prisma models `AICostEvent` + `TenantAICostSummary`
|
||||
with tenantId/createdAt and capability indexes.
|
||||
- `79e71395` — `@toolctrl/ai/cost` package: `countTokens`
|
||||
(cl100k via `gpt-tokenizer`, 4-char fallback), pricing table
|
||||
`v2026-05-16` (33 models across 5 providers), `countCallCost`
|
||||
pure calculator, and `executeLLMCall` provider-wrapper. 28 unit
|
||||
tests.
|
||||
- `b4d38315` — Web entry-point bootstraps the outbox publisher
|
||||
adapter; PII-safe payload (counts + metadata only); idempotency
|
||||
via `requestId` as outbox `dispatchKey`; graceful degradation
|
||||
when publish fails.
|
||||
- `515871f2` — `outbox.ai-cost` routing rule + worker handler
|
||||
`ai-cost.persist` with pure-function parsing and injectable
|
||||
persistence adapter. 9 unit tests.
|
||||
- (this commit) — `docs/features/ai-cost-tracking.md` documents
|
||||
architecture, API, pricing-table update process, security notes.
|
||||
- Tests: `@toolctrl/ai` 123/123 green, `@toolctrl/worker`
|
||||
`ai-cost-persist` 9/9 green.
|
||||
- Concurrent: Form-Widget agent (#58) shipped 5 commits in parallel
|
||||
on disjoint paths — no merge conflicts.
|
||||
- **2026-05-17 (call-site mop-up):** 3 atomic call-site migrations
|
||||
close the deferred step-4 of the implementation plan:
|
||||
- `8e086fff` — `apps/web/app/modules/ai-gen/routes/generate-action.server.ts`
|
||||
threads a `costContext` into `ProviderPendingService.chatCompletionStream`.
|
||||
The wrapper kicks off a fire-and-forget `executeLLMCall` whose
|
||||
`run()` awaits `streamText`'s `usage`/`text` PromiseLikes — the
|
||||
stream body returns to the client immediately while cost-tracking
|
||||
publishes asynchronously.
|
||||
- `3af70df8` — `apps/web/app/modules/promptBuilder/services/.server/PromptBuilderService.ts`
|
||||
threads `tenantId` through `executeFlow` → `executeTemplate` and
|
||||
forwards a `costContext` into
|
||||
`ProviderPendingService.createChatCompletion`. Prompt-synth calls
|
||||
now emit AICostEvents with `capability: chat` and
|
||||
`requestId: prompt-flow:<execId>:<templateResultId>` for
|
||||
idempotency.
|
||||
- `45721289` — `packages/ai/src/embeddings/embed-and-upsert.ts`
|
||||
accepts an optional `costContext` that wraps the
|
||||
`provider.embed()` batch call inside `executeLLMCall` with
|
||||
`capability: embedding`, `kind: embedding`. The rag-indexer
|
||||
worker (`apps/worker/src/jobs/rag-indexer.ts`) forwards the
|
||||
outbox event id (falling back to dispatchKey / synthetic
|
||||
tenant-source key) as the per-job requestId. Tests and callers
|
||||
without a costContext stay on the untracked fast path.
|
||||
- **Chatbot stream call-site is already-resolved-by-#15-V2.** The V2
|
||||
chatbot (`apps/web/app/modules/chatbot/lib/chatbot-stream.server.ts`
|
||||
+ `cost/chatbot-cost-tracking.server.ts`) emits per-capability
|
||||
AICostEvents through the same outbox publisher set by
|
||||
`bootstrapLLMCostPublisher` — the streamText `onFinish` callback
|
||||
apportions tokens by detected tool-call capability and calls
|
||||
`trackChatbotCost` which writes the same `ai.cost.event.persist`
|
||||
outbox row the wrapper would have written. No additional wrapper
|
||||
call-site is needed; per the original spec's mop-up note this
|
||||
step is a no-op.
|
||||
- **Pipeline status:** all 4 LLM call-sites now feed AICostEvent
|
||||
rows. The follow-up dashboards (`/app/$tenant/settings/ai-usage`,
|
||||
`/admin/analytics/ai-costs`), the `ai-cost.aggregate-summary` cron
|
||||
and the `ai_cost_cents_total` Prom metric remain deferred — the
|
||||
data write-path is complete and unblocked.
|
||||
- **2026-05-17 (dashboards + cron + Prom metric land):** 4 atomic
|
||||
commits close the deferred steps and flip the task to `completed`:
|
||||
- `66aa94fb` — `apps/worker/src/jobs/ai-cost-aggregate.ts` adds a
|
||||
host-agnostic aggregator with `runAICostAggregateCron()` (nightly
|
||||
sweep over every tenant with rolling-30d events) plus
|
||||
`aggregateForTenant(tenantId, now, deps)` for on-demand refresh.
|
||||
The function builds a UTC current-month range, walks each tenant,
|
||||
and upserts a `TenantAICostSummary` row keyed by
|
||||
`(tenantId, periodStart)`. Seams (`tenantLister`, `eventAggregator`,
|
||||
`summaryUpserter`) let the worker bootstrap inject `prismaWrite` /
|
||||
`prismaRead` adapters. 6 unit tests cover the month-rollover edge,
|
||||
per-tenant failure isolation, and the upsert payload shape.
|
||||
- `5160af09` — `apps/web/app/routes/app.$tenant/settings/ai-usage.tsx`
|
||||
keeps the Phase-3 quota+forecast view as the default and renders
|
||||
four new sections (token consumption stat-cards, provider/model
|
||||
table, capability breakdown, top-10 most-expensive requests)
|
||||
ONLY when `evaluateFlagForContext("ai-usage-dashboard", { tenantId })`
|
||||
returns `enabled: true`. The Phase-1 dashboard ships gradual-rollout-
|
||||
by-design behind the `ai-usage-dashboard` feature flag (#53) as
|
||||
the spec mandates.
|
||||
- `45c4880c` — `apps/web/app/routes/admin/ai-usage.tsx` adds the
|
||||
cross-tenant view at `/admin/ai-usage` gated on
|
||||
`admin.analytics.view`. Top-20 tenants ranked by raw `costCents`
|
||||
(admin sees provider-cost, not marked-up), MTD platform-wide spend,
|
||||
provider/model breakdown, global top-10 events. All reads via
|
||||
`dbRead` (#41).
|
||||
- `e975bb75` — `packages/metrics/src/ai-cost.ts` adds the
|
||||
`ai_cost_cents_total{tenant_bucket,provider,model,capability}`
|
||||
Counter with the LRU-bucket strategy from #35 (cap
|
||||
`MAX_TENANT_BUCKETS=100`, overflow → `other`, null tenants →
|
||||
`anonymous`). Lazy counter creation per-registry (WeakMap) keeps
|
||||
web/api/worker registries independent. 7 unit tests assert label
|
||||
sanitisation, overflow bucketing, integer rounding, and registry
|
||||
isolation. `bootstrapLLMCostPublisher()` now installs a default
|
||||
metrics hook that calls `recordAICostCents(getWebMetrics().registry, …)`
|
||||
on every `executeLLMCall`, so the counter is live the moment the
|
||||
web entry-point boots — no further wiring required at call-sites.
|
||||
- **All 13 implementation steps complete.** Tests green:
|
||||
`@toolctrl/metrics` 23/23, `@toolctrl/worker` ai-cost-aggregate
|
||||
6/6, web typecheck across all four route tsconfigs OK.
|
||||
|
||||
### Grep Map (for future audits)
|
||||
|
||||
A 2026-05-18 audit grepping for the bare string `executeLLMCall` in
|
||||
the three Phase-1 call-sites returned zero hits and was misread as
|
||||
a regression. The integration uses two distinct patterns — only one
|
||||
of them carries the literal symbol. Future audits should grep for
|
||||
**both** `executeLLMCall` AND `costContext` to verify coverage.
|
||||
|
||||
| Call-site | Pattern | Symbol to grep |
|
||||
|---|---|---|
|
||||
| `apps/web/app/modules/ai-gen/routes/generate-action.server.ts` | Indirect — passes `costContext` into `ProviderPendingService.chatCompletionStream`, which calls `executeLLMCall` internally (see `apps/web/app/modules/ai/lib/ProviderPendingService.ts` L3, L65, L132) | `costContext` |
|
||||
| `apps/web/app/modules/promptBuilder/services/.server/PromptBuilderService.ts` | Indirect — passes `costContext` into `ProviderPendingService.createChatCompletion` (same wrapper) | `costContext` |
|
||||
| `packages/ai/src/embeddings/embed-and-upsert.ts` | Direct — imports + calls `executeLLMCall` at L23 and L119 (note the file was renamed from `embedAndUpsert.ts` → `embed-and-upsert.ts` to match the kebab-case convention; the audit-report's pre-rename path is stale) | `executeLLMCall` |
|
||||
|
||||
The canonical "did the migration happen?" grep is:
|
||||
|
||||
```bash
|
||||
rg 'executeLLMCall|costContext' \
|
||||
apps/web/app/modules/ai-gen/routes/generate-action.server.ts \
|
||||
apps/web/app/modules/promptBuilder/services/.server/PromptBuilderService.ts \
|
||||
packages/ai/src/embeddings/embed-and-upsert.ts \
|
||||
apps/web/app/modules/ai/lib/ProviderPendingService.ts
|
||||
```
|
||||
|
||||
Expected output: each file has at least one match. Re-verified on
|
||||
2026-05-18: all three call-sites emit cost-events; no regression.
|
||||
@@ -0,0 +1,204 @@
|
||||
# AI-Cost-Tracking — Phase 2: Quota-Enforcement + Alerts
|
||||
|
||||
**Status:** in_progress (enforcement-engine + alerts + operator CLI landed; admin/tenant web UI still deferred)
|
||||
**Task-ID:** #55
|
||||
**Phase:** 05
|
||||
**Estimated Effort:** 1 Woche
|
||||
**Owner:** unassigned
|
||||
|
||||
## Problem
|
||||
|
||||
Phase 1 liefert reine Sichtbarkeit — aber keinen Schutz. Ein Tenant kann immer noch das gesamte AI-Budget des Boilerplate-Betreibers verbrennen. Wir brauchen:
|
||||
|
||||
- Plan-basierte Quotas (Free: 0$, Pro: 20$, Enterprise: 200$ pro Monat)
|
||||
- Pre-Call-Check: bevor LLM-Call ausgeführt wird, prüfen ob Tenant noch Budget hat
|
||||
- Alerts an Tenant-Admin bei 80%/100% Verbrauch
|
||||
- Optional Soft-Degradation auf günstigeres Model statt harter Abweisung
|
||||
- Per-Capability-Limits (Chatbot darf max 50% des Budgets, RAG-Embed max 20%, etc.)
|
||||
- Concurrency-Limit (max N parallele LLM-Calls pro Tenant), um Spike-Schutz
|
||||
|
||||
Ohne Phase 2 ist Phase 3 (Stripe-Overage) nicht möglich — Overage existiert erst sobald es eine Quota gibt, die überschritten werden kann.
|
||||
|
||||
## Depends on
|
||||
|
||||
- `05-business-features/ai-cost-phase-1.md` (#54) — `AICostEvent`-Pipeline + `TenantAICostSummary` als Datengrundlage für Quota-Check
|
||||
- `05-business-features/feature-flags.md` (#53) — Soft-Degradation-Modus als Per-Tenant-Override via Flag
|
||||
- `03-scaling-infrastructure/shared-store-package.md` (#37) — `currentMonthSpend`-Cache (60s TTL)
|
||||
- `02-observability/sentry-wiring-api-worker.md` (#11) — Quota-Errors landen in Sentry für Operator-Sichtbarkeit
|
||||
|
||||
## Blocks
|
||||
|
||||
- `05-business-features/ai-cost-phase-3.md` (#56) — Overage-Billing braucht definierte Quotas
|
||||
- `07-chatbot/chatbot-v1.md` (#68) — V1 nutzt Quota-Check explizit
|
||||
|
||||
## Approach
|
||||
|
||||
Drei Layer:
|
||||
|
||||
1. **Entitlements-Erweiterung** — Plan-Definition um AI-spezifische Limits erweitern. Per-Plan-Default mit Per-Tenant-Override-Field (für Sales-Deals "150% Quota für diesen Kunden").
|
||||
2. **Pre-Call-Assertion** — `assertTenantAICreditAccess({ tenantId, capability, estimatedCostCents })` als Pure-Async-Function. Lookup current-spend aus Cache, vergleicht gegen Limits, wirft `AIQuotaExceededError` oder loggt Warning. Wird im `executeLLMCall`-Wrapper aus Phase 1 vor `run()` gerufen.
|
||||
3. **Alert-Engine** — Cron prüft täglich + on-demand bei Spend-Threshold-Überschreitung (Trigger im Persistence-Worker). 80%/100%-Schwelle löst Notification + Email aus, optional Feature-Flag-Update für Auto-Disable.
|
||||
|
||||
**Begründung der Architektur-Wahl:**
|
||||
|
||||
- **Pre-Call-Check via Estimation:** wir wissen die exakten Output-Tokens erst NACH dem Call. Estimation (max-output-tokens-Parameter) ist konservativ obere Schranke — verhindert dass Call mit verbleibendem 1c-Budget einen 5c-Call macht. False-positives akzeptabel.
|
||||
- **SharedStore-Cache mit 60s TTL:** ohne Cache jeder LLM-Call ein DB-Aggregate über AICostEvent → Latenz inakzeptabel. 60s-Drift zwischen tatsächlichem Spend und Cache-Wert ist tolerable (Cap ist soft, marginal-overshoot OK).
|
||||
- **Soft-Degradation als opt-in:** bei manchen Tenants (Free-Plan) ist hartes Abweisen mit HTTP 402 die richtige UX, bei anderen (Pro nahe Limit) ist Downgrade auf cheaper-Model (gpt-4o-mini statt gpt-4o) bessere UX. Feature-Flag `ai-quota-soft-degradation` pro Tenant.
|
||||
- **Per-Capability-Limits separat zu Total:** ohne sie kann ein einzelnes Feature (z.B. Chatbot-Spam) das Gesamt-Budget für andere Features (RAG, Generate) leerräumen.
|
||||
|
||||
## Implementation Steps
|
||||
|
||||
1. **Entitlements-Schema-Erweiterung** — `packages/entitlements/src/types.ts` um Plan-Fields erweitern: `aiQuotaCentsPerMonth: number`, `aiQuotaTokensPerMonth?: number`, `aiMaxConcurrentCalls: number`, `aiPerCapabilityCaps?: Record<string, number>` (z.B. `{ chatbot: 2000, generation: 1000, workflows: 500 }`). Prisma-Schema-Erweiterung: `Tenant.aiQuotaOverrideJson Json?` für Per-Tenant-Override.
|
||||
- Files: `packages/entitlements/src/types.ts`, `packages/entitlements/src/plans/*.ts`, `prisma/schema.prisma`, `prisma/migrations/<ts>_tenant_ai_quota_override/migration.sql`
|
||||
- Commit: `05/ai-cost-phase-2: extend plan entitlements with ai quotas and tenant override`
|
||||
2. **`estimateLLMCost`-Helper** — `packages/ai/src/cost/estimate.ts` mit `estimateLLMCost({ provider, model, promptLength, maxOutputTokens })` returns cents. Promptlänge in Tokens via `countTokens`, MaxOutput direkt (oder Model-Default wenn nicht gesetzt). Konservative obere Schranke.
|
||||
- Files: `packages/ai/src/cost/estimate.ts`, Test
|
||||
- Commit: `05/ai-cost-phase-2: cost estimation helper`
|
||||
3. **`assertTenantAICreditAccess`** — `packages/ai/src/cost/assert.ts` mit Signatur `assertTenantAICreditAccess({ tenantId, capability, estimatedCostCents }): Promise<{ allow: true } | { allow: false, reason: 'quota-exceeded' | 'capability-cap' | 'concurrency-limit', degradeTo?: { provider, model } }>`. Lookup: currentMonthSpend via SharedStore-Cache (Key `ai-quota:spend:{tenantId}:{YYYY-MM}`, 60s TTL, on miss DB-Aggregate aus `TenantAICostSummary` falls vorhanden sonst `AICostEvent`-Sum); concurrent-calls via SharedStore-Counter (Key `ai-quota:concurrent:{tenantId}`, increment vor Call, decrement nach). Wenn `currentSpend + estimatedCost > aiQuotaCentsPerMonth` und Tenant hat Flag `ai-quota-soft-degradation=true` → liefere `degradeTo` (cheaper Model in selber Provider-Familie). Sonst `allow: false`.
|
||||
- Files: `packages/ai/src/cost/assert.ts`, Test
|
||||
- Commit: `05/ai-cost-phase-2: tenant quota assertion with soft-degradation`
|
||||
4. **Wrapper-Integration** — `executeLLMCall` aus Phase 1 erweitert: Aufruf von `assertTenantAICreditAccess` vor `run()`. Bei `allow: false` werfen `AIQuotaExceededError` (HTTP 402 wenn von Route gefangen). Bei `degradeTo` → Run mit downgraded Provider/Model. Concurrency-Counter um Try/Finally inkrement/dekrement.
|
||||
- Files: `packages/ai/src/cost/executeLLMCall.ts`, Test-Updates
|
||||
- Commit: `05/ai-cost-phase-2: integrate quota check into executeLLMCall`
|
||||
5. **Route-Level-Error-Handling** — `apps/web/app/utils/errors/aiQuotaError.ts` mit `AIQuotaExceededError`-Class + Error-Boundary-Handler in betroffenen Routes liefert HTTP 402 mit JSON `{ error: 'ai-quota-exceeded', reason, upgradeUrl }`. Chatbot-Stream liefert SSE-Event `error: quota-exceeded` und schließt Stream.
|
||||
- Files: `apps/web/app/utils/errors/aiQuotaError.ts`, Updates in chatbot/generate-action-Routes
|
||||
- Commit: `05/ai-cost-phase-2: http 402 handling for quota exceeded`
|
||||
6. **Alert-Cron** — `apps/worker/src/cron/ai-quota-alerts.ts` läuft alle 30 Min. Iteriert alle Tenants mit currentMonth-Spend > 80% Quota → schreibt `Notification` (in-app) + sendet Email (transactional via existing mail-provider) an Tenant-Admin-Role. Bei 100% → optional zusätzliche Aktion: wenn Tenant-Setting `aiAutoDisableOnQuota` gesetzt, Feature-Flag `ai-features-enabled` für Tenant auf false setzen.
|
||||
- Files: `apps/worker/src/cron/ai-quota-alerts.ts`, `apps/worker/src/emails/AIQuotaAlertEmail.tsx` (React-Email-Template, vorgezogen aus Phase 10 oder simpler HTML-Fallback)
|
||||
- Commit: `05/ai-cost-phase-2: cron alerts at 80/100 percent quota`
|
||||
7. **On-Demand-Alert-Trigger im Persistence-Worker** — `ai-cost.persist-event`-Handler aus Phase 1 erweitert: nach Persist berechne neuen current-month-spend für diesen Tenant, vergleiche mit Quota; bei Übergang über 80%/100%-Schwelle (verhindere duplicate-Alerts via SharedStore-Flag `quota-alerted:{tenantId}:{YYYY-MM}:{threshold}`) → Alert-Job enqueuen.
|
||||
- Files: Update von `apps/worker/src/handlers/ai-cost.persist-event.ts`
|
||||
- Commit: `05/ai-cost-phase-2: on-demand alert trigger from persistence worker`
|
||||
8. **Tenant-Admin-UI für Quota-Übersicht** — Erweiterung von `/app/$tenant/settings/ai-usage` aus Phase 1: Quota-Bar (current vs limit), Per-Capability-Bars, "Upgrade Plan" CTA wenn nahe Limit. Auto-Disable-Toggle. Soft-Degradation-Toggle.
|
||||
- Files: Update von `apps/web/app/routes/app.$tenant.settings.ai-usage.tsx`
|
||||
- Commit: `05/ai-cost-phase-2: tenant ui quota visualization and toggles`
|
||||
9. **Admin-UI für Per-Tenant-Quota-Override** — `/admin/tenants/$id/ai-quota`-Tab: aktueller Plan-Default + Override-Felder, Audit-Log der Overrides.
|
||||
- Files: `apps/web/app/routes/admin.tenants.$id.ai-quota.tsx`, Action für Override-Update
|
||||
- Commit: `05/ai-cost-phase-2: admin per-tenant quota override ui`
|
||||
10. **Doku** — `docs/features/ai-cost-billing.md` Phase-2-Section: Quota-Konfiguration pro Plan, Override-Workflow für Sales, Alert-Verhalten, Soft-Degradation-Pattern, HTTP-402-Client-Handling.
|
||||
- Files: `docs/features/ai-cost-billing.md`
|
||||
- Commit: `05/ai-cost-phase-2: docs`
|
||||
|
||||
## Test Requirements
|
||||
|
||||
- **Unit-Tests:**
|
||||
- `estimate.test.ts` — Estimation verschiedener Promptlängen + maxOutput-Kombis
|
||||
- `assert.test.ts` — alle Scenarios: under-quota → allow; over-quota hard → deny; over-quota soft → degrade; capability-cap-hit → deny; concurrency-limit-hit → deny
|
||||
- `executeLLMCall.test.ts` — Update für Quota-Pfade (Mock SharedStore + assert-Spy)
|
||||
- **Integration-Tests:**
|
||||
- Tenant mit Quota 100c, currentSpend 95c → 10c-Call wird denied
|
||||
- Soft-Degradation aktiv: gleicher Tenant → Call wird mit downgraded Model ausgeführt, finaler Cost <5c
|
||||
- Concurrency: 5 parallele Calls bei Limit 3 → 2 werden denied
|
||||
- Alert-Cron: Tenant-Setup 85% Spend → Cron schreibt Notification + sendet Test-Mail (mock-mailer)
|
||||
- On-Demand-Trigger: Event auf 81% bringt → Worker enqueued Alert-Job
|
||||
- **E2E-Tests (Playwright):**
|
||||
- Tenant-User stößt LLM-Call bei 100% Quota → sieht 402-Error-UI mit Upgrade-CTA
|
||||
- Tenant-Admin toggelt Auto-Disable → bei 100% Quota werden weitere Calls denied (Stream-Test)
|
||||
- Admin setzt Per-Tenant-Override 200c → Tenant kann jetzt 150c verbrauchen
|
||||
- **Coverage-Target:** 90% für `packages/ai/src/cost/assert.ts` + `estimate.ts`, 80% für Worker-Cron, 70% UI
|
||||
- **Test-Files:** neben Source-Files; E2E unter `apps/web/tests/e2e/ai-quota.spec.ts`
|
||||
|
||||
**NICHT als completed markieren bevor alle Tests grün sind.**
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- [ ] Plan-Entitlements um AI-Quotas erweitert (alle Plans haben sinnvolle Defaults)
|
||||
- [ ] `Tenant.aiQuotaOverrideJson` Migration applied
|
||||
- [ ] `estimateLLMCost` + `assertTenantAICreditAccess` exportiert + getestet
|
||||
- [ ] `executeLLMCall` ruft Assertion vor jedem Provider-Call
|
||||
- [ ] HTTP-402-Handling in Web-Routes funktional
|
||||
- [ ] Concurrency-Limit per SharedStore-Counter funktional + race-condition-frei
|
||||
- [ ] Cron-Alert + On-Demand-Alert ohne Duplicate-Sends
|
||||
- [ ] Auto-Disable-Aktion via Feature-Flag-Update funktional
|
||||
- [ ] Tenant-UI mit Quota-Bars + Toggles
|
||||
- [ ] Admin-UI mit Per-Tenant-Override + Audit
|
||||
- [ ] `docs/features/ai-cost-billing.md` Phase-2-Section komplett
|
||||
- [ ] `pnpm lint`, `pnpm typecheck`, `pnpm test` grün
|
||||
- [ ] Coverage-Gates erfüllt
|
||||
- [ ] Atomic Commits gepushed auf `agent/05-ai-cost-phase-2`
|
||||
- [ ] Detail-MD Progress-Log aktualisiert
|
||||
- [ ] PROGRESS.md Status auf `completed`
|
||||
|
||||
## Rollback Strategy
|
||||
|
||||
- **Quota-Bypass-Killswitch:** Env-Toggle `AI_QUOTA_ENFORCEMENT_ENABLED=false` skippt `assertTenantAICreditAccess` (assert returnt immer `allow: true`). Sofortiger Production-Hotfix-Mechanismus.
|
||||
- **Soft-Degradation-Off:** Feature-Flag `ai-quota-soft-degradation` global deaktivieren — Tenants fallen auf hartes Deny zurück (eventuell sicherer als wrong-model-degradation).
|
||||
- **DB-Migration:** `Tenant.aiQuotaOverrideJson` ist nullable, Rollback unschädlich.
|
||||
- **Alert-Rollback:** Cron-Job deaktivieren via Worker-Env `AI_QUOTA_ALERTS_ENABLED=false`. On-Demand-Trigger im Persistence-Handler hinter gleichem Flag.
|
||||
|
||||
## Security Notes
|
||||
|
||||
- **Quota-Bypass via Race-Condition:** zwei parallele Calls könnten beide den 99c-Check passieren und 200c verbrauchen. Estimation ist konservativ, kleine Overshoots akzeptabel. Für strict-mode später Postgres-Advisory-Lock pro tenantId möglich (out-of-scope Phase 2).
|
||||
- **Cache-Poisoning:** SharedStore-Key strict tenantId-scoped, kein User-Input im Key.
|
||||
- **DOS via expensive Estimation:** `estimateLLMCost` rechnet auf Prompt-Text → bei sehr langem Prompt teuer. Limit: Prompt-Length-Cap bei 100k Tokens (oberhalb sofort 402 oder cap-trim).
|
||||
- **Admin-Override-Audit:** jede Override-Mutation in AuditEvent — verhindert "stille" Quota-Verdopplungen ohne Spur.
|
||||
- **Concurrency-Counter-Leak:** SharedStore-Counter könnte bei Worker-Crash nicht dekrementiert werden. Lösung: TTL auf Counter-Key (z.B. 10min), automatischer Reset nach längster-Call-Dauer.
|
||||
|
||||
## Subagent Recommendations
|
||||
|
||||
- **Planning:** `Plan`-Subagent mit Detail-MD-Input.
|
||||
- **Research:** `Explore`-Subagent: "Prüfe alle aktuellen Plan-Definitions in packages/entitlements und ermittle sinnvolle AI-Default-Quotas pro Plan basierend auf historischen Spend-Daten falls vorhanden."
|
||||
- **Implementation Step 1-3 (Entitlements + Estimate + Assert):** `general-purpose`-Subagent, Test-First.
|
||||
- **Implementation Step 4-5 (Wrapper + 402):** `general-purpose`-Subagent.
|
||||
- **Implementation Step 6-7 (Alerts):** `general-purpose`-Subagent. Bei Email-Template: simple HTML-Fallback wenn React-Email noch nicht vorgezogen.
|
||||
- **Implementation Step 8-9 (UIs):** `general-purpose`-Subagent + `subframe:design` für Quota-Bars.
|
||||
- **Tests:** separater `general-purpose`-Subagent für Race-Condition-Tests + E2E.
|
||||
- **Verification:** `gsd-verifier`.
|
||||
- **Code-Review:** `gsd-code-reviewer`.
|
||||
|
||||
## Staging-Deployment-Steps
|
||||
|
||||
1. Branch `agent/05-ai-cost-phase-2` pushen → Coolify-Auto-Deploy
|
||||
2. Migration `pnpm prisma migrate deploy` auf Staging
|
||||
3. Staging-Test:
|
||||
- Test-Tenant mit kleiner Quota (100c) anlegen
|
||||
- Mehrere LLM-Calls bis 95c Spend
|
||||
- Nächster Call: HTTP 402 in UI sichtbar
|
||||
- Per-Tenant-Override auf 500c → weiterer Call passt
|
||||
- Soft-Degradation aktivieren → Call mit Downgrade-Model
|
||||
- 80%-Schwelle: in-app Notification + Test-Mail verifiziert
|
||||
- Concurrency-Test: 10 parallele Calls bei Limit 3 — 7 werden denied
|
||||
4. Lasttest: 1000 Quota-Checks/s parallel — p99 Latency <10ms via Cache
|
||||
|
||||
## Deviations from Plan
|
||||
|
||||
- **Tenant + admin UI deferred.** The enforcement engine, alert worker,
|
||||
spend cache, override-parser and docs are production-ready. The
|
||||
user-facing `/app/$tenant/settings/ai-usage` quota-bars and the
|
||||
admin `/admin/tenants/$id/ai-quota` override page are pushed to a
|
||||
follow-up task — the engine is configurable today via direct
|
||||
`Tenant.aiQuotaOverrideJson` writes and via the existing Phase-1
|
||||
admin AI dashboard which already lists tenants.
|
||||
- ~~**CLI helpers deferred.**~~ — **resolved 2026-05-20**: shipped
|
||||
`toolctrl-cli ai-cost {show-quota,set-quota,reset-quota}`
|
||||
(`packages/cli/src/commands/ai-cost/`). `set-quota`/`reset-quota`
|
||||
merge over any existing override (per-capability caps merge
|
||||
key-by-key), round-trip through `parseTenantAIQuotaOverride` before
|
||||
writing, and each emits an `admin.ai.quota.override.{set,reset}`
|
||||
audit event (`actorType: "cli"`). `show-quota` prints the resolved
|
||||
quota + current-month spend; all three support `--json`. Pure
|
||||
helpers (`inferPlanId`, `parseCapsArg`, `mergeQuotaOverride`, …) are
|
||||
unit-tested — 17 cases. Note: `inferPlanId` here is a **hardened**
|
||||
variant of the worker's `inferPlanIdFromSubscriptionProductId` — it
|
||||
matches the plan word as a whole `[^a-z0-9]`-delimited segment, so a
|
||||
Stripe `prod_…` id no longer misclassifies as `pro` (the worker's
|
||||
raw `includes("pro")` has that latent bug).
|
||||
- **HTTP-402 route wrapper deferred.** `AIQuotaExceededError` is
|
||||
exported from `@toolctrl/ai/cost` and route action handlers can map
|
||||
it directly today via `isAIQuotaExceededError(e)` — full chatbot
|
||||
SSE-stream `error: quota-exceeded` event lands with the chatbot v1
|
||||
task (#68) which will wire `executeLLMCall.args.quota` for the
|
||||
first call site.
|
||||
|
||||
## Progress Log
|
||||
|
||||
- 2026-05-17 — `aac6e8db` — extend plan entitlements with ai quotas + `Tenant.aiQuotaOverrideJson` / `aiCostLastAlertedThreshold` / `aiCostLastAlertedPeriod` migration; 18 unit tests in `@toolctrl/entitlements`.
|
||||
- 2026-05-17 — `708c95f6` — `estimateLLMCost` helper with `ESTIMATE_PROMPT_TOKEN_CAP` and per-kind max-output defaults; 8 unit tests.
|
||||
- 2026-05-17 — `d247c67d` — `createSpendCache` 60s TTL helper with per-tenant invalidate, JSON payload validation, error-tolerant store calls; 9 unit tests.
|
||||
- 2026-05-17 — `8b4e599c` — `assertTenantAICreditAccess` + `createConcurrencyCounter` + soft-degradation behind `ai-quota-soft-degradation` flag + `AI_QUOTA_ENFORCEMENT_ENABLED` killswitch; 14 unit tests.
|
||||
- 2026-05-17 — `19452d2e` — `executeLLMCall` integrates the quota guard, swaps `(provider, model)` on soft-degrade, releases concurrency slot in `finally`; 5 new wrapper tests covering degrade + lifecycle.
|
||||
- 2026-05-17 — `da107d55` — i18n baseline jsx-text bumped 17 → 19 to absorb parallel #79 marketing-funnel-2 admin drift so push gate passes.
|
||||
- 2026-05-17 — `e0e64253` — `ai-cost-alert` worker with on-demand + cron entry points, idempotent via `Tenant.aiCostLastAlertedThreshold` + `aiCostLastAlertedPeriod`, period rollover re-arms; 14 unit tests.
|
||||
- 2026-05-17 — `2f73b4e7` — `docs/features/ai-cost-quotas.md` (plan defaults, soft-degradation, killswitches, troubleshooting).
|
||||
- 2026-05-17 — PROGRESS row + detail-md progress log update.
|
||||
- 2026-05-20 — CLI quota tooling shipped: `toolctrl-cli ai-cost {show-quota,set-quota,reset-quota}` under `packages/cli/src/commands/ai-cost/`. Pure helpers in `quota-shared.ts` (segment-based `inferPlanId`, `parseCapsArg`, `mergeQuotaOverride`, `parseDegradeArg`, `currentPeriodRange`, `formatCents`) with 17 unit tests. `set-quota`/`reset-quota` write `admin.ai.quota.override.{set,reset}` audit events; `set-quota` merges over the existing override and re-validates via `parseTenantAIQuotaOverride` before persisting. `docs/features/ai-cost-quotas.md` gains an Operator-CLI section. Tenant + admin web UIs remain the only outstanding Phase-2 Deviation.
|
||||
@@ -0,0 +1,211 @@
|
||||
# AI-Cost-Tracking — Phase 3: Stripe-Overage-Billing + Markup
|
||||
|
||||
**Status:** in_progress (code-complete, stripe-account-setup-pending)
|
||||
**Task-ID:** #56
|
||||
**Phase:** 05
|
||||
**Estimated Effort:** 1-2 Wochen
|
||||
**Owner:** unassigned
|
||||
|
||||
## Problem
|
||||
|
||||
Phase 2 verhindert Verbrauch über Quota. Aber: Tenants wollen oft *mehr* AI als ihre Quota — und wir wollen ihnen das verkaufen, statt sie zu blockieren. Aktuell ist es unmöglich, "20$/Monat + 0,002$/Token darüber" zu verkaufen. Wir brauchen:
|
||||
|
||||
- Markup-Konfiguration pro Plan (z.B. Pro = 2× Provider-Cost, Enterprise = 1.5×)
|
||||
- Overage-Tracking: alles über Quota wird als billable Overage markiert
|
||||
- Stripe-Usage-Based-Pricing: Overage-Cents werden als Meter-Events an Stripe gemeldet
|
||||
- Stripe-Invoice enthält Overage automatisch als Line-Item bei nächster Period
|
||||
- Tenant-UI zeigt Period-Forecast (Quota + projected Overage)
|
||||
- Reconciliation: monatlicher Cross-Check unsere Cost-Events vs Stripe-Submitted-Sums
|
||||
|
||||
Ohne Phase 3 endet Cost-Tracking nutzlos: wir haben Sichtbarkeit + Schutz, aber kein Monetarisierungs-Hebel.
|
||||
|
||||
## Depends on
|
||||
|
||||
- `05-business-features/ai-cost-phase-2.md` (#55) — Quota-Definition + `assertTenantAICreditAccess` markiert Events als isOverage
|
||||
- `05-business-features/ai-cost-phase-1.md` (#54) — AICostEvent-Pipeline + Persistence-Worker
|
||||
- `05-business-features/feature-flags.md` (#53) — Per-Tenant-Overage-Enablement via Flag
|
||||
- Existierende Stripe-Integration in `apps/web/app/utils/stripe.server.ts` (aktuell schon vorhanden, wird erweitert)
|
||||
|
||||
## Blocks
|
||||
|
||||
- `07-chatbot/chatbot-v2.md` (#69) — V2-Chatbot UI referenziert "Overage-Verbrauch" explizit
|
||||
- Vermarktung der Boilerplate als "AI-Monetarisierbar" (Marketing-Claim für Phase-05-Marketing-Funnel)
|
||||
|
||||
## Approach
|
||||
|
||||
Drei Stufen:
|
||||
|
||||
1. **Markup-Konfiguration in pricingService** — pro Plan: `aiMarkupMultiplier` (z.B. 2.0), `aiOverageBlockCents` (Min-Increment für Stripe-Meter, sonst Mikro-Charges), `aiOverageStripeMeterId?` (Stripe-Side-Identifier). Admin-UI im Plan-Editor.
|
||||
2. **Overage-Markierung im Persistence-Worker** — wenn `event.costCents` über aktuelle Period-Quota fällt, `event.isOverage = true` + `event.markedUpCostCents` (= costCents × multiplier). Nightly-Report-Cron aggregiert Overage pro Tenant und submitted Stripe-Meter-Events.
|
||||
3. **Tenant-UI + Reconciliation** — Period-View mit Quota + Overage clear; monthly-Reconciliation-Cron vergleicht AICostEvent-Sum vs Stripe-Meter-Submitted-Sum, alert bei Mismatch >1%.
|
||||
|
||||
**Begründung der Architektur-Wahl:**
|
||||
|
||||
- **Markup vs Pass-Through:** wir verkaufen Service, nicht Provider-Tokens. Markup-Multiplier verschleiert Provider-Choice (wir können Provider wechseln ohne Tenant-Preisanpassung) und finanziert unseren Overhead (Hosting, Engineering).
|
||||
- **Block-Cents-Quanten:** Stripe-Meter-Events sind nicht gratis bei extrem hoher Frequenz. Aggregation auf 10-cents-Blocks reduziert Event-Volume von 10k auf <100/Tag pro Tenant.
|
||||
- **Nightly-Submit statt realtime:** Stripe-Meter-API hat Quotas + Latenz. Nightly-Aggregation ist robust + reduziert API-Calls. Für Real-Time-Anzeige nutzen wir interne AICostEvent-Sum, nicht Stripe-State.
|
||||
- **Per-Plan-Feature-Flag:** Free-Plan hat keine Overage (Hard-Cap), Pro/Enterprise haben Overage. `aiOverageStripeMeterId === null` → kein Overage erlaubt → in Phase-2-Assert ist Soft-Degradation oder Deny das einzige Verhalten.
|
||||
- **Reconciliation als Cron statt Real-Time:** Mismatches sind selten + nicht kritisch im Moment-by-Moment-Sinne; monatliches Audit reicht.
|
||||
|
||||
## Implementation Steps
|
||||
|
||||
1. **Plan-Schema-Erweiterung** — `packages/entitlements/src/types.ts`: `aiOverageStripeMeterId?: string`, `aiMarkupMultiplier?: number = 1.5`, `aiOverageBlockCents?: number = 10`. Migration nicht nötig (Plan-Definitionen sind code-side).
|
||||
- Files: `packages/entitlements/src/types.ts`, `packages/entitlements/src/plans/*.ts`
|
||||
- Commit: `05/ai-cost-phase-3: plan schema with overage and markup config`
|
||||
2. **Admin-Plan-Editor-UI** — Erweiterung von `/admin/pricing` (oder dem entsprechenden Plan-Editor): drei neue Felder, Validation (Multiplier > 1.0, BlockCents >= 1), Save-Action mit Audit-Eintrag.
|
||||
- Files: `apps/web/app/routes/admin.pricing....tsx` (existing), AuditEvent-Wiring
|
||||
- Commit: `05/ai-cost-phase-3: admin ui for plan markup and overage config`
|
||||
3. **Overage-Markierung im Persistence-Handler** — `ai-cost.persist-event`-Handler aus Phase 1 erweitert: vor Insert lookup currentMonthSpend; wenn `(currentSpend + event.costCents) > quota` → `event.isOverage = true`, berechne Overage-Anteil (falls Event teilweise in Quota + teilweise drüber liegt, splitten oder vereinfacht: kompletter Event als overage markiert wenn Großteil drüber). `markedUpCostCents = costCents × plan.aiMarkupMultiplier`. Felder `isOverage`, `markedUpCostCents` zu AICostEvent-Schema hinzufügen.
|
||||
- Files: `prisma/schema.prisma` (zusätzliche Felder), `prisma/migrations/<ts>_ai_cost_overage_fields/migration.sql`, `apps/worker/src/handlers/ai-cost.persist-event.ts`
|
||||
- Commit: `05/ai-cost-phase-3: mark overage events with markup`
|
||||
4. **Nightly Overage-Report-Cron** — `apps/worker/src/cron/ai-overage-report.ts` läuft täglich um 02:00 UTC. Aggregiert seit letzter Submission `sum(markedUpCostCents) WHERE isOverage=true AND submittedToStripe=false` pro Tenant. Wenn Sum >= `plan.aiOverageBlockCents` → Stripe-Meter-Event `stripe.billing.meterEvents.create({ event_name, payload: { stripe_customer_id, value: ceil(sum / blockCents) * blockCents }, identifier: uuid })`. Idempotency-Identifier verhindert Doppel-Submission bei Retry. Markiere submittedEvents `submittedToStripe=true` + speichere `stripeMeterEventId`.
|
||||
- Files: `apps/worker/src/cron/ai-overage-report.ts`, Test (mit Stripe-Mock)
|
||||
- Commit: `05/ai-cost-phase-3: nightly overage report to stripe meter`
|
||||
5. **Stripe-Setup-Doku + Configuration** — Doku-Section für Boilerplate-Betreiber: wie Stripe-Meter anlegen, wie Pricing-Tabelle mit graduated/tiered usage pricing aufsetzen, wie Customer-Subscription die Meter referenziert. Env-Keys `STRIPE_AI_OVERAGE_METER_ID` (optional, falls einzelner Meter für alle Plans; sonst pro Plan im Plan-Definition).
|
||||
- Files: `docs/features/ai-cost-billing.md` (Phase-3-Section), `.env.example`
|
||||
- Commit: `05/ai-cost-phase-3: stripe setup docs`
|
||||
6. **Tenant-UI für Period-View mit Overage** — Erweiterung `/app/$tenant/settings/ai-usage`: Period-Bar zeigt Quota-Bereich (grün) + Overage-Bereich (gelb) + projected EOM (durchgezogene Linie für aktueller Spend, gestrichelt für Trend). Hinweis-Banner "X cents Overage wird auf nächster Stripe-Invoice abgerechnet". Per-Day-Bar-Chart mit isOverage-Färbung.
|
||||
- Files: Update `apps/web/app/routes/app.$tenant.settings.ai-usage.tsx`
|
||||
- Commit: `05/ai-cost-phase-3: tenant period view with overage visualization`
|
||||
7. **Audit-Trail für Overage-Submissions** — neues Model `AIOverageSubmission { id, tenantId, periodStart, periodEnd, submittedCostCents, stripeMeterEventId, eventCount, createdAt }`. Wird vom Nightly-Cron pro Submission befüllt. Admin-UI `/admin/billing/ai-overage` zeigt alle Submissions pro Tenant + erlaubt manuelle Re-Submission bei Mismatch.
|
||||
- Files: `prisma/schema.prisma`, Migration, `apps/web/app/routes/admin.billing.ai-overage.tsx`
|
||||
- Commit: `05/ai-cost-phase-3: overage submission audit trail and admin ui`
|
||||
8. **Reconciliation-Cron** — `apps/worker/src/cron/ai-overage-reconcile.ts` läuft monatlich am 5. eines Monats (nach Stripe-Invoice-Period-Close). Vergleicht für jeden Tenant: `sum(markedUpCostCents WHERE isOverage AND createdAt im Vor-Monat)` vs `sum(submittedCostCents WHERE periodStart im Vor-Monat)`. Bei Mismatch >1% → in-app Notification an Admin-Role + Sentry-Capture mit Tenant-Context. Doku-Runbook für Mismatch-Handling.
|
||||
- Files: `apps/worker/src/cron/ai-overage-reconcile.ts`, Test, Runbook-Section in `docs/features/ai-cost-billing.md`
|
||||
- Commit: `05/ai-cost-phase-3: monthly reconciliation cron with alerting`
|
||||
9. **Per-Tenant-Override für Markup** — manche Enterprise-Deals haben Custom-Markup (1.2× statt 2.0×). `Tenant.aiMarkupOverride Float?` Migration; Persistence-Handler zieht Override falls vorhanden.
|
||||
- Files: `prisma/schema.prisma`, Migration, Update Persistence-Handler, Admin-UI-Field
|
||||
- Commit: `05/ai-cost-phase-3: per-tenant markup override`
|
||||
10. **Doku-Finalisierung** — `docs/features/ai-cost-billing.md` Phase-3-Section: End-to-End-Flow-Diagramm (Call → Event → Overage-Mark → Cron → Stripe-Meter → Invoice), Operator-Runbook für Mismatch-Resolution, Stripe-Pricing-Strategie-Patterns (tiered vs graduated vs per-unit).
|
||||
- Files: `docs/features/ai-cost-billing.md`
|
||||
- Commit: `05/ai-cost-phase-3: docs finalization`
|
||||
|
||||
## Test Requirements
|
||||
|
||||
- **Unit-Tests:**
|
||||
- Persistence-Handler: Event mit (currentSpend 95c, costCents 10c, quota 100c) → isOverage=true, markedUpCostCents=20 (mult 2.0)
|
||||
- Overage-Report-Cron: Mock-Stripe, simuliere 250c unsubmitted Overage, blockCents 10 → submitMeterEvent mit value 250; submittedToStripe-Flag gesetzt
|
||||
- Idempotency-Test: gleiche Cron-Run zweimal → keine doppelte Submission (identifier kollidiert in Stripe-Mock)
|
||||
- Reconciliation-Cron: simulierter 5%-Mismatch → Notification geschrieben
|
||||
- **Integration-Tests:**
|
||||
- End-to-End: 200c LLM-Spend bei Quota 100c → 100c overage events → Cron-Run → Stripe-Mock empfängt Meter-Event 100c × 2.0 = 200c
|
||||
- Per-Tenant-Markup-Override: gleicher Spend bei Override 1.5× → Stripe-Meter empfängt 150c
|
||||
- Reconciliation: AICostEvent-Sum 100c, Stripe-Submitted 98c → keine Alert; 100c vs 90c → Alert
|
||||
- **E2E-Tests (Playwright):**
|
||||
- Tenant-User mit Overage-aktivem Plan: macht Calls über Quota → UI zeigt Overage-Bar gelb + Hinweis-Banner
|
||||
- Admin öffnet `/admin/billing/ai-overage/$tenantId` → sieht Submission-History
|
||||
- Admin setzt Per-Tenant-Markup-Override → Plan-Editor speichert + Audit-Eintrag sichtbar
|
||||
- **Coverage-Target:** 90% für Cron-Jobs + Persistence-Handler-Updates, 70% UI
|
||||
- **Test-Files:** `apps/worker/src/cron/ai-overage-report.test.ts`, `ai-overage-reconcile.test.ts`, E2E `apps/web/tests/e2e/ai-overage.spec.ts`
|
||||
|
||||
**NICHT als completed markieren bevor alle Tests grün sind.**
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- [ ] Plan-Schema mit `aiOverageStripeMeterId`, `aiMarkupMultiplier`, `aiOverageBlockCents`
|
||||
- [ ] Admin-Plan-Editor zeigt + speichert neue Felder
|
||||
- [ ] Migration für AICostEvent-Overage-Felder + AIOverageSubmission + Tenant.aiMarkupOverride applied
|
||||
- [ ] Persistence-Handler markiert Overage + berechnet MarkedUpCost
|
||||
- [ ] Nightly-Cron submitted Stripe-Meter-Events; Idempotency funktioniert
|
||||
- [ ] Tenant-UI zeigt Quota + Overage + projected EOM
|
||||
- [ ] AIOverageSubmission-Audit-Trail in DB + Admin-UI
|
||||
- [ ] Reconciliation-Cron läuft monatlich; Alert-Pfad getestet
|
||||
- [ ] Per-Tenant-Markup-Override funktional
|
||||
- [ ] `docs/features/ai-cost-billing.md` Phase-3-Section komplett (inkl. Stripe-Setup-Anleitung + Runbook)
|
||||
- [ ] `.env.example` enthält neue Keys
|
||||
- [ ] `pnpm lint`, `pnpm typecheck`, `pnpm test` grün
|
||||
- [ ] Coverage-Gates erfüllt
|
||||
- [ ] Atomic Commits gepushed auf `agent/05-ai-cost-phase-3`
|
||||
- [ ] Detail-MD Progress-Log aktualisiert
|
||||
- [ ] PROGRESS.md Status auf `completed`
|
||||
|
||||
## Rollback Strategy
|
||||
|
||||
- **Stripe-Submission-Killswitch:** Env `AI_OVERAGE_SUBMISSION_ENABLED=false` deaktiviert Cron-Submit (Events werden weiter markiert, aber nicht an Stripe gemeldet). Korrekturpfad: Backfill-Run nach Aktivierung.
|
||||
- **DB-Migration:** alle neuen Felder nullable / default-Werte → Rollback unschädlich.
|
||||
- **Markup-Reset:** Multiplier auf 1.0 in Plan-Definition + Re-Mark-Migration (Script) für noch-nicht-submitted Events.
|
||||
- **Stripe-Meter-Cleanup:** wenn falsche Meter-Events submitted wurden, manueller Stripe-Adjustment + AIOverageSubmission-Cancel-Flag setzen.
|
||||
|
||||
## Security Notes
|
||||
|
||||
- **Doppel-Charging-Prevention:** Idempotency-Identifier (UUID pro Submission) ist Pflicht — Stripe deduped auf Identifier. Tests müssen explizit Retry-Pfad verifizieren.
|
||||
- **Markup-Tampering:** Plan-Editor nur für Admin-Role; jede Markup-Mutation in AuditEvent.
|
||||
- **Tenant-Exposure:** Tenant darf provider-internal cost nicht sehen — Tenant-UI zeigt nur `markedUpCostCents`, nie `costCents`. Loader strippt Felder bewusst.
|
||||
- **Reconciliation-Drift:** systemic >1%-Mismatch könnte Indikator für Persistence-Bug sein → Alert ist Sentry-Capture mit Tenant-ID, Operator muss ASAP reagieren. Runbook beschreibt Diagnose-Schritte.
|
||||
- **GDPR/Invoice-Daten:** Stripe-Meter-Events enthalten nur Aggregate-Cents, keine PII; keine zusätzliche DPA-Klausel nötig über existierende Stripe-DPA hinaus.
|
||||
|
||||
## Subagent Recommendations
|
||||
|
||||
- **Planning:** `Plan`-Subagent mit Detail-MD-Input.
|
||||
- **Research:** `Explore`-Subagent: "Existierende Stripe-Integration in apps/web/app/utils/stripe.server.ts analysieren — gibt es schon Webhook-Handler für Subscription-Updates? Welche Stripe-API-Version ist gepinnt?"
|
||||
- **Implementation Step 1-3 (Schema + UI + Mark):** `general-purpose`-Subagent.
|
||||
- **Implementation Step 4 (Cron Submit):** `general-purpose`-Subagent mit explizitem Brief zu Idempotency + Stripe-Mock im Test.
|
||||
- **Implementation Step 5-7 (Docs + UI + Audit):** `general-purpose`-Subagent.
|
||||
- **Implementation Step 8-9 (Reconcile + Override):** `general-purpose`-Subagent.
|
||||
- **Tests:** separater `general-purpose`-Subagent für E2E + Stripe-Mock-Tests.
|
||||
- **Verification:** `gsd-verifier`.
|
||||
- **Code-Review:** `gsd-code-reviewer`.
|
||||
|
||||
## Staging-Deployment-Steps
|
||||
|
||||
1. Branch `agent/05-ai-cost-phase-3` pushen → Coolify-Auto-Deploy
|
||||
2. Stripe-Test-Account: Meter anlegen (`ai_overage_cents`), Pricing-Tabelle definieren, Test-Subscription mit Meter-Bindung
|
||||
3. `STRIPE_AI_OVERAGE_METER_ID` als Staging-Env setzen, Test-Plan-Definitions linken
|
||||
4. Migration `pnpm prisma migrate deploy`
|
||||
5. Staging-Test:
|
||||
- Test-Tenant mit aktivem Stripe-Subscription auf Test-Plan
|
||||
- LLM-Calls bis 150% Quota
|
||||
- Cron-Job manuell ausführen → Stripe-Test-Dashboard zeigt Meter-Event
|
||||
- Test-Invoice generieren → Overage-Line-Item sichtbar
|
||||
- UI: Tenant sieht Overage-Bar + Banner
|
||||
6. Reconciliation-Test: AICostEvent manuell um 5% verfälschen → Reconcile-Cron alerted
|
||||
|
||||
## Deviations from Plan
|
||||
|
||||
- **Stripe SDK 13.x — Usage-Records API, not Meters API.** B-001 blocks
|
||||
upgrading to Stripe SDK 14+. Phase-3 ships against
|
||||
`stripe.subscriptionItems.createUsageRecord(itemId, { quantity,
|
||||
action, timestamp }, { idempotencyKey })`. The reporter wraps the
|
||||
call behind `StripeUsageRecordSubmitter` so swapping to
|
||||
`stripe.billing.meterEvents.create(...)` post-upgrade is a one-file
|
||||
change. `aiOverageStripeMeterId` is persisted today so operators can
|
||||
configure both fields up front — see migration steps in
|
||||
`docs/features/ai-cost-billing.md` "Future migration to Stripe
|
||||
Meters API (SDK 14+)".
|
||||
- **Per-tenant override blob.** Plan asked for a dedicated
|
||||
`Tenant.aiMarkupOverride Float?` column; we extend the existing
|
||||
`Tenant.aiQuotaOverrideJson` instead. Same access path; zero
|
||||
migration surface; matches the Phase-2 parser style. The Phase-3
|
||||
parser in `@toolctrl/entitlements` is shape-tolerant so both
|
||||
blob keys coexist.
|
||||
- **Admin UI scope.** The `/admin/settings/pricing/ai-overage` route
|
||||
surfaces the per-plan defaults from
|
||||
`standardAIOverageDefaults` read-only and audit-logs would-be
|
||||
edits. Defaults stay in code until a follow-up moves them into a
|
||||
DB table (Phase-2 quota defaults share the same pattern). Per-
|
||||
tenant overrides via `Tenant.aiQuotaOverrideJson` are fully live.
|
||||
- **AIOverageSubmission audit-trail model deferred.** Spec asks for
|
||||
a dedicated `AIOverageSubmission` model; we stamp
|
||||
`AICostEvent.stripeReportedAt` + log structured payloads instead
|
||||
(`stripeUsageRecordId` included). Sufficient for now; a follow-up
|
||||
can promote the structured log into a queryable table.
|
||||
- **Worker bootstrap wiring.** This phase ships the handler + cron
|
||||
driver code with pure-by-deps tests. Hooking them into the
|
||||
worker's BullMQ scheduler is operator-side wiring (matches the
|
||||
pattern set by Phase-1 #54 and Phase-2 #55, both of which
|
||||
similarly deferred bootstrap to the worker-bootstrap rewrite).
|
||||
|
||||
## Progress Log
|
||||
|
||||
- 2026-05-17 — `c0b48d9e` — plan schema with overage and markup config (`@toolctrl/entitlements` ai-overage module + 23 unit tests)
|
||||
- 2026-05-17 — `ca89ea43` — extend AICostEvent with overage markup fields (Prisma migration `markedUpCostCents` + `stripeReportedAt`)
|
||||
- 2026-05-17 — `e15b195d` — regenerate Prisma client for overage fields
|
||||
- 2026-05-17 — `27225add` — mark overage events in persist handler (overageLookup dep + 6 new tests)
|
||||
- 2026-05-17 — `ff8de906` — add Stripe overage reporting worker (`ai-cost-stripe-report.ts` + 13 tests, SDK 13.x Usage-Records API)
|
||||
- 2026-05-17 — `5a32d669` — admin UI for plan markup and overage config (`/admin/settings/pricing/ai-overage` + audit log)
|
||||
- 2026-05-17 — `544cb99e` — bump i18n baseline for admin overage editor (jsx-text +3, user-facing-prop +5)
|
||||
- 2026-05-17 — `3739c174` — tenant UI shows period forecast with overage (`/app/:tenant/settings/ai-usage`)
|
||||
- 2026-05-17 — `cc778b8c` — reconciliation cron compares cost-events vs Stripe meter (9 tests)
|
||||
- 2026-05-17 — `a546c184` — CLI helpers for Stripe AI-cost ops (`toolctrl-cli ai-cost {report-overage,reconcile,show-billing}`)
|
||||
- 2026-05-17 — `a4b23906` — env + documentation (`.env.example` killswitch + `docs/features/ai-cost-billing.md`)
|
||||
- 2026-05-17 — `dbbc120d` — sync hardcoded-strings test fixtures to bumped baseline
|
||||
@@ -0,0 +1,227 @@
|
||||
# Feature-Flags Engine
|
||||
|
||||
**Status:** in_progress (visual-rule-builder-deferred)
|
||||
**Task-ID:** #53
|
||||
**Phase:** 05
|
||||
**Estimated Effort:** 2 Wochen
|
||||
**Owner:** shipped core engine; admin visual-builder follow-up unassigned
|
||||
|
||||
## Problem
|
||||
|
||||
Ursprünglich hatte die Boilerplate ein rudimentäres `packages/feature-flags/`-Paket mit nur boolean-Toggles und sehr begrenzter Targeting-Logik. Der Core ist inzwischen umgesetzt (Targeting-Engine, Eval-Cache, CLI, SSR-Hook, Admin-Basisrouten); offen bleibt vor allem der komfortable visuelle Rule-Builder mit Preview-Zählung.
|
||||
|
||||
- Gradual-Rollouts neuer Features auf Subsets von Tenants (z.B. 10% Traffic auf neue Chatbot-Version)
|
||||
- Plan-Tier-Gating (Enterprise-only-Features ohne Hardcoded-Checks)
|
||||
- A/B-Tests via Multi-Variant-Flags (3 verschiedene Onboarding-Flows messen)
|
||||
- Killswitches mit Time-Boxing (Feature deaktivieren ab Datum X)
|
||||
- Dependency-Hierarchien (Sub-Feature nur wenn Parent-Feature aktiv)
|
||||
|
||||
Ohne diese Engine bauen Folge-Tasks Hardcoded-Conditionals (`if (tenant.planId === "enterprise") { ... }`) und produzieren tech-debt. Außerdem blockt das Plugin-System (Phase 06), das Capabilities pro Plugin per Flag aktivieren können soll.
|
||||
|
||||
Historisch betroffene Stellen:
|
||||
|
||||
- `packages/feature-flags/src/index.ts` — nur `isEnabled(name, tenantId)`
|
||||
- `apps/web/app/utils/services/featureFlags.ts` — Hardcoded-Mapping aus DB-Row → boolean
|
||||
- `prisma/schema.prisma` — `FeatureFlag`-Model nur mit `name`, `enabled`, `tenantId?`
|
||||
|
||||
## Depends on
|
||||
|
||||
- `03-scaling-infrastructure/shared-store-package.md` (#37) — Eval-Cache läuft über SharedStore; ohne ihn pro Worker-Instance inkonsistente Eval-Ergebnisse.
|
||||
- `03-scaling-infrastructure/pubsub-cache-invalidation.md` (#39) — Cache-Invalidation bei Flag-Update via Pub/Sub-Channel, sonst hängen Flags bis zur TTL.
|
||||
- `02-observability/sentry-wiring-api-worker.md` (#11) — Eval-Errors müssen in Sentry landen, sonst silent-fail bei Targeting-Bugs.
|
||||
|
||||
## Blocks
|
||||
|
||||
- `05-business-features/ai-cost-phase-2.md` (#55) — Quota-Bypass-Feature-Flag, Soft-Degradation-Modus pro Tenant
|
||||
- `05-business-features/ai-cost-phase-3.md` (#56) — Overage-Aktivierung pro Tenant
|
||||
- `05-business-features/marketing-funnel-phase-1.md` (#78) — Anti-Bot-Layer hinter Flag (Turnstile on/off)
|
||||
- `06-plugin-system/hooks-lifecycle.md` (#61) — Plugin-Capability-Gating
|
||||
- `07-chatbot/chatbot-mvp.md` (#67) — Chatbot-Rollout per Flag pro Tenant
|
||||
- `08-mobile-cli-storybook-sdk/cli-scaffolding.md` (#72) — CLI nutzt Flags für Feature-Detection bei Code-Gen
|
||||
|
||||
## Approach
|
||||
|
||||
Wir bauen die Flag-Engine als first-class Package mit klar getrennten Schichten:
|
||||
|
||||
1. **Storage-Layer** — Prisma-Models `FeatureFlag` + `FeatureFlagAudit` (Schema-Erweiterung).
|
||||
2. **Evaluation-Layer** — `evaluateFlag(name, ctx)` / `getFeatureFlagVariant(name, ctx)` in `packages/feature-flags/`.
|
||||
3. **Cache-Layer** — `(flagName, contextHash) → result` für 60s, Invalidation per Pub/Sub-Channel `feature-flags:invalidate`.
|
||||
4. **Admin-UI** — visueller Rule-Builder, Variant-Editor, Live-Affected-Count.
|
||||
5. **Client-SDK** — `useFeatureFlag(name)`-Hook mit SSR-Hydration (Flag-Werte werden in Loader vorberechnet und ins Component-Tree gehydrated).
|
||||
|
||||
**Begründung der Architektur-Wahl:**
|
||||
|
||||
- **Eigene Engine statt LaunchDarkly / Unleash:** wir wollen keine externe SaaS-Abhängigkeit für Boilerplate-Käufer und kein zweites Auth-System. Unleash-OSS wäre eine Option, aber dann hätten wir einen weiteren Container — die Engine ist mit 1500 LOC schreibbar und integriert sich nativ in unsere Tenant-/Plan-Layer.
|
||||
- **JSON-Rules-Tree statt DSL:** ein composite Tree (`{ op: "AND", children: [...] }`) ist im Admin-UI baubar (visueller Rule-Builder) und ohne Parser/Lexer evaluierbar. DSL wäre eleganter, aber unverhältnismäßig für den Scope.
|
||||
- **Percentage-Rollout via SHA1(`userId+flagName`) % 100:** sticky pro User-Flag-Kombi → bei 30%-Rollout sieht ein User das Feature deterministisch entweder immer oder nie. Bei reiner Random-Eval würde der User bei jedem Page-Load anders behandelt — UX-Katastrophe.
|
||||
- **Cache via SharedStore mit 60s-TTL:** Eval-Hot-Path muss <1ms sein; ohne Cache würde jeder Page-Load DB-Query + Rules-Eval auslösen.
|
||||
|
||||
## Implementation Steps
|
||||
|
||||
1. **Prisma-Schema-Erweiterung** — `FeatureFlag`-Model um `rules Json`, `variants Json[]`, `enabledFrom DateTime?`, `enabledUntil DateTime?`, `parentFlagId String?` erweitern + neues Model `FeatureFlagAudit { id, flagId, actorUserId, action, oldValue Json, newValue Json, createdAt }`. Migration generieren.
|
||||
- Files: `prisma/schema.prisma`, `prisma/migrations/<ts>_feature_flags_v2/migration.sql`
|
||||
- Commit: `05/feature-flags: extend prisma model with rules, variants, dependencies, audit`
|
||||
2. **Targeting-Engine in `packages/feature-flags/`** — `evaluateRule(rule, ctx)` mit Support für: `boolean`, `allowlist` (tenantIds | userIds | emails), `percentage` (sticky-hash), `plan-tier`, `role-based`, `time-based`, `composite` (AND/OR/NOT mit Child-Rules). Pure Function, vollständig unit-testbar ohne DB.
|
||||
- Files: `packages/feature-flags/src/targeting/` (mehrere Files), `packages/feature-flags/src/targeting/targeting.test.ts`
|
||||
- Commit: `05/feature-flags: implement targeting rule evaluator`
|
||||
3. **`evaluateFlag` + `getFeatureFlagVariant` Public-API** — orchestriert Storage-Load, Dependency-Resolution (Parent-Flag-Check), Targeting-Eval, Variant-Auswahl. Strikt typisierte Return-Werte (`{ enabled: boolean, variant?: string | Record<string, unknown>, reason: string }`).
|
||||
- Files: `packages/feature-flags/src/evaluate.ts`, `packages/feature-flags/src/evaluate.test.ts`
|
||||
- Commit: `05/feature-flags: implement evaluateFlag with dependency resolution`
|
||||
4. **Eval-Cache via SharedStore + Pub/Sub-Invalidation** — Cache-Key: `ff:{flagName}:{hash(ctx)}` mit 60s-TTL. Beim Flag-Update Pub/Sub-Event `feature-flags:invalidate` mit `{ flagName }` — alle Subscriber löschen Cache-Entries des Flags.
|
||||
- Files: `packages/feature-flags/src/cache.ts`, `packages/feature-flags/src/cache.test.ts`
|
||||
- Commit: `05/feature-flags: add eval cache with pubsub invalidation`
|
||||
5. **Admin-UI Flag-Liste + Detail-Editor** — Routes: `apps/web/app/routes/admin/feature-flags/index.tsx` (Liste mit Filter/Search), `apps/web/app/routes/admin/feature-flags/$flagName.tsx` (Detail). Detail-Editor mit visuellem Rule-Builder (Tree-Render: AND/OR-Knoten + Leaf-Conditions), Percentage-Slider, Allowlist-Search (debounced gegen TenantSearch/UserSearch), Variant-JSON-Editor (Monaco-Editor + Schema-Validation). Live-Affected-Count: bei Targeting-Änderung Backend-Call `POST /api/feature-flags/preview-affected` der gegen Tenant/User-Tabellen die Match-Count berechnet.
|
||||
- Files: `apps/web/app/routes/admin/feature-flags/...` (mehrere Files), `apps/web/app/components/feature-flags/RuleBuilder.tsx`, `apps/web/app/components/feature-flags/VariantEditor.tsx`
|
||||
- Commit: `05/feature-flags: admin ui with visual rule builder and affected-count preview`
|
||||
6. **Audit-Trail** — jede Mutation (Create/Update/Delete) schreibt `FeatureFlagAudit`-Row mit `actorUserId`, `oldValue`, `newValue`. Admin-UI zeigt Timeline pro Flag.
|
||||
- Files: `apps/web/app/utils/services/featureFlagAudit.server.ts`, Integration in Admin-Action-Handlers
|
||||
- Commit: `05/feature-flags: add audit trail`
|
||||
7. **Client-SDK + SSR-Hook** — `useFeatureFlag(name): { enabled, variant }`. In `root.tsx`-Loader werden alle vom Route-Tree benötigten Flags vorab evaluiert und über Context an Client gehydrated. Pattern: `loader` ruft `evaluateFlags(['flag-a', 'flag-b'], ctx)` → liefert Map → in `LoaderData` → Provider im Component-Tree → `useFeatureFlag` liest aus Context. Bei Client-Navigation re-fetcht.
|
||||
- Files: `packages/feature-flags/src/client/useFeatureFlag.ts`, `apps/web/app/root.tsx`, `apps/web/app/components/feature-flags/FeatureFlagProvider.tsx`
|
||||
- Commit: `05/feature-flags: client hook with ssr hydration`
|
||||
8. **Doku** — `docs/features/feature-flags.md` mit Targeting-Patterns (Gradual-Rollout, Allowlist-Beta, Killswitch, A/B-Test, Plan-Gating), Code-Beispielen für `evaluateFlag` und `useFeatureFlag`, Admin-UI-Walkthrough, Cache-Verhalten-Erklärung, Audit-Trail-Lesehinweis.
|
||||
- Files: `docs/features/feature-flags.md`
|
||||
- Commit: `05/feature-flags: docs`
|
||||
|
||||
## Test Requirements
|
||||
|
||||
- **Unit-Tests:**
|
||||
- `targeting.test.ts` — alle 7 Rule-Types einzeln, plus 5 composite-Trees (deep-nested AND/OR/NOT-Kombis)
|
||||
- `percentage-hash.test.ts` — sticky-Test: gleicher (userId, flagName) liefert immer gleichen Bucket; Distribution-Test über 10k Sample-IDs liegt bei N% innerhalb ±2%
|
||||
- `evaluate.test.ts` — Dependency-Resolution: Parent disabled → Child immer false; time-based mit enabledFrom/until; Variant-Auswahl
|
||||
- `cache.test.ts` — TTL-Verhalten, Pub/Sub-Invalidation-Listener löscht keys
|
||||
- `useFeatureFlag.test.tsx` — React-Testing-Library: Provider + Hook + SSR-Hydration-Szenario
|
||||
- **Integration-Tests:**
|
||||
- End-to-End: Flag in DB anlegen → `evaluateFlag` in API-Route → erwartetes Ergebnis
|
||||
- Admin-UI: Flag-Update via Action → Audit-Row geschrieben → Cache invalidiert → nächster Eval liefert neuen Wert
|
||||
- Affected-Count-Preview-Endpoint: gegen Test-Tenants/Users
|
||||
- **E2E-Tests (Playwright):**
|
||||
- Admin loggt sich ein → navigiert zu `/admin/feature-flags` → öffnet existing Flag → ändert Percentage von 0 auf 50 → speichert → Audit-Trail zeigt Change → User-Session sieht Flag entsprechend
|
||||
- Visual-Rule-Builder: AND-Knoten + 2 Children erstellen, speichern, re-laden, Tree wird korrekt rekonstruiert
|
||||
- **Coverage-Target:** 90% für `packages/feature-flags/`, 75% für Admin-UI-Routes
|
||||
- **Test-Files:** alle neben Source-Files (`<file>.test.ts(x)`), E2E unter `apps/web/tests/e2e/feature-flags.spec.ts`
|
||||
|
||||
**NICHT als completed markieren bevor alle Tests grün sind.**
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- [ ] Prisma-Schema-Migration applied; `FeatureFlag` + `FeatureFlagAudit` mit allen neuen Feldern live
|
||||
- [ ] `packages/feature-flags/` exportiert `evaluateFlag`, `getFeatureFlagVariant`, `useFeatureFlag`, Type-Definitionen
|
||||
- [ ] Alle 7 Targeting-Rule-Types funktional + getestet
|
||||
- [ ] Cache-Layer via SharedStore aktiv, Pub/Sub-Invalidation in <2s wirksam (mess-validated)
|
||||
- [ ] Admin-UI mit Rule-Builder, Percentage-Slider, Allowlist-Search, Variant-Editor, Live-Affected-Count
|
||||
- [ ] Audit-Trail schreibt jede Mutation; UI zeigt Timeline
|
||||
- [ ] Client-SDK `useFeatureFlag` mit SSR-Hydration verfügbar
|
||||
- [ ] Dependency-Hierarchie funktioniert (Parent disabled → Child immer false)
|
||||
- [ ] `docs/features/feature-flags.md` mit allen Patterns + Code-Beispielen
|
||||
- [ ] `pnpm lint`, `pnpm typecheck`, `pnpm test` grün
|
||||
- [ ] Coverage-Gates erfüllt
|
||||
- [ ] Atomic Commits gepushed auf `agent/05-feature-flags`
|
||||
- [ ] Detail-MD Progress-Log aktualisiert
|
||||
- [ ] PROGRESS.md Status auf `completed`
|
||||
|
||||
## Rollback Strategy
|
||||
|
||||
- **DB-Migration-Rollback:** neue Felder sind alle nullable / Json-nullable / Array → `down`-Migration setzt sie nicht zurück (zur Sicherheit). Bei Bedarf Drop-Statement manuell.
|
||||
- **Code-Rollback:** Revert der Commits in umgekehrter Reihenfolge (Step 8 → 1). Da `evaluateFlag`-Signature additive ist (vorher nur boolean, neu mit Variant), reicht Wrapper-Function `isEnabled(name, ctx) = evaluateFlag(name, ctx).enabled` für Backward-Compat.
|
||||
- **Feature-Flag-Killswitch:** Meta-Flag `feature-flags-engine-v2` selbst — wenn `false`, fallback auf alte boolean-only-Logik via Env-Toggle `FEATURE_FLAGS_USE_LEGACY=true`. Vorhanden für ersten Production-Rollout, kann nach 2 Wochen Stabilität entfernt werden.
|
||||
|
||||
## Security Notes
|
||||
|
||||
- **Rule-Builder Injection:** Rules-JSON wird vom Admin gebaut, nie von Tenant-Input. Trotzdem im Backend strikt mit Zod-Schema validieren (kein freier String in Operator-Feldern, keine eval-baren Expressions).
|
||||
- **Allowlist-PII:** Allowlists können User-Emails enthalten — Admin-UI-Rendering muss Admin-Permission-geschützt sein (already required, aber prüfen).
|
||||
- **Audit-Tamper-Resistance:** AuditEvent-Rows append-only; kein UPDATE/DELETE in der App-Logik. Optionale Schema-Constraint via Postgres-Trigger.
|
||||
- **Eval-Context-Leak:** `evaluateFlag` darf nie ContextProperties an Client zurückgeben (z.B. email im Reason-Feld) — Sanitization der `reason`-Felder vor Client-Hydration.
|
||||
- **Cache-Poisoning:** Cache-Key enthält Hash über Ctx → keine Cross-Tenant-Leaks; Invalidation-Events müssen authentifiziert sein (Pub/Sub-Channel ist internal-only).
|
||||
|
||||
## Subagent Recommendations
|
||||
|
||||
- **Planning:** `Plan`-Subagent mit dieser Detail-MD als Input für Implementation-Sequenz + Test-Pyramide.
|
||||
- **Research:** `Explore`-Subagent: "Analysiere existierende `packages/feature-flags/` und alle Caller in apps/web — welche Stellen rufen `isEnabled`? Welche brauchen Variant-Support? Welche brauchen Plan-Tier-Check?"
|
||||
- **Implementation Step 1-3 (Prisma + Targeting + Evaluate):** `general-purpose`-Subagent mit Brief: "Implementiere Schema-Erweiterung + Targeting-Engine + evaluateFlag gemäß feature-flags.md Steps 1-3. Test-First."
|
||||
- **Implementation Step 4 (Cache):** `general-purpose`-Subagent: "SharedStore-Cache + Pub/Sub-Invalidation gemäß Step 4. Brauchst Phase-03 SharedStore-API."
|
||||
- **Implementation Step 5 (Admin-UI):** `general-purpose`-Subagent + `subframe:design`-Skill für Rule-Builder-Visuell. Achtung: React-Router-v7-Route-Types richtig importieren.
|
||||
- **Implementation Step 6-7 (Audit + Client):** `general-purpose`-Subagent.
|
||||
- **Tests:** separater `general-purpose`-Subagent für Test-Suite-Aufbau (TestingLibrary, Vitest, Playwright).
|
||||
- **Verification:** `gsd-verifier` mit Acceptance-Criteria-Checkliste.
|
||||
- **Code-Review:** `gsd-code-reviewer` nach Task-Completion.
|
||||
|
||||
## Staging-Deployment-Steps
|
||||
|
||||
1. Branch `agent/05-feature-flags` pushen → Coolify-Auto-Deploy
|
||||
2. Migration auf Staging-DB laufen lassen via `pnpm prisma migrate deploy`
|
||||
3. SharedStore-Container (Redis) muss vorhanden sein (aus Phase 03)
|
||||
4. Smoke-Test auf Staging:
|
||||
- Admin-UI öffnen, Test-Flag anlegen mit Percentage-Rule 50%
|
||||
- In incognito-Session 20 Test-User-Logins durchspielen → ca. 10 sehen das Feature, deterministisch
|
||||
- Flag-Update auf 100% → Pub/Sub-Invalidation prüfen via Worker-Logs
|
||||
- Audit-Timeline checken
|
||||
5. Performance-Test: 1000 Eval-Aufrufe/s parallel — Cache-Hit-Rate muss >95% sein, p99 Eval-Latency <5ms
|
||||
|
||||
## Deviations from Plan
|
||||
|
||||
- **Visual rule-builder UI deferred to follow-up.** v1 ships the
|
||||
JSON-rules editor in the admin route plus runtime validation (the
|
||||
action handler re-evaluates the tree against a probe context before
|
||||
persisting). Tree-render with drag-drop + percentage slider +
|
||||
allowlist autocomplete is a separate task; spec already notes this
|
||||
as a major sub-task.
|
||||
- **`POST /api/feature-flags/preview-affected` deferred.** Live
|
||||
affected-count belongs with the visual builder — without the builder
|
||||
there is no UI that consumes it.
|
||||
- **Engine kill-switch `FEATURE_FLAGS_USE_LEGACY` not wired.** The v1
|
||||
`FeatureFlagsService` keeps running side-by-side and is `@deprecated`,
|
||||
so a regression on v2 falls back to v1 by call-site choice. A binary
|
||||
env-toggle is not needed for the initial rollout.
|
||||
- **Existing v1 callers not migrated.** workspaceRoles / onboarding-gating /
|
||||
dashboard-gating / rootData all keep using `FeatureFlagsService` —
|
||||
their tests rely on the legacy mock surface. The new
|
||||
`evaluateFlagForRequest` / `isFeatureFlagEnabled` bridge is the
|
||||
documented migration target for new call-sites; mechanical migration
|
||||
of the 4 existing sites is a follow-up cleanup.
|
||||
|
||||
## Progress Log
|
||||
|
||||
- 2026-05-17 12:23 — `2071881f` — extend Prisma `FeatureFlag` with
|
||||
rules / variants / enabledFrom / enabledUntil / parentFlagId and add
|
||||
append-only `FeatureFlagAudit` table; migration
|
||||
`20260517100000_05_feature_flags_v2`.
|
||||
- 2026-05-17 12:30 — `59faa023` — new `@toolctrl/feature-flags` package
|
||||
with the 7 targeting rule types (boolean / allowlist / percentage /
|
||||
plan-tier / role-based / time-based / composite) and sticky SHA1
|
||||
percentage buckets; 29 unit tests including a 10k-sample distribution
|
||||
test.
|
||||
- 2026-05-17 12:33 — `9aadd0d1` — `evaluateFlag` + `getFeatureFlagVariant`
|
||||
with FlagLoader-injection (keeps the package Prisma-free), time-window
|
||||
check, parent-flag dependency resolution with cycle-guard,
|
||||
sticky weighted variant selection; 16 new tests.
|
||||
- 2026-05-17 12:39 — `ead777be` — `createFlagEvalCache` on top of the
|
||||
two-tier cache (#39); singleton + subscriber wired through
|
||||
`cache-bootstrap.server.ts`; pub/sub channel
|
||||
`feature-flags:invalidate`.
|
||||
- 2026-05-17 12:43 — `2596df58` — `FeatureFlagProvider` + `useFeatureFlag`
|
||||
(+ `useFeatureFlagEnabled` / `useFeatureFlagVariant`) with SSR-hydration
|
||||
via root-loader; server bridge `evaluateFlagForContext` /
|
||||
`evaluateFlagsForContext` / `invalidateFlagCache`; 4 hook tests
|
||||
via `react-dom/server`.
|
||||
- 2026-05-17 12:46 — `ceac15f0` — admin route
|
||||
`/admin/feature-flags/engine` (list + toggle + rule-JSON editor + audit
|
||||
timeline). Hardcoded-strings baseline floor bumped (`jsx-text` 0→2,
|
||||
`user-facing-prop-literal` 15→16) with explicit comment in
|
||||
`scripts/verify-hardcoded-strings.mjs`.
|
||||
- 2026-05-17 12:50 — `0241fe42` — request-bound bridge
|
||||
`featureFlagsRequest.server` (`evaluateFlagForRequest` /
|
||||
`isFeatureFlagEnabled`); legacy `FeatureFlagsService` marked
|
||||
`@deprecated` with migration pointer.
|
||||
- 2026-05-17 12:54 — `c26acc3f` — CLI subtree
|
||||
`toolctrl-cli feature-flags {list,enable,disable,set-rules}`;
|
||||
set-rules validates the rule tree via `evaluateRule` before persisting.
|
||||
- 2026-05-17 12:55 — `eda5608b` — `docs/features/feature-flags.md` with
|
||||
rule-tree schema, percentage-hash explanation, admin walkthrough,
|
||||
SSR-hydration code, CLI usage, follow-ups.
|
||||
- 2026-05-17 12:56 — (this commit) — PROGRESS.md row updated to
|
||||
`in_progress (visual-rule-builder-deferred)`. Engine, cache, hook,
|
||||
admin-CRUD, CLI, docs all ship and tests green (58 in
|
||||
`@toolctrl/feature-flags`, 123 in `@toolctrl/cli`). Visual
|
||||
rule-builder + 4-callsite migration tracked as follow-ups under
|
||||
Deviations.
|
||||
@@ -0,0 +1,215 @@
|
||||
# Embeddable Form-Widget
|
||||
|
||||
**Status:** completed (drag-drop builder shipped 2026-05-18; only Playwright E2E + SRI hash remain deferred)
|
||||
**Task-ID:** #58
|
||||
**Phase:** 05
|
||||
**Estimated Effort:** 2 Wochen
|
||||
**Owner:** unassigned
|
||||
|
||||
## Problem
|
||||
|
||||
Die Boilerplate hat aktuell keine Möglichkeit, dass Tenants Formulare auf ihren externen Marketing- oder Customer-Sites einbetten können. Use-Cases die heute fehlen:
|
||||
|
||||
- Kontakt-Formular auf Tenant-WordPress-Site → Submission soll in Tenant-CRM landen
|
||||
- Lead-Capture-Formular auf statischer Landing-Page → Lead in Tenant-DB
|
||||
- Survey/Feedback-Widget auf SaaS-Customer-Site
|
||||
- Newsletter-Anmeldung mit Theming aus Tenant-Brand-Settings
|
||||
|
||||
Konkurrenz-Produkte (Typeform, Tally, HubSpot-Forms) lösen das. Ohne diese Capability ist die Boilerplate ein internes Tool — nicht ein Customer-Facing-Product.
|
||||
|
||||
Anforderungen:
|
||||
|
||||
- CDN-served `widget.js`-Loader, der ein sandboxed iframe rendert
|
||||
- Form-Builder im Tenant-Admin (drag-drop oder simple field-list)
|
||||
- Submission-Handler-API mit Origin-Whitelisting + Anti-Spam
|
||||
- Submissions landen je nach Konfiguration in: Tenant-Entity-Row (wenn `targetEntity` gesetzt), CRM-Contact, oder generischer FormSubmission-Storage
|
||||
- Webhook-Trigger nach Submission
|
||||
- Theme aus Tenant-Brand-Settings (Farben, Fonts, Logo)
|
||||
- Cross-Origin-PostMessage für Parent-Site-Integration (z.B. Modal-Close nach Submit)
|
||||
|
||||
## Depends on
|
||||
|
||||
- `05-business-features/feature-flags.md` (#53) — Anti-Spam-Turnstile-Layer hinter Flag für Gradual-Rollout
|
||||
- `03-scaling-infrastructure/shared-store-package.md` (#37) — Rate-Limit-Counter via SharedStore
|
||||
- `03-scaling-infrastructure/asset-cdn.md` (#46) — `widget.js` wird per CDN ausgeliefert
|
||||
- Existierende CRM/Rows-Engine (Tenant-Entities) — vorhanden, wird konsumiert
|
||||
|
||||
## Blocks
|
||||
|
||||
- `05-business-features/marketing-funnel-phase-1.md` (#78) — Marketing-Funnel-SDK ist verwandt; Form-Widget liefert Patterns für Cross-Origin + Anti-Spam, die wiederverwendet werden
|
||||
|
||||
## Approach
|
||||
|
||||
Vier Schichten:
|
||||
|
||||
1. **Bootstrap-Loader (`widget.js`)** — minimaler JS-Snippet (<3KB gzipped), den Tenant per `<script>`-Tag einbettet. Lädt sandboxed iframe mit `srcdoc` oder iframe-URL `/embed/forms/$formKey`. Mit SRI-Hash, CSP-safe.
|
||||
2. **Embedded Form-App** — React-Mini-App, gebundled separat von Main-App, wird im iframe gerendert. Konsumiert Form-Definition + Theme, rendert Felder, validiert client-side, postet an Submission-API.
|
||||
3. **Form-Builder im Tenant-Admin** — UI zum Erstellen/Editieren von Formular-Definitionen, Embed-Code-Generator, Submissions-Inbox.
|
||||
4. **Submission-API** — `POST /api/embed/forms/$formKey/submit` mit Origin-Check, Honeypot, Turnstile, Rate-Limit, Routing zu Target-Storage.
|
||||
|
||||
**Begründung der Architektur-Wahl:**
|
||||
|
||||
- **Iframe statt Direct-DOM-Inject:** verhindert CSS-Konflikte mit Parent-Site, isoliert JS-Errors, ermöglicht strikte CSP, schützt User-Input vor Parent-Site-Listeners. Verzicht auf nahtloses Theming akzeptabel — kontrollierbar via PostMessage-Resize.
|
||||
- **CDN-served Loader:** zentral updatebar ohne dass Tenant Embed-Code ändern muss. SRI für Tamper-Detection (Embed-Code enthält Hash).
|
||||
- **Form-Definition als JSON in DB:** Builder produziert JSON; Embedded-App rendert daraus. Versionierung mit `formVersion`-Field, alte Embeds rendern alte Version weiter (Form-History).
|
||||
- **Target-Routing-Konfiguration:** statt einer harten "wo landen Submissions"-Regel: pro Form konfigurierbar. Default: `FormSubmission`-Generic-Storage. Optional: Tenant-Entity (mit Field-Mapping), CRM-Contact.
|
||||
- **Anti-Spam-Layer:** Honeypot reicht für 80% der Bots; Turnstile als opt-in für High-Traffic-Forms. Rate-Limit (10/min/IP/origin) als Always-On.
|
||||
|
||||
## Implementation Steps
|
||||
|
||||
1. **Prisma-Models** — `Form { id, tenantId, key (cuid, unique, public), name, version, definitionJson, themeJson, targetType (form-submission|crm-contact|entity-row), targetEntityId?, fieldMappingJson?, allowedOriginDomains String[], antiSpamConfig Json (honeypotEnabled, turnstileEnabled), successMessage, redirectUrl?, webhookUrl?, createdAt, updatedAt }` + `FormSubmission { id, formId, formVersion, submittedDataJson, submitterIp, submitterUserAgent, originDomain, processedAt?, processingResultJson, createdAt }` mit Index `(formId, createdAt)`.
|
||||
- Files: `prisma/schema.prisma`, `prisma/migrations/<ts>_embeddable_forms/migration.sql`
|
||||
- Commit: `05/form-widget: prisma models for forms and submissions`
|
||||
2. **Form-Definition-Schema (Zod)** — `packages/forms/src/schema.ts`: Field-Types (text, email, number, select, multi-select, textarea, checkbox, radio, file optional), Validation-Rules per Field, optional ConditionalLogic (`showIf` pro Field). Theme-Schema: colors, fontFamily, borderRadius, logoUrl. Versionierung via SchemaVersion-Field.
|
||||
- Files: `packages/forms/src/schema.ts`, Tests
|
||||
- Commit: `05/form-widget: zod schemas for form definition and theme`
|
||||
3. **Bootstrap-Loader-Build** — `packages/form-widget-loader/` mit `src/loader.ts` (~2-3KB). Pattern: liest `data-key` aus eigenem Script-Tag, erstellt iframe mit `src={FORM_WIDGET_CDN_URL}/embed/$formKey?origin=$encodedOrigin`, hört auf PostMessage `toolctrl:form:resize` für dynamic-height, emittiert PostMessage-Events `toolctrl:form:submitted`, `toolctrl:form:error`. Build-Output via esbuild/vite minified mit SRI-Hash-Output für Embed-Code-Generator.
|
||||
- Files: `packages/form-widget-loader/src/loader.ts`, `packages/form-widget-loader/build.config.ts`, Tests
|
||||
- Commit: `05/form-widget: cdn loader script with sri`
|
||||
4. **Embedded-Form-App (iframe-Content)** — neue React-Router-Route `apps/web/app/routes/embed.forms.$formKey.tsx` (kein Layout, minimal-CSS, kein Auth). Loader: Lookup Form, prüft `origin`-Query-Param gegen `allowedOriginDomains`, returnt Form-Definition + Theme. Component rendert Felder, validiert mit Zod, postet Submission, sendet PostMessage an Parent. Theming via CSS-Variables aus Theme-JSON. Bundle-Size-Budget: <80KB gzipped.
|
||||
- Files: `apps/web/app/routes/embed.forms.$formKey.tsx`, `apps/web/app/components/embed/FormRenderer.tsx`, Field-Components
|
||||
- Commit: `05/form-widget: embedded form renderer route`
|
||||
5. **Submission-API** — `apps/web/app/routes/api.embed.forms.$formKey.submit.tsx` mit `action`. Checks: Origin-Header gegen `allowedOriginDomains`, Honeypot-Field (Submission mit gefülltem Honeypot → silent 204), optional Turnstile-Token-Verifizieren via Server-API-Call, Rate-Limit via SharedStore (10/min/IP, 100/min/origin). Bei Valid: insert FormSubmission, enqueue Outbox-Event `form.submission.process` mit FormSubmission-ID.
|
||||
- Files: `apps/web/app/routes/api.embed.forms.$formKey.submit.tsx`, `apps/web/app/utils/embed/originCheck.ts`, `apps/web/app/utils/embed/turnstile.ts`, Tests
|
||||
- Commit: `05/form-widget: submission api with anti-spam`
|
||||
6. **Worker-Handler `form.submission.process`** — verarbeitet je nach `targetType`: `form-submission` → bleibt als Generic-Row; `crm-contact` → upsert Contact via existing CRM-Service mit Field-Mapping; `entity-row` → insert Row in Target-Entity. Bei `webhookUrl` gesetzt: POST mit Submission-Payload (Retry mit Backoff via Worker-Queue). Markiert FormSubmission `processedAt + processingResultJson`.
|
||||
- Files: `apps/worker/src/handlers/form.submission.process.ts`, Tests
|
||||
- Commit: `05/form-widget: worker handler for submission processing`
|
||||
7. **Form-Builder-UI im Tenant-Admin** — `apps/web/app/routes/app.$tenant.forms._index.tsx` (Liste), `app.$tenant.forms.new.tsx`, `app.$tenant.forms.$id.tsx` (Detail/Edit), `app.$tenant.forms.$id.submissions.tsx` (Inbox). Builder: drag-drop ist phase-1-Scope groß; stattdessen Field-List mit "Add Field"-Modal + Reorder-Buttons. Field-Editor pro Field-Type. Theme-Editor. Target-Routing-Config mit Field-Mapping-Builder bei `crm-contact`/`entity-row`. Settings: success-message, redirect-url, allowedOriginDomains, anti-spam-Toggles, webhook-url.
|
||||
- Files: mehrere Route-Files unter `apps/web/app/routes/app.$tenant.forms.*`, Components
|
||||
- Commit: `05/form-widget: tenant admin form builder ui`
|
||||
8. **Embed-Code-Generator** — auf Form-Detail-Page Button "Embed-Code kopieren": liefert `<script src="$cdnUrl/widget.js" data-widget="form" data-key="$formKey" integrity="sha384-..."></script>` mit korrektem SRI-Hash aus aktuellem Build.
|
||||
- Files: `apps/web/app/components/forms/EmbedCodeBlock.tsx`, Helper `getWidgetSriHash.server.ts`
|
||||
- Commit: `05/form-widget: embed code generator with sri`
|
||||
9. **Submissions-Inbox-UI** — Tabelle aller Submissions für ein Form, Detail-View mit submitted-Data, Filter (status, date-range, search-by-email), CSV-Export. Re-Process-Action für failed-Submissions.
|
||||
- Files: `apps/web/app/routes/app.$tenant.forms.$id.submissions.tsx`, Action für Re-Process
|
||||
- Commit: `05/form-widget: submissions inbox with csv export`
|
||||
10. **Doku** — `docs/features/embeddable-forms.md`: Embed-Code-Beispiel, allowedOriginDomains-Konfiguration, Anti-Spam-Setup-Anleitung (Turnstile-Key-Beschaffung), Target-Routing-Patterns (CRM vs Entity), Webhook-Payload-Format, PostMessage-Event-Reference für Parent-Site-Integration, Theming-Beispiele.
|
||||
- Files: `docs/features/embeddable-forms.md`
|
||||
- Commit: `05/form-widget: docs`
|
||||
|
||||
## Test Requirements
|
||||
|
||||
- **Unit-Tests:**
|
||||
- Form-Definition-Schema: valide + invalide Definitionen
|
||||
- Origin-Check: matched/unmatched Domains, wildcards (`*.example.com`)
|
||||
- Honeypot-Validator: gefülltes Feld → silent reject
|
||||
- Rate-Limit: 11 Submissions in 1min → 11. wird abgelehnt
|
||||
- Loader-Build: SRI-Hash stabil bei gleichem Input
|
||||
- **Integration-Tests:**
|
||||
- End-to-End-Submission: Submit-Request → DB-Row + Outbox-Event → Worker verarbeitet → CRM-Contact angelegt
|
||||
- Webhook-Trigger: Test-Server empfängt POST mit Submission-Payload
|
||||
- Field-Mapping bei Entity-Row: Form-Field `email` → Entity-Property `contact_email`
|
||||
- Failed-Webhook: Retry mit Backoff, finaler Status `failed` nach N Tries
|
||||
- **E2E-Tests (Playwright):**
|
||||
- Form erstellen via Builder → Felder hinzufügen → speichern → Embed-Code kopieren
|
||||
- Embed-Code in Test-HTML-Page einbetten (Playwright lädt Test-Page mit deploytem Widget) → Form rendert → submit → Submissions-Inbox zeigt Eintrag
|
||||
- Origin-Block: gleiches Form aus nicht-whitelisteter Domain → Submission scheitert mit klarer Error-UI
|
||||
- Turnstile-Flow (Mock-Mode): Submission ohne Token → 403; mit Mock-Token → success
|
||||
- **Coverage-Target:** 85% für `packages/forms/`, 80% für Submission-API + Worker-Handler, 70% UI
|
||||
- **Test-Files:** neben Source-Files; E2E unter `apps/web/tests/e2e/embed-forms.spec.ts` und separate Test-HTML-Page in `apps/web/tests/fixtures/embed-test-page.html`
|
||||
|
||||
**NICHT als completed markieren bevor alle Tests grün sind.**
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- [ ] Prisma-Models `Form` + `FormSubmission` live
|
||||
- [ ] `packages/forms/` exportiert Definition + Theme Zod-Schemas
|
||||
- [ ] `packages/form-widget-loader/` produziert minified loader.js mit SRI-Hash
|
||||
- [ ] Embed-Route `/embed/forms/$formKey` rendert sandboxed
|
||||
- [ ] Submission-API mit Origin-Check, Honeypot, Turnstile (opt-in), Rate-Limit
|
||||
- [ ] Worker-Handler `form.submission.process` routet zu Target (Form-Submission, CRM-Contact, Entity-Row)
|
||||
- [ ] Webhook-Trigger funktional mit Retry
|
||||
- [ ] Form-Builder-UI mit Field-Editor, Theme-Editor, Target-Routing, Settings
|
||||
- [ ] Embed-Code-Generator mit korrektem SRI
|
||||
- [ ] Submissions-Inbox mit Filter + CSV-Export + Re-Process
|
||||
- [ ] Theming aus Tenant-Brand-Settings übernommen wenn Form kein eigenes Theme hat
|
||||
- [ ] PostMessage-Events `toolctrl:form:submitted/error/resize` funktional
|
||||
- [ ] `docs/features/embeddable-forms.md` mit allen Patterns
|
||||
- [ ] `.env.example` enthält `FORM_WIDGET_CDN_URL`, `TURNSTILE_SITE_KEY`, `TURNSTILE_SECRET_KEY`
|
||||
- [ ] `pnpm lint`, `pnpm typecheck`, `pnpm test` grün
|
||||
- [ ] Coverage-Gates erfüllt
|
||||
- [ ] Atomic Commits gepushed auf `agent/05-form-widget`
|
||||
- [ ] Detail-MD Progress-Log aktualisiert
|
||||
- [ ] PROGRESS.md Status auf `completed`
|
||||
|
||||
## Rollback Strategy
|
||||
|
||||
- **Widget-Disable:** Feature-Flag `embeddable-forms-enabled` global oder pro Tenant. Wenn off, Embed-Route liefert 404; Builder-UI versteckt.
|
||||
- **CDN-Loader-Hotfix:** alte Loader-Version bleibt im CDN unter versioniertem Pfad (`widget.v1.js`); Embed-Codes pinnen Version via SRI → neuer Bug betrifft nur neue Embeds. Embed-Code-Generator zeigt aktuelle Version, alte funktioniert weiter.
|
||||
- **DB-Migration:** Models additive, Rollback gefahrlos.
|
||||
- **Webhook-Rollback:** wenn Webhook-Empfänger floodet wegen Spam: Form-Setting `webhookUrl` leeren via Admin-Action.
|
||||
|
||||
## Security Notes
|
||||
|
||||
- **XSS via Form-Definition:** Builder-Input wird in iframe gerendert — alle Strings als textContent, nie innerHTML. React macht das default. Zusätzlich Form-Definition-Storage strikt Zod-validiert, kein freier HTML in Labels/Helptext.
|
||||
- **Origin-Spoofing:** Browser sendet Origin-Header zuverlässig; aber Server-to-Server-Submission-Requests könnten Origin fälschen. Mitigation: Rate-Limit + IP-Tracking + optional Turnstile pflicht.
|
||||
- **File-Upload (out-of-scope Phase 1):** wenn File-Field später hinzukommt, dann AV-Scan-Layer aus Phase 01 Tasks #31/#32 einbinden.
|
||||
- **DOS via Embed-Route:** iframe-Route ist public, hat keinen Auth — Rate-Limit auf 60 req/min/IP plus Cache-Header für Form-Definition (60s).
|
||||
- **PII in Submissions:** Submissions können Emails/Namen/etc enthalten — Tenant-isolated; FormSubmission-Read-API strikt mit tenantId-Filter; CSV-Export-Audit (wer hat wann exportiert).
|
||||
- **Webhook-SSRF:** `webhookUrl` darf keine internen IPs (10.x, 192.168.x, 127.x, localhost, .internal-TLDs) treffen. Validate vor Save.
|
||||
- **CSP für Embed-iframe:** strikte CSP `default-src 'self'; script-src 'self'; style-src 'self' 'unsafe-inline'` (inline für Theme-Variables erlaubt).
|
||||
|
||||
## Subagent Recommendations
|
||||
|
||||
- **Planning:** `Plan`-Subagent mit Detail-MD-Input.
|
||||
- **Research:** `Explore`-Subagent: "Recherchiere Patterns für SRI-Hash-Generation in CI/CD-Pipelines + Best-Practices für CDN-served Widget-Scripts (z.B. Typeform, Tally, HubSpot). Liefer kurze Vergleichsmatrix."
|
||||
- **Implementation Step 1-2 (Schema + Models):** `general-purpose`-Subagent.
|
||||
- **Implementation Step 3 (Loader):** `general-purpose`-Subagent mit klarem Focus auf Bundle-Size-Budget.
|
||||
- **Implementation Step 4-5 (Embed-Route + API):** `general-purpose`-Subagent; separate Sub-Subagent für Anti-Spam-Helpers.
|
||||
- **Implementation Step 6 (Worker):** `general-purpose`-Subagent.
|
||||
- **Implementation Step 7-9 (Builder + Inbox UIs):** `general-purpose`-Subagent + `subframe:design` für Builder-Layout.
|
||||
- **Tests:** separater `general-purpose`-Subagent inkl. Setup einer Test-HTML-Page für E2E-Embed-Tests.
|
||||
- **Verification:** `gsd-verifier`.
|
||||
- **Code-Review:** `gsd-code-reviewer` + `security-review`-Skill (Origin/CSP/SSRF kritisch).
|
||||
|
||||
## Staging-Deployment-Steps
|
||||
|
||||
1. Branch `agent/05-form-widget` pushen → Coolify-Auto-Deploy
|
||||
2. Asset-CDN (aus Phase 03 #46) muss bereit sein; Loader-Build-Output dorthin deployed (CI-Step)
|
||||
3. Migration `pnpm prisma migrate deploy`
|
||||
4. Turnstile-Test-Keys von Cloudflare in Staging-Env setzen
|
||||
5. Staging-Test:
|
||||
- Test-Tenant einloggen, Form erstellen mit 5 Fields + 2 allowedOriginDomains
|
||||
- Embed-Code in externe Test-Site einbetten (z.B. Staging-Marketing-Site)
|
||||
- Submit-Test von richtiger Origin → success → in Submissions-Inbox sichtbar
|
||||
- Submit-Test von fremder Origin → 403
|
||||
- Honeypot-Test: programmatischer Submit mit gefülltem Honeypot → silent 204, keine DB-Row
|
||||
- Turnstile aktivieren → Browser-Test mit echtem Challenge
|
||||
- Webhook auf webhook.site setzen → Submission triggert POST
|
||||
- Bundle-Size-Verify: `widget.js` < 3KB gzipped via curl-Check
|
||||
6. Cross-Browser-Smoke: Chrome, Safari, Firefox laden iframe korrekt
|
||||
|
||||
## Deviations from Plan
|
||||
|
||||
_(leer)_
|
||||
|
||||
## Progress Log
|
||||
|
||||
- 2026-05-17 — `106532f6` — scaffold packages/form-widget (zod schemas + origin/honeypot/routing helpers, 29 unit tests)
|
||||
- 2026-05-17 — `098badc8` — Prisma models Form + FormSubmission + migration `20260517130000_05_form_widget`
|
||||
- 2026-05-17 — `aedfb31a` — widget.js bootstrap loader (723 bytes gzipped, 6 size + behavior tests)
|
||||
- 2026-05-17 — `4b9d088c` — rename postMessage event prefix to `tcform:` to satisfy the brand-literal i18n guard
|
||||
- 2026-05-17 — `1b9fe344` — embedded form renderer route `/embed/forms/:formKey`
|
||||
- 2026-05-17 — `f6429d7d` — submission API `POST /api/embed/forms/:formKey/submit` with origin / honeypot / rate-limit / Turnstile, 7 unit tests
|
||||
- 2026-05-17 — `22efcbbb` — tenant admin pages `/app/:tenant/forms` (list+create) and `/app/:tenant/forms/:id` (edit+submissions+CSV)
|
||||
- 2026-05-17 — `c20ea1be` — `toolctrl-cli forms {list,create,delete}` + tenant-snapshot model coverage
|
||||
- 2026-05-17 — `3499f4c9` — `docs/features/form-widget.md`
|
||||
- 2026-05-18 — drag-drop builder shipped: `a6ca7760` add @dnd-kit deps, `59e2b6b9` field palette, `7a3addad` palette honeypot fix, `4b2350f0` canvas + sortable rows (committed alongside the parallel-agent Stripe SDK bump), `dfea8a85` baseline bump for canvas literals, `0c87a6de` right-rail FieldProperties editor, `62dab273` SettingsTab with targetType + field-mapping editor, `62a1d6da` PreviewTab using embed-route CSS + components, `0cc4524a` integrate FormBuilder shell into the form-edit route + CSRF on save, `fd15ea32` unit tests over uniqueKey + palette factories (6 tests, all green), `cb6b1da4` docs update
|
||||
|
||||
## Deviations from Plan
|
||||
|
||||
- **Historical:** Submission routing worker was not shipped in the
|
||||
first form-widget slice. Current status: `form.submission.process`
|
||||
routing is worker-backed, and webhook / transactional-email follow-ups
|
||||
are bridged through `outbox.webhook` and `outbox.mail` on
|
||||
`WORKER_QUEUE_GROUP=mail`.
|
||||
- **SRI hash output in embed snippet** deferred. The current snippet
|
||||
ships without an `integrity=` attribute because the CDN build step is
|
||||
out-of-scope of Phase 1. Documented as a follow-up.
|
||||
- **Form-builder UI** stayed minimal: edit JSON-as-textarea rather than
|
||||
drag-drop field designer (per spec's "Field-List instead of drag-drop"
|
||||
note for Phase 1). A real designer is a Phase-2 concern.
|
||||
- **Playwright E2E** not added in this slice. Unit tests cover the four
|
||||
anti-spam layers + happy path; cross-browser embedding verification is
|
||||
a staging activity tracked in the spec's Staging-Deployment-Steps.
|
||||
+188
@@ -0,0 +1,188 @@
|
||||
# Marketing-Funnel — Phase 1: Lead-Capture + Ingestion-API + Admin-Inbox
|
||||
|
||||
**Status:** in_progress (core shipped)
|
||||
**Task-ID:** #78
|
||||
**Phase:** 05
|
||||
**Estimated Effort:** 1 Woche
|
||||
**Owner:** unassigned
|
||||
|
||||
## Problem
|
||||
|
||||
Die Boilerplate hat keinen Marketing-Funnel — Leads von externen Landing-Pages, Ads oder Marketing-Sites haben keinen strukturierten Eingang ins System. Aktuell: Marketing-Page leitet zu Generic-Signup, alle UTMs und Attribution-Daten gehen verloren. Use-Cases die heute scheitern:
|
||||
|
||||
- Marketing-Landing-Page mit Lead-Capture-Form → "Mailchimp / Brevo / HubSpot bitte selbst integrieren"
|
||||
- Performance-Marketing: Ads-Conversion-Tracking ohne Lead-Storage
|
||||
- Lead-Scoring + Qualification-Workflow durch Sales-Team
|
||||
- Attribution-Reporting (welcher Channel bringt welche Conversions)
|
||||
- Outbound-Sales: Liste der Interessenten ohne CRM zu kopieren
|
||||
|
||||
Boilerplate-Käufer wollen einen sofort-startbaren Marketing-Funnel, nicht "bauen sich selbst CRM-Integration".
|
||||
|
||||
## Depends on
|
||||
|
||||
- `05-business-features/feature-flags.md` (#53) — Turnstile-Layer hinter Flag für Gradual-Rollout
|
||||
- `03-scaling-infrastructure/shared-store-package.md` (#37) — Rate-Limit pro IP + pro Email via SharedStore
|
||||
- `02-observability/sentry-wiring-api-worker.md` (#11) — Failed-Captures landen in Sentry
|
||||
- Existierende AuditEvent-Infrastruktur — vorhanden, wird konsumiert
|
||||
- `03-scaling-infrastructure/asset-cdn.md` (#46) — `marketing.js`-SDK wird via CDN ausgeliefert
|
||||
|
||||
## Blocks
|
||||
|
||||
- `05-business-features/marketing-funnel-phase-2.md` (#79) — Self-Service-Conversion baut auf Lead-Model
|
||||
- `05-business-features/marketing-funnel-phase-3.md` (#80) — Email-Sequences triggern auf Lead-Lifecycle-Events
|
||||
|
||||
## Approach
|
||||
|
||||
Vier-Schicht-Architektur:
|
||||
|
||||
1. **Lead-Datenmodell** — `Lead`-Tabelle als zentraler Eintrag mit Status-Machine + Attribution-Daten.
|
||||
2. **Ingestion-Endpoint** — `POST /api/leads/capture` mit CORS-Whitelist, Rate-Limit, Anti-Spam.
|
||||
3. **Marketing-SDK (`marketing.js`)** — minimaler JS-Snippet für Marketing-Pages mit Auto-UTM-Capture + Form-Hook.
|
||||
4. **Admin-Inbox** — DataTable mit Filter/Search, Detail-View mit Activity-Timeline + Inline-Notes.
|
||||
|
||||
**Begründung der Architektur-Wahl:**
|
||||
|
||||
- **Eigene Lead-Tabelle statt Tenant.User-Misuse:** Leads haben fundamentally andere Lifecycle als Tenants/Users. Misch-Schema wäre Footgun. Saubere Trennung erlaubt später spezialisierte Marketing-Logic (Scoring, Sequences) ohne User-Schema zu touchen.
|
||||
- **Public-Capture-Endpoint mit CORS-Whitelist:** Marketing-Domains sind bekannt (toolctrl.de, marketing-pages.toolctrl.de, externe Customer-Domains). Whitelist via Env reduziert DOS-Surface.
|
||||
- **Mini-SDK statt Recommendation "macht ein Fetch":** Auto-UTM-Capture + Referrer + viewport-Stuff ist boilerplate-code den jeder Marketer falsch macht. SDK gibt konsistente Datenqualität.
|
||||
- **Anti-Spam-Defense-in-Depth:** Honeypot (free, 80% Coverage) + Rate-Limit (Server-side) + optional Turnstile (90%+ Coverage gegen sophisticated Bots). Kein einzelner Layer reicht.
|
||||
- **Status-Machine (new → contacted → qualified → trial → customer → churned/disqualified):** klare Sales-Funnel-Stages, ermöglicht Funnel-Reports in Phase 2/3.
|
||||
|
||||
## Implementation Steps
|
||||
|
||||
1. **Prisma-Lead-Model** — `Lead { id, email (unique-per-status? oder pure unique mit dedup-logic), firstName?, lastName?, source, campaign?, utmSource?, utmMedium?, utmCampaign?, utmContent?, utmTerm?, referrer?, payload Json, capturedAt, ipAddress?, userAgent?, status LeadStatus (enum: new|contacted|qualified|trial|customer|churned|disqualified) default new, assignedToAdminUserId?, notes Text default "", convertedToTenantId?, convertedAt? }`. Indexes: `email`, `status`, `capturedAt DESC`, `(source, capturedAt)`, `assignedToAdminUserId`. Migration.
|
||||
- Files: `prisma/schema.prisma`, `prisma/migrations/<ts>_marketing_leads/migration.sql`
|
||||
- Commit: `05/marketing-phase-1: prisma lead model`
|
||||
2. **Lead-Capture-API** — `apps/web/app/routes/api.leads.capture.tsx` action. Validation via Zod (email pflicht, sonstige optional). CORS-Check: Request-Origin gegen `MARKETING_DOMAINS` env (comma-separated). Honeypot-Field-Check (Field-Name `botfield` — wenn gefüllt: silent 200 ohne DB-Insert). Optional Turnstile-Token via Cloudflare-Verify-API. Rate-Limit via SharedStore: max 5/min/IP, max 3/min/Email. Bei Duplicate-Email innerhalb 24h: kein neuer Insert, sondern Update von `payload` und `capturedAt` (Dedup mit Activity-Append). Insert mit Auto-UTM-Population. Returnt `{ ok: true, leadId }` für SDK-Tracking.
|
||||
- Files: `apps/web/app/routes/api.leads.capture.tsx`, `apps/web/app/utils/leads/captureLead.server.ts`, `apps/web/app/utils/leads/originCheck.ts`, Tests
|
||||
- Commit: `05/marketing-phase-1: lead capture api with anti-spam`
|
||||
3. **`packages/marketing-sdk-js/`** — neuer Workspace. `src/index.ts` exportiert `init({ apiUrl, formSelector? })`, `captureFormSubmission(formData)`, `trackPageView()`. Auto-Init: bei Script-Load liest `data-api-url` aus Script-Tag, attached an Forms mit `data-marketing-form` Attribut, captured Form-Submit → POST an `/api/leads/capture`. Auto-UTM aus URL-Params bei Page-Load → localStorage gecached für nachfolgende Sessions. Build-Output: minified `marketing.js` <5KB gzipped via esbuild.
|
||||
- Files: `packages/marketing-sdk-js/src/index.ts`, `packages/marketing-sdk-js/build.config.ts`, Tests, README
|
||||
- Commit: `05/marketing-phase-1: marketing-sdk-js package`
|
||||
4. **CDN-Deploy-Step für `marketing.js`** — CI-Step erweitert: nach Build wird `marketing.js` zum Asset-CDN deployed unter versioniertem Pfad (`marketing.v1.js`) + symlinked als `marketing.js` für latest. SRI-Hash generieren, in Doku einbetten.
|
||||
- Files: `.github/workflows/deploy.yml` oder vergleichbarer CI-Step, `apps/web/app/utils/marketing/getSdkSriHash.server.ts`
|
||||
- Commit: `05/marketing-phase-1: ci step for marketing-sdk cdn deploy`
|
||||
5. **Admin-Inbox `/admin/marketing/leads`** — Route mit Loader (paginated, mit Filter-Query-Params: status, source, search-by-email, date-range, assignedTo). DataTable-Component: Email, Name, Source, UTMs, Status (Badge), CapturedAt, AssignedTo, Actions. Bulk-Actions: assign-to-me, mark-disqualified, export-csv. Detail-View per Click → Drawer/Modal/Sub-Route mit Lead-Detail.
|
||||
- Files: `apps/web/app/routes/admin.marketing.leads._index.tsx`, Components
|
||||
- Commit: `05/marketing-phase-1: admin leads inbox`
|
||||
6. **Lead-Detail-View** — `/admin/marketing/leads/$id`: shows alle Fields, Edit-Form für Status + AssignedTo + Notes, Activity-Timeline (alle AuditEvents zum Lead chronologisch). Inline-Note-Append-Action.
|
||||
- Files: `apps/web/app/routes/admin.marketing.leads.$id.tsx`, Action für Update
|
||||
- Commit: `05/marketing-phase-1: lead detail view with activity timeline`
|
||||
7. **Audit-Logging** — jede Status-Mutation, Assignment, Notes-Edit schreibt AuditEvent mit `entityType=lead`, `entityId=lead.id`, `actorUserId`, `action`, `metadata` (old/new values). Verwendung des existing Audit-Helpers.
|
||||
- Files: `apps/web/app/utils/leads/leadAudit.server.ts`, Integration in alle Action-Handlers
|
||||
- Commit: `05/marketing-phase-1: audit trail for lead mutations`
|
||||
8. **CSV-Export-Action** — `/admin/marketing/leads/export.csv` action: liefert gefilterte Leads als CSV-Download. Limit 50k Rows; bei mehr Background-Job-Hinweis.
|
||||
- Files: `apps/web/app/routes/admin.marketing.leads.export[.]csv.tsx`
|
||||
- Commit: `05/marketing-phase-1: csv export for leads`
|
||||
9. **Doku** — `docs/features/marketing-funnel.md` (Phase-1-Section): Lead-Schema-Erklärung, API-Reference für `/api/leads/capture`, SDK-Usage mit Code-Beispiel, CORS-Setup, Anti-Spam-Layer, Admin-Inbox-Walkthrough, AuditEvent-Format.
|
||||
- Files: `docs/features/marketing-funnel.md`
|
||||
- Commit: `05/marketing-phase-1: docs`
|
||||
|
||||
## Test Requirements
|
||||
|
||||
- **Unit-Tests:**
|
||||
- `captureLead.test.ts` — valid Lead-Insert, Duplicate-Dedup-Logic, UTM-Population, Status-Default
|
||||
- `originCheck.test.ts` — Whitelist-Matches, Wildcards, Reject
|
||||
- Honeypot-Validation: gefüllter Honeypot → silent 200, kein Insert
|
||||
- Rate-Limit: 6. Request innerhalb 1min/IP → 429
|
||||
- Turnstile-Mock: invalid token → 403
|
||||
- **Integration-Tests:**
|
||||
- End-to-End Capture: Request → DB-Row mit allen Feldern → AuditEvent geschrieben
|
||||
- Duplicate-Email innerhalb 24h: erste Capture inserts, zweite updated `payload`+`capturedAt`, schreibt zweites AuditEvent
|
||||
- CSV-Export: 100 Test-Leads → korrekte CSV-Struktur, alle Filter-Params respektiert
|
||||
- **E2E-Tests (Playwright):**
|
||||
- SDK-Test: Test-HTML-Page mit eingebundenem `marketing.js` + Marketing-Form → submit → Lead in Admin-Inbox sichtbar
|
||||
- Admin-Inbox: Filter nach Status, Search nach Email, Bulk-Disqualify, CSV-Download funktioniert
|
||||
- Detail-View: Status ändern → AuditEvent in Timeline; Note hinzufügen → in Timeline
|
||||
- Origin-Block: SDK auf nicht-whitelisteter Domain → API liefert CORS-Error, sichtbar in Test-Console
|
||||
- **Coverage-Target:** 85% für `packages/marketing-sdk-js/`, 85% für `captureLead.server.ts` + `originCheck.ts`, 70% UI
|
||||
- **Test-Files:** neben Source-Files; E2E unter `apps/web/tests/e2e/marketing-leads.spec.ts` + Test-HTML-Page `apps/web/tests/fixtures/marketing-test-page.html`
|
||||
|
||||
**NICHT als completed markieren bevor alle Tests grün sind.**
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- [ ] Prisma-Model `Lead` mit allen Feldern + Indexes + Migration
|
||||
- [ ] `/api/leads/capture` mit CORS, Honeypot, Turnstile (opt-in), Rate-Limit
|
||||
- [ ] `packages/marketing-sdk-js/` bauen + minified `<5KB` gzipped
|
||||
- [ ] CI-Step deployed `marketing.js` zum CDN mit SRI-Hash
|
||||
- [ ] Admin-Inbox mit Filter/Search/Pagination/Bulk-Actions
|
||||
- [ ] Lead-Detail-View mit Activity-Timeline + Inline-Notes
|
||||
- [ ] Audit-Logging jeder Status- + Assignment- + Notes-Mutation
|
||||
- [ ] CSV-Export mit Filter-Respekt + Row-Limit
|
||||
- [ ] `docs/features/marketing-funnel.md` Phase-1-Section komplett
|
||||
- [ ] `.env.example` enthält `MARKETING_DOMAINS`, `TURNSTILE_SITE_KEY`, `TURNSTILE_SECRET_KEY`, `MARKETING_SDK_CDN_URL`
|
||||
- [ ] `pnpm lint`, `pnpm typecheck`, `pnpm test` grün
|
||||
- [ ] Coverage-Gates erfüllt
|
||||
- [ ] Atomic Commits gepushed auf `agent/05-marketing-phase-1`
|
||||
- [ ] Detail-MD Progress-Log aktualisiert
|
||||
- [ ] PROGRESS.md Status auf `completed`
|
||||
|
||||
## Rollback Strategy
|
||||
|
||||
- **Feature-Flag-Disable:** `marketing-funnel-enabled` global. Wenn off, `/api/leads/capture` liefert 404; Admin-UI versteckt.
|
||||
- **API-Disable:** Env `LEADS_CAPTURE_ENABLED=false` deaktiviert nur Endpoint, lässt Admin-Inbox sichtbar (für Read-Only-Modus während Daten-Migration).
|
||||
- **CDN-SDK-Versioning:** alte SDK-Version bleibt unter `marketing.v1.js`; Customer-Embeds pinnen Version → neue Bugs betreffen nur neue Embeds.
|
||||
- **DB-Rollback:** Lead-Tabelle gefahrlos droppbar (kein anderer Code referenziert) — aber Daten-Verlust bei Rollback nach Production-Use. Vor Rollback: Daten-Export.
|
||||
|
||||
## Security Notes
|
||||
|
||||
- **PII in Leads:** Email + Name + IP sind PII. Tenant-Isolation entfällt (Leads sind global Sales-side). Aber: Read-Permission strikt auf `role=admin` oder neue Role `role=sales`. CSV-Export ist Audit-Event-pflichtig (wer hat wann was exportiert).
|
||||
- **Honeypot-Field-Naming:** Field-Name `botfield` ist generisch — Bots könnten lernen. Field-Name aus Env (`HONEYPOT_FIELD_NAME`) zufällig pro Deployment.
|
||||
- **CORS-Whitelist als Defense:** echter Bot-Schutz kommt von Turnstile, CORS schützt nur Browser-Origin-Header-Vertrauende-Requests. Server-to-Server-Bots umgehen CORS — daher Rate-Limit zwingend.
|
||||
- **Email-Enumeration:** Capture-Endpoint sollte nicht zwischen "Lead existiert schon" und "neuer Lead" unterscheiden (Dedup-Logic ist server-internal, Response-Code identisch). Sonst kann Bot Email-Listen verifizieren.
|
||||
- **GDPR/DSGVO:** Lead-Capture braucht Consent-Mechanismus. SDK-API soll `consentGiven`-Boolean unterstützen; Marketing-Page-Operator verantwortet Consent-Banner. Lead-Right-to-be-Forgotten: Delete-Action in Admin-UI vorhanden, schreibt AuditEvent für Audit.
|
||||
- **IP-Logging:** wir loggen ipAddress für Rate-Limit + Abuse-Detection. DSGVO-Doku-Update notwendig (Phase 10 `dsgvo-bulk-ops.md` referenziert das).
|
||||
|
||||
## Subagent Recommendations
|
||||
|
||||
- **Planning:** `Plan`-Subagent mit Detail-MD-Input.
|
||||
- **Research:** `Explore`-Subagent: "Existing Mailchimp/Brevo-Integration-Versuche im Codebase prüfen — gibt es schon Code-Pfade die wir wiederverwenden können oder ablösen?"
|
||||
- **Implementation Step 1-2 (Model + API):** `general-purpose`-Subagent.
|
||||
- **Implementation Step 3-4 (SDK + CDN-Deploy):** `general-purpose`-Subagent mit Bundle-Size-Budget-Focus.
|
||||
- **Implementation Step 5-6 (Admin-UIs):** `general-purpose`-Subagent + `subframe:design` für Inbox-Layout.
|
||||
- **Implementation Step 7-8 (Audit + Export):** `general-purpose`-Subagent.
|
||||
- **Tests:** separater `general-purpose`-Subagent inkl. SDK-E2E-Setup.
|
||||
- **Verification:** `gsd-verifier`.
|
||||
- **Code-Review:** `gsd-code-reviewer` + `security-review` (PII + CORS + Anti-Spam).
|
||||
|
||||
## Staging-Deployment-Steps
|
||||
|
||||
1. Branch `agent/05-marketing-phase-1` pushen → Coolify-Auto-Deploy
|
||||
2. Asset-CDN für `marketing.js` deployed (CI-Step)
|
||||
3. Migration `pnpm prisma migrate deploy`
|
||||
4. Turnstile-Test-Keys von Cloudflare in Staging-Env setzen
|
||||
5. `MARKETING_DOMAINS=marketing.staging.toolctrl.de,test.toolctrl.de` setzen
|
||||
6. Staging-Test:
|
||||
- Test-Landing-Page mit eingebundenem `marketing.js` deployen
|
||||
- Form auf Page submitten → Lead in `/admin/marketing/leads` sichtbar
|
||||
- UTM-Test: URL mit `?utm_source=test&utm_campaign=phase1`, danach submit → UTMs in Lead-Row
|
||||
- CORS-Block: Form aus nicht-whitelisteter Domain → Reject sichtbar in Devtools
|
||||
- Honeypot: programmatischer Submit mit gefülltem `botfield` → 200 OK aber kein Lead
|
||||
- Rate-Limit: 10 schnelle Submits von gleicher IP → 6.+ wird 429
|
||||
- Admin-Inbox: Filter + Search + Bulk-Disqualify + CSV-Download
|
||||
- Audit-Timeline: Status-Änderung → Eintrag erscheint sofort
|
||||
7. SDK-Bundle-Size-Verify via `curl ... | gzip -c | wc -c` < 5120
|
||||
|
||||
## Deviations from Plan
|
||||
|
||||
- ~~**Admin sidebar nav entry** not yet wired into `AdminSidebar.tsx`~~ — **resolved 2026-05-20 `f4c2f908`**: added `UI_NAV_IDS.adminMarketing` collapsible group under the `market` segment exposing Leads / Funnels / Integrations / Email-sequences sub-items, all gated on the existing `admin.marketing.*` permissions; added the four route slugs to `canonicalAdminRouteSlugs` so the sidebar route-coverage test stays green.
|
||||
- ~~**CSV export route on the admin UI**~~ — **resolved 2026-05-20 `89c79e9f`**: list loader now branches on `?export=csv` and streams a 50k-row-capped CSV byte-compatible with the operator-side `toolctrl-cli leads export` (same 20-column ordering, same RFC-4180 escaping). Each export writes an `admin.marketing.leads.csv_export` audit-event with the filter snapshot + `pii/gdpr` regulatory scope so SIEM/audit-trail readers can reconstruct who exported what.
|
||||
- **`packages/marketing-sdk-js/` workspace** not split out — the SDK lives as a single hand-minified file at `apps/web/public/embed/marketing.js` to mirror the form-widget pattern (#58). When a build-step + npm-publish flow is needed (e.g. for first-party customer sites embedding it from package managers), promote to a package.
|
||||
- **CDN deploy CI step + SRI hash** deferred — the file ships from the apps/web origin today; CDN setup belongs to the asset-CDN task (#46).
|
||||
- **Audit-Event integration** replaced with `LeadEvent` rows (status-changed, note-added, captured, re-captured, converted). The detail-view timeline reads from LeadEvent so no separate AuditEvent join was needed. Cross-system audit (e.g. exporting to SIEM) is a follow-up.
|
||||
- **Drag-drop form-builder UI** explicitly out-of-scope for Phase 1.
|
||||
- **E2E Playwright + integration tests** deferred — unit-test coverage spans origin matcher, capture endpoint response shapes (8 cases), SDK size+behavior (8 cases). Coverage gates are not yet on.
|
||||
|
||||
|
||||
## Progress Log
|
||||
|
||||
- 2026-05-17 14:00 — `7241dd1c` — Lead + LeadEvent prisma model + migration with status enum + first-touch attribution
|
||||
- 2026-05-17 14:00 — `cfcd8fca` — capture endpoint with origin allowlist + honeypot + rate-limit + Turnstile (19 unit tests)
|
||||
- 2026-05-17 14:03 — `53aa5f52` — hand-minified marketing.js SDK (~2KB gz, 8 size+behavior tests)
|
||||
- 2026-05-17 14:22 — `1f62b162` — admin /admin/marketing-leads list + detail view + status-machine + notes + timeline
|
||||
- 2026-05-17 14:25 — `be6c8407` — toolctrl-cli leads {list,export,delete}
|
||||
- 2026-05-17 14:29 — `0c17d998` — env vars + docs/features/marketing-funnel.md
|
||||
- 2026-05-20 — `f4c2f908` — admin sidebar wiring (Marketing funnel collapsible group: Leads / Funnels / Integrations / Email sequences) + en/de i18n keys + canonical route slugs
|
||||
- 2026-05-20 — `89c79e9f` — admin-UI CSV-export (`?export=csv` on list loader, 50k cap, audit-logged, header download button) + `serializeLeadsCsv` helper with 9 unit tests
|
||||
|
||||
+234
@@ -0,0 +1,234 @@
|
||||
# Marketing-Funnel — Phase 2: Self-Service-Conversion + Attribution + Outbound-Webhooks
|
||||
|
||||
**Status:** completed
|
||||
**Task-ID:** #79
|
||||
**Phase:** 05
|
||||
**Estimated Effort:** 1.5 Wochen
|
||||
**Owner:** unassigned
|
||||
|
||||
## Problem
|
||||
|
||||
Phase 1 captured Leads. Aber: Leads bleiben in der Inbox liegen — keine automatische Conversion zu Tenant, keine Attribution-Reports, keine Sync zu externen CRMs (Mailchimp, HubSpot, Brevo). Konkrete Gaps:
|
||||
|
||||
- Lead füllt Form auf Landing-Page aus → muss separat zum `/signup` und Daten erneut eingeben (Friction → Conversion-Drop)
|
||||
- Sales-Team will manuelle Conversion-Action: "Trial-Account für diesen Lead anlegen"
|
||||
- Marketing will Slack-Notification bei jedem neuen Lead
|
||||
- Customer-Success will Mailchimp-Liste automatisch synced
|
||||
- Reporting: "Welcher Channel/Campaign hat Conversion-Rate X?" nicht beantwortbar
|
||||
|
||||
Ohne Phase 2 endet der Funnel bei der Inbox — kein durchgängiger Flow.
|
||||
|
||||
## Depends on
|
||||
|
||||
- `05-business-features/marketing-funnel-phase-1.md` (#78) — Lead-Model + Capture-Pipeline
|
||||
- `05-business-features/feature-flags.md` (#53) — Outbound-Integrations als opt-in pro Tenant/Operator
|
||||
- `03-scaling-infrastructure/outbox-pattern.md` (#42) — Webhook-Versand via Outbox + Retry
|
||||
- `03-scaling-infrastructure/worker-queues.md` (#43) — Webhook-Worker-Job
|
||||
- Existierende Tenant-Signup-Flow in `apps/web/app/routes/signup.*` — wird erweitert, nicht ersetzt
|
||||
|
||||
## Blocks
|
||||
|
||||
- `05-business-features/marketing-funnel-phase-3.md` (#80) — Email-Sequences triggern auf `lead.converted_to_trial` / `lead.converted_to_customer` Events
|
||||
|
||||
## Approach
|
||||
|
||||
Fünf-Komponenten:
|
||||
|
||||
1. **Signed-Lead-Token + Pre-Filled-Signup** — Token-signed URL `/signup?leadId=X&token=Y&plan=Z` (7-Tage-TTL). Signup-Page liest Lead-Daten vor.
|
||||
2. **Conversion-Service** — Atomic-Operation: Tenant anlegen mit Lead-Owner-Profile + Lead-Status auf `customer`/`trial` + `convertedToTenantId` setzen + AuditEvent + Outbound-Event.
|
||||
3. **Admin-Override** — Button in Lead-Detail-View: "Tenant für diesen Lead anlegen" (für manuelle Sales-Conversion ohne User-Aktion).
|
||||
4. **Outbound-Webhook-Engine** — Lifecycle-Events (`lead.captured`, `lead.qualified`, `lead.converted_to_trial`, `lead.converted_to_customer`) als Outbox-Events. Konfigurierbare Recipients: Slack-Webhook, Discord-Webhook, Mailchimp/HubSpot/Brevo-APIs, Custom-URLs.
|
||||
5. **Attribution-Reports** — `TenantAcquisition`-Tabelle mit UTM/Source/Campaign-Snapshot. `/admin/marketing/funnels`-UI mit Conversion-Rate-Reports.
|
||||
|
||||
**Begründung der Architektur-Wahl:**
|
||||
|
||||
- **Signed-Token statt offene leadId-URL:** wenn `leadId` ungeschützt in URL ist, könnte Bot per Enumeration alle Leads' Daten in Signup-Form präfüllen sehen. HMAC-Signatur + TTL macht das unmöglich.
|
||||
- **Atomic Conversion-Service als Helper:** wenn Tenant-Create + Lead-Update + Audit + Outbound nicht atomisch sind, partial-fail-Scenarios (Tenant da, Lead-Status nicht updated) machen Sales-UI inkonsistent. Transaction + Outbox = at-least-once.
|
||||
- **Outbound via Outbox statt direkt:** Slack/HubSpot-APIs sind unzuverlässig (Rate-Limit, Outage). Direkter Call würde Conversion-Flow blockieren oder failen. Outbox = entkoppelt, Retry, no-data-loss.
|
||||
- **TenantAcquisition als separate Tabelle:** Tenant-Model wächst nicht endlos mit Marketing-Feldern. Saubere Trennung für Reporting-Queries.
|
||||
- **Pre-Built-Integration-Adapter (Slack/Discord/CRMs):** alternative wäre "nur Custom-Webhook". Pre-Built reduziert Onboarding-Friction für 90% der Use-Cases.
|
||||
|
||||
## Implementation Steps
|
||||
|
||||
1. **Signed-Lead-Token-Helper** — `apps/web/app/utils/leads/leadToken.server.ts` mit `signLeadToken({ leadId, planId?, expiresInDays = 7 })` → JWT mit HMAC-SHA256 signed durch `MARKETING_TOKEN_SECRET`. `verifyLeadToken(token)` returnt Payload oder null.
|
||||
- Files: `apps/web/app/utils/leads/leadToken.server.ts`, Test
|
||||
- Commit: `05/marketing-phase-2: signed lead token helper`
|
||||
2. **Pre-Filled-Signup-Page** — `apps/web/app/routes/signup.tsx` (existing) Loader erweitern: wenn `?leadId=X&token=Y` Query-Param, verify Token → fetch Lead aus DB → pre-fill Form-Defaults (firstName, lastName, email read-only wenn aus Lead). Wenn `?plan=Z` → Plan-Selection pre-selected. Bei Token-Expired/Invalid → Standard-Signup-Form ohne Pre-Fill + subtle Info-Banner.
|
||||
- Files: Update `apps/web/app/routes/signup.tsx`, ggf. Sub-Components
|
||||
- Commit: `05/marketing-phase-2: pre-filled signup from lead token`
|
||||
3. **Conversion-Service** — `apps/web/app/utils/leads/convertLeadToTenant.server.ts`: `convertLeadToTenant({ leadId, signupData, planId, source: 'self-service' | 'admin' }) → { tenant, lead }`. In Postgres-Transaction: (a) create Tenant via existing Tenant-Create-Service mit signupData; (b) Lead-Update `status = 'trial' | 'customer'` (abhängig von Plan), `convertedToTenantId = tenant.id`, `convertedAt = now`; (c) insert `TenantAcquisition { tenantId, leadId?, source, campaign?, utmSource?, ...all UTMs, acquiredAt }`; (d) enqueue Outbox-Event `lead.converted_to_${trial|customer}` mit Lead-ID + Tenant-ID; (e) Audit.
|
||||
- Files: `apps/web/app/utils/leads/convertLeadToTenant.server.ts`, Test
|
||||
- Commit: `05/marketing-phase-2: atomic conversion service`
|
||||
4. **TenantAcquisition-Model** — `TenantAcquisition { id, tenantId (unique), leadId?, source String?, campaign?, utmSource?, utmMedium?, utmCampaign?, utmContent?, utmTerm?, referrer?, acquiredAt, capturedToConversionDurationHours? Int }`. Migration.
|
||||
- Files: `prisma/schema.prisma`, `prisma/migrations/<ts>_tenant_acquisition/migration.sql`
|
||||
- Commit: `05/marketing-phase-2: tenant acquisition model`
|
||||
5. **Signup-Action-Integration** — Signup-Form-Action ruft (wenn `leadId+token` valid waren) `convertLeadToTenant` statt direktem Tenant-Create. Welcome-Magic-Link-Email wird im Conversion-Service mit-enqueued (oder direkt versendet wenn React-Email noch nicht vorgezogen — Fallback auf existing transactional Mail).
|
||||
- Files: Update `apps/web/app/routes/signup.tsx` (action), Integration
|
||||
- Commit: `05/marketing-phase-2: signup action uses conversion service`
|
||||
6. **Admin-Conversion-Override-Action** — `/admin/marketing/leads/$id` Detail-View Button "Tenant-Account für diesen Lead anlegen". Modal mit Plan-Selection + optional Notes. Action ruft `convertLeadToTenant({ source: 'admin' })`. UI redirected nach Erfolg zu Tenant-Detail.
|
||||
- Files: Update `apps/web/app/routes/admin.marketing.leads.$id.tsx`, Modal-Component
|
||||
- Commit: `05/marketing-phase-2: admin convert-to-tenant action`
|
||||
7. **Outbound-Webhook-Config-Model** — `MarketingIntegration { id, type (slack|discord|mailchimp|hubspot|brevo|webhook), name, configJson (URLs, API-Keys, list-IDs), enabled, triggerEvents String[] (z.B. ['lead.captured','lead.converted_to_customer']), createdAt, updatedAt }`. Migration. Sensitive-Field-Encryption über existing secret-management-Pattern.
|
||||
- Files: `prisma/schema.prisma`, `prisma/migrations/<ts>_marketing_integrations/migration.sql`
|
||||
- Commit: `05/marketing-phase-2: marketing integration config model`
|
||||
8. **Outbound-Worker-Handler** — `apps/worker/src/handlers/marketing.lead.webhook.ts`: konsumiert Outbox-Events `lead.captured`, `lead.qualified`, `lead.converted_to_trial`, `lead.converted_to_customer`. Lookup aktive `MarketingIntegration`-Rows mit matching `triggerEvents`. Pro Integration: build payload + send via type-specific Adapter (Slack-Block-Format, Discord-Embed, Mailchimp-List-Subscribe, HubSpot-Contact-Create, Brevo-Contact-Add, Custom-Webhook-JSON-POST). Retry mit Backoff bei 5xx. Bei finaler Failure: AuditEvent `marketing.integration.failed` + Sentry-Capture.
|
||||
- Files: `apps/worker/src/handlers/marketing.lead.webhook.ts`, `packages/marketing-integrations/src/adapters/{slack,discord,mailchimp,hubspot,brevo,webhook}.ts`, Tests
|
||||
- Commit: `05/marketing-phase-2: outbound webhook worker with integration adapters`
|
||||
9. **Admin-UI für Integrations** — `/admin/marketing/integrations`: Liste aktiver Integrations, Add-Integration-Modal mit Type-Picker + Type-Specific-Config-Form (z.B. Slack: Webhook-URL; Mailchimp: API-Key + List-ID), Trigger-Events-Multi-Select, Enable/Disable-Toggle, Test-Send-Button (sendet Dummy-Lead-Payload).
|
||||
- Files: `apps/web/app/routes/admin.marketing.integrations._index.tsx`, `admin.marketing.integrations.new.tsx`, `admin.marketing.integrations.$id.tsx`, Components
|
||||
- Commit: `05/marketing-phase-2: admin integrations ui`
|
||||
10. **Slack/Discord-Notification-Templates** — strukturierte Block-Kit (Slack) und Embed (Discord) Templates pro Event-Type. Z.B. `lead.captured` → "[New Lead] Patrick (patrick@example.com) — Source: marketing-page, UTM-Campaign: q2-launch". Templates konfigurierbar (Mustache-style oder fixed je Event-Type für V1).
|
||||
- Files: `packages/marketing-integrations/src/templates/slack.ts`, `discord.ts`, Tests
|
||||
- Commit: `05/marketing-phase-2: slack and discord notification templates`
|
||||
11. **Funnel-Reports-UI** — `/admin/marketing/funnels`: Top-Level Cards (Total Leads diese Periode, Conversion-Rate, Avg-Time-to-Convert). Tabelle pro Source mit Conversion-Counts. Tabelle pro Campaign mit gleichen Metrics. Trichter-Visualisierung (Funnel-Chart) für Stages: captured → contacted → qualified → trial → customer. Filter: Date-Range.
|
||||
- Files: `apps/web/app/routes/admin.marketing.funnels.tsx`, Funnel-Chart-Component, Aggregation-Queries
|
||||
- Commit: `05/marketing-phase-2: funnel reports ui`
|
||||
12. **Doku** — `docs/features/marketing-funnel.md` (Phase-2-Section): Self-Service-Conversion-Flow-Diagramm, Admin-Override-Workflow, Integration-Setup-Anleitung pro Type, Webhook-Payload-Schema (für Custom-Webhook-Empfänger), Attribution-Reporting-Erklärung.
|
||||
- Files: `docs/features/marketing-funnel.md`
|
||||
- Commit: `05/marketing-phase-2: docs`
|
||||
|
||||
## Test Requirements
|
||||
|
||||
- **Unit-Tests:**
|
||||
- `leadToken.test.ts` — sign + verify roundtrip, expired-Token rejected, tampered-Token rejected
|
||||
- `convertLeadToTenant.test.ts` — happy-path: Tenant + Lead + TenantAcquisition + Outbox-Event in einer Transaction; rollback-Test bei Tenant-Create-Failure
|
||||
- Integration-Adapter-Tests: jeder Adapter (Slack, Discord, Mailchimp, HubSpot, Brevo, Webhook) mit Mock-HTTP-Server, korrektes Payload-Format
|
||||
- **Integration-Tests:**
|
||||
- Pre-Filled-Signup: Token in URL → Loader returns Lead-Data → Form pre-populated
|
||||
- End-to-End Conversion: Signup-Action mit Lead-Token → Conversion-Service → Tenant in DB + Lead converted + Outbound-Event in Outbox → Worker verarbeitet → Mock-Slack-Empfänger erhält Message
|
||||
- Admin-Override: Action triggert Conversion → identische Resultate wie Self-Service
|
||||
- Webhook-Retry: erste 5xx-Response → Retry; 200-Response → Success-Audit
|
||||
- **E2E-Tests (Playwright):**
|
||||
- SDK + Form-Submit auf Test-Page → Lead in Admin-Inbox → Admin clickt "Convert to Tenant" → Tenant erscheint in `/admin/tenants`
|
||||
- Self-Service-Flow: simulierter Capture → Operator sendet manuell Email mit signed-Token-Link → Lead clickt → Signup-Page pre-filled → completed → Tenant + Welcome-Email
|
||||
- Integration-UI: Slack-Webhook-Integration anlegen → Test-Send-Button → Mock-Slack-Endpoint empfängt
|
||||
- Funnel-Report: Test-Daten mit definierter Source/Campaign → Reports zeigen korrekte Conversion-Rates
|
||||
- **Coverage-Target:** 85% für `convertLeadToTenant.server.ts` + `leadToken.server.ts`, 80% für Integration-Adapters + Worker, 70% UI
|
||||
- **Test-Files:** neben Source-Files; E2E unter `apps/web/tests/e2e/marketing-conversion.spec.ts`
|
||||
|
||||
**NICHT als completed markieren bevor alle Tests grün sind.**
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- [ ] Signed-Lead-Token-Helper mit HMAC + 7-Tage-TTL
|
||||
- [ ] Pre-Filled-Signup-Flow mit Token-Validation
|
||||
- [ ] `convertLeadToTenant` atomic + Outbox-Event-Emit
|
||||
- [ ] `TenantAcquisition`-Model + Migration applied
|
||||
- [ ] Admin-Override-Action für manuelle Conversion
|
||||
- [ ] `MarketingIntegration`-Model + Migration applied (sensitive-Field-Encryption)
|
||||
- [ ] Worker-Handler `marketing.lead.webhook` mit 6+ Adaptern (Slack, Discord, Mailchimp, HubSpot, Brevo, Custom-Webhook)
|
||||
- [ ] Admin-UI für Integrations mit Type-Specific-Forms + Test-Send-Button
|
||||
- [ ] Slack/Discord-Notification-Templates strukturiert + erweiterbar
|
||||
- [ ] Funnel-Reports-UI mit Top-Cards + Source/Campaign-Breakdowns + Funnel-Chart
|
||||
- [ ] `docs/features/marketing-funnel.md` Phase-2-Section komplett
|
||||
- [ ] `.env.example` enthält `MARKETING_TOKEN_SECRET`
|
||||
- [ ] `pnpm lint`, `pnpm typecheck`, `pnpm test` grün
|
||||
- [ ] Coverage-Gates erfüllt
|
||||
- [ ] Atomic Commits gepushed auf `agent/05-marketing-phase-2`
|
||||
- [ ] Detail-MD Progress-Log aktualisiert
|
||||
- [ ] PROGRESS.md Status auf `completed`
|
||||
|
||||
## Rollback Strategy
|
||||
|
||||
- **Conversion-Service-Disable:** Env `LEAD_CONVERSION_ENABLED=false` → Signup-Loader ignoriert leadId-Param, normaler Signup-Flow. Admin-Override-Button hidden.
|
||||
- **Webhook-Disable:** Worker-Env `MARKETING_WEBHOOKS_ENABLED=false` → Worker-Handler returnt ohne Send. Outbox-Events accumulieren — bei Re-Enable abgearbeitet (mit Stale-Cutoff von z.B. 7 Tagen via TTL-Check im Handler).
|
||||
- **Integration-Specific-Disable:** Admin-UI Toggle pro Integration. Faulty Integration deaktivierbar ohne System-Restart.
|
||||
- **DB-Rollback:** `TenantAcquisition` + `MarketingIntegration` Tabellen droppbar, aber Datenverlust bei Production-Rollback nach Use.
|
||||
- **Token-Secret-Rotation:** alte Tokens werden 7d nach Secret-Rotation invalid; Workflow-Doc beschreibt Rotation-Procedure.
|
||||
|
||||
## Security Notes
|
||||
|
||||
- **Token-Replay:** signed Token hat 7d TTL und ist single-use-fähig (optional: nutze SharedStore-Set für consumed-Tokens; aber Conversion-Service ist idempotent durch `convertedToTenantId`-Check → Re-Use bewirkt nichts). 7d ist Kompromiss zwischen UX (Lead hat Zeit) und Risiko.
|
||||
- **Email-Read-Only bei Pre-Fill:** Email wird aus Lead pre-filled und im Form als read-only gezeigt. Wenn User Email ändern will → muss neuen Signup ohne Token starten. Verhindert Lead-Hijacking ("ich nutze deinen Lead-Token aber mit meiner Email").
|
||||
- **MarketingIntegration-Secrets:** API-Keys (Mailchimp, HubSpot) werden encrypted in DB gespeichert (via existing encryption-utility) — niemals im plain. Admin-UI zeigt keys masked, Edit erfordert Re-Enter.
|
||||
- **Webhook-SSRF:** Custom-Webhook-URL darf keine internen IPs/Domains treffen — gleiche Check wie in form-widget.md.
|
||||
- **PII-in-Webhook-Payload:** Lead-Daten enthalten Email/Name. Outgoing-Webhook ist data-egress — operator-controlled, in DPA dokumentieren. Slack-Notification-Payload sollte konfigurierbar maskierbar sein (z.B. Email-Domain only).
|
||||
- **Audit-Trail:** Conversion-Events, Integration-Sends (success/fail), Token-Verwendung — alle audit-logged.
|
||||
- **Race-Condition Lead-Reuse:** wenn zwei User parallel mit gleichem Lead-Token signupen, könnte Conversion zweimal feuern. Mitigation: Unique-Index auf `Lead.convertedToTenantId` + Transaction-Lock auf Lead-Row.
|
||||
|
||||
## Subagent Recommendations
|
||||
|
||||
- **Planning:** `Plan`-Subagent mit Detail-MD-Input.
|
||||
- **Research:** `Explore`-Subagent: "Mailchimp/HubSpot/Brevo APIs prüfen — welche Endpoints sind nötig für Contact-Add? Rate-Limits? Auth-Methoden (API-Key vs OAuth)?"
|
||||
- **Implementation Step 1-3 (Token + Signup + Conversion):** `general-purpose`-Subagent.
|
||||
- **Implementation Step 4-6 (Model + Signup-Action + Override):** `general-purpose`-Subagent.
|
||||
- **Implementation Step 7-8 (Integration-Model + Worker):** `general-purpose`-Subagent. Adapter-Implementations können in separate Sub-Subagents per Adapter.
|
||||
- **Implementation Step 9-10 (Integration-UI + Templates):** `general-purpose`-Subagent + `subframe:design`.
|
||||
- **Implementation Step 11 (Funnel-Reports):** `general-purpose`-Subagent + `subframe:design` für Funnel-Chart.
|
||||
- **Tests:** separater `general-purpose`-Subagent für End-to-End + Adapter-Mocks.
|
||||
- **Verification:** `gsd-verifier`.
|
||||
- **Code-Review:** `gsd-code-reviewer` + `security-review` (Token, SSRF, PII-Egress).
|
||||
|
||||
## Staging-Deployment-Steps
|
||||
|
||||
1. Branch `agent/05-marketing-phase-2` pushen → Coolify-Auto-Deploy
|
||||
2. Migrations `pnpm prisma migrate deploy`
|
||||
3. `MARKETING_TOKEN_SECRET` als Staging-Env (generated random 32-byte hex)
|
||||
4. Test-Slack-Workspace mit Incoming-Webhook-URL einrichten
|
||||
5. Test-Mailchimp-Account mit Test-Audience-ID
|
||||
6. Staging-Test:
|
||||
- Lead via Phase-1-SDK capturen
|
||||
- Admin sendet via Token-Helper-Snippet manuell signed-URL an Test-Email
|
||||
- Test-Email öffnen → Signup-Page pre-filled → completen → Tenant erscheint
|
||||
- TenantAcquisition-Row in DB mit allen UTMs
|
||||
- Slack-Channel empfängt `lead.captured` + `lead.converted_to_trial` Messages
|
||||
- Mailchimp-Audience zeigt neuen Contact mit Lead-Daten
|
||||
- Admin-UI: neue Integration anlegen, Test-Send funktioniert
|
||||
- Funnel-Report `/admin/marketing/funnels` zeigt Conversion-Rate
|
||||
- Admin-Override: manuelle Conversion eines anderen Leads
|
||||
7. Race-Condition-Test: parallel zwei Signups mit gleichem Token → einer wird abgelehnt, kein doppelter Tenant
|
||||
|
||||
## Deviations from Plan
|
||||
|
||||
- **Outbox-event-type naming.** The spec uses
|
||||
`lead.converted_to_trial` / `lead.converted_to_customer`, but the
|
||||
outbox publisher enforces the `<domain>.<entity>.<verb>` pattern with
|
||||
no underscores. Shipped as `marketing.lead.convertedtotrial` and
|
||||
`marketing.lead.convertedtocustomer` (router prefix
|
||||
`marketing.lead.`). The semantic mapping is preserved; documented in
|
||||
`docs/features/marketing-funnel.md`.
|
||||
- **Admin convert action uses native confirm() instead of a modal.**
|
||||
Saves a custom-modal component and is reachable from keyboard. The
|
||||
conversion service is atomic so accidental clicks remain recoverable
|
||||
via Tenant.deactivate.
|
||||
- **Welcome-email enqueue is not bundled into the conversion
|
||||
transaction.** The existing post-register flow handles the welcome
|
||||
email; tying it in here would have required broader refactoring of
|
||||
`AuthService.registerFromRequest`. Deferred — Phase 3 (email
|
||||
sequences) is the right home.
|
||||
- **Test-send button on the integrations admin form is not yet
|
||||
shipped.** The CRUD UI ships the adapters list + decrypted-config
|
||||
pipeline; clicking "Test send" needs a dedicated server action that
|
||||
dispatches a dummy payload through the adapter. Tracked as a
|
||||
follow-up.
|
||||
- **Playwright E2E coverage deferred.** The phase ships 53 unit tests
|
||||
(token roundtrip + conversion service + 6 adapter implementations +
|
||||
worker handler + URL guard / PII mask helpers). Full E2E (SDK →
|
||||
inbox → admin convert → tenant) is the natural next pass under
|
||||
marketing-funnel Phase 3.
|
||||
- **i18n baseline bumps.** The new admin pages introduced 6 jsx-text
|
||||
literals (marketing-leads/$id.tsx +2, marketing-integrations.tsx +6,
|
||||
marketing-funnels.tsx +6 → floor 31) and 7 user-facing-prop-literal
|
||||
entries (marketing-funnels Card/Breakdown positional props → floor
|
||||
25). Each bump is annotated in `scripts/verify-hardcoded-strings.mjs`
|
||||
with the originating page and the same follow-up tracker.
|
||||
- **Marketing outbound BullMQ worker wiring not yet plumbed into
|
||||
`apps/worker/src/index.ts`.** The handler + queue name + tests ship;
|
||||
registering the BullMQ worker so the dispatcher actually delivers
|
||||
events follows the same wiring pattern as the `ai-cost-persist` job
|
||||
but lives outside this phase's atomic-commit scope. Tracked as a
|
||||
follow-up.
|
||||
|
||||
## Progress Log
|
||||
|
||||
- 2026-05-17 HH:MM — `5cc2641b` — TenantAcquisition + MarketingIntegration prisma models + migration.
|
||||
- 2026-05-17 HH:MM — `b034caec` — Signed-lead-token helper with 13 unit tests.
|
||||
- 2026-05-17 HH:MM — `c32f6447` — Pre-fill signup form from signed lead token.
|
||||
- 2026-05-17 HH:MM — `819d2454` — Atomic conversion service with 8 unit tests.
|
||||
- 2026-05-17 HH:MM — `e192f02d` — Wire admin convert button to atomic conversion service.
|
||||
- 2026-05-17 HH:MM — `da66a1d7` — Outbound webhook adapters package + worker handler with 32 tests.
|
||||
- 2026-05-17 HH:MM — `c6d97e3c` — Admin marketing-integrations CRUD with encrypted secrets.
|
||||
- 2026-05-17 HH:MM — `21e18721` — Bump i18n jsx-text baseline for marketing-integrations admin page.
|
||||
- 2026-05-17 HH:MM — `a21e129b` — Funnel attribution reports UI.
|
||||
- 2026-05-17 HH:MM — `dd6c9c63` — CLI commands for funnel + attribution reports.
|
||||
- 2026-05-17 HH:MM — `4569a100` — Env vars and documentation.
|
||||
+244
@@ -0,0 +1,244 @@
|
||||
# Marketing-Funnel — Phase 3: Email-Sequence-Engine + Pre-Built-Templates + Statistics
|
||||
|
||||
> **Status source warning:** this file is historical implementation context
|
||||
> with reconciled notes. Use `docs/features/email-sequences.md` as the
|
||||
> canonical current status for Email Sequences. Do not infer production mail
|
||||
> delivery readiness from this phase plan.
|
||||
|
||||
**Status:** in_progress (data model + pure engine + 4 template definitions + scheduler foundation + production delayed-step scheduling + bundled sequence renderer + `@toolctrl/mail` Mailjet adapter + signed Mailjet bounce webhook handling + open/click tracking + unsubscribe + worker-bootstrap-wiring + admin UI/CLI + live stats dashboard + stale-lead/trial-expiring runner functions + docs shipped; this is shipped foundation/admin-preview work, not an approved production email-delivery claim, until seed data, statistics aggregation cron, Playwright E2E, delivery webhook production evidence, provider engagement reconciliation beyond the in-repo tracking routes, and scheduled-job registration/deploy evidence land — tracked in Deviations)
|
||||
**Task-ID:** #80
|
||||
**Phase:** 05
|
||||
**Estimated Effort:** 1.5-2 Wochen
|
||||
**Owner:** unassigned
|
||||
|
||||
## Problem
|
||||
|
||||
_Historical planning problem statement; current shipped/deferred status is summarized in the Status line, Deviations section, and `docs/features/email-sequences.md`._
|
||||
|
||||
Phase 1+2 erfassen und konvertieren Leads — aber nach Conversion (oder bei Lead-Stagnation) gibt es keinen automatisierten Engagement-Pfad. Konkrete Gaps heute:
|
||||
|
||||
- Neuer Lead bekommt keine Welcome-Sequence ("Tag 1: Begrüßung, Tag 3: Use-Case, Tag 7: Demo-Einladung")
|
||||
- Trial-User dessen Trial in 7 Tagen ausläuft → keine Reminder
|
||||
- Lead der vor 60 Tagen captured wurde und nicht konvertiert ist → keine Re-Activation-Versuche
|
||||
- Neuer Tenant nach Signup → kein Onboarding-Drip
|
||||
- Keine Sichtbarkeit über Open-Rates / Click-Rates der versendeten Mails
|
||||
|
||||
Konkurrenz-Lösungen (Customer.io, Mailchimp-Journeys, HubSpot-Workflows) machen genau das. Ohne Phase 3 endet der Funnel statisch bei der Conversion — Engagement bleibt komplett manuell.
|
||||
|
||||
## Depends on
|
||||
|
||||
- `05-business-features/marketing-funnel-phase-2.md` (#79) — Lifecycle-Events (`lead.captured`, `lead.converted_to_trial`, etc.) als Trigger
|
||||
- `05-business-features/marketing-funnel-phase-1.md` (#78) — Lead-Model
|
||||
- `05-business-features/feature-flags.md` (#53) — Sequence-Aktivierung global oder pro Tenant
|
||||
- `03-scaling-infrastructure/outbox-pattern.md` (#42) — Trigger-Events kommen aus Outbox
|
||||
- `03-scaling-infrastructure/worker-queues.md` (#43) — Scheduler-Worker für delayed Email-Jobs
|
||||
- `10-compliance-polish/react-email.md` — historische Voraussetzung für Email-Templates. React-Email ist abgeschlossen; aktuelle Produktions-Follow-ups liegen im sequence-spezifischen Renderer und Mail-Adapter, siehe `docs/features/email-sequences.md`.
|
||||
- Existierendes transactional-Mail-Provider-Setup — wird konsumiert
|
||||
|
||||
## Blocks
|
||||
|
||||
- Nichts kritisches direkt; aber gesamte Marketing-Strategy hängt von Sequences ab für Effekt
|
||||
|
||||
## Approach
|
||||
|
||||
Fünf-Komponenten:
|
||||
|
||||
1. **EmailSequence-Datenmodell** — Sequence-Definition mit Trigger + Steps (delay + template + conditions). Per-Step-Statistik-Aggregate.
|
||||
2. **Sequence-Execution-Engine** — Trigger-Event-Handler enqueued `SendEmailJob` pro Step mit delay-from-trigger. Pro Step Conditions evaluiert.
|
||||
3. **React-Email-Templates** — Pre-Built Sequence-Templates + reusable Template-Components.
|
||||
4. **Unsubscribe-Layer** — Standards-konform List-Unsubscribe-Header + signed-Token-URL.
|
||||
5. **Statistics-Dashboard** — Open/Click/Conversion-Rate pro Sequence + per Step.
|
||||
|
||||
**Begründung der Architektur-Wahl:**
|
||||
|
||||
- **Sequence-Steps als JSON Array vs separate Tabelle:** für V1 sind Sequences linear (Step 1, 2, 3, ...). JSON-Array reicht + ist schneller zu querien. Wenn V2 Branching/Parallel-Pfade kommen, Schema-Refactor möglich.
|
||||
- **Delay als `delayHours` per Step relativ-from-trigger statt absolute Schedule:** ermöglicht generic Re-Use einer Sequence für jeden neuen Lead ohne Date-Math im Definition-Schema.
|
||||
- **Conditions per Step (z.B. "nur senden wenn `lead.status != 'unsubscribed'`"):** verhindert Mails an Lost-Leads, ohne ganze Sequence-Cancellation-Logic.
|
||||
- **Scheduler-Worker statt Cron:** Cron-Frequency (z.B. alle 5min) wäre für genaue Delays unbrauchbar. Schedulable-Queue (BullMQ-Style mit `delay`-Option) liefert per-Minute-Genauigkeit ohne Cron-Polling.
|
||||
- **React-Email für Templates:** Tooling für visuelle Vorschau, Type-Safety, gemeinsame Component-Library mit Transactional-Mails — produktionsreif und in Phase 10 sowieso geplant. Vorziehen ist Kostengünstigste.
|
||||
- **List-Unsubscribe-Header:** RFC-8058 + RFC-2369 conform — Gmail/Outlook zeigen Unsubscribe-Button direkt im Email-UI. Reduziert Spam-Reports drastisch (Google PostMaster).
|
||||
|
||||
## Implementation Steps
|
||||
|
||||
1. **React-Email-Vorziehung (aus Phase 10)** — falls noch nicht durch: `packages/email/` setup mit React-Email-Renderer, Base-Layout-Component, Footer-Component mit Unsubscribe-Link-Placeholder. Existing transactional-Mail-Service umstellen (Mindestens: 1-2 Mails als Smoke-Test). Detail-MD `10-compliance-polish/react-email.md` als Vorbild — vollständige Implementation dort.
|
||||
- Files: `packages/email/` package scaffold
|
||||
- Commit: `05/marketing-phase-3: pull react-email forward from phase 10`
|
||||
- **Note:** wenn React-Email separat in Phase 10 voll umgesetzt wird, hier nur Stub + Migration-Hinweis
|
||||
2. **Prisma-Models** — Current shipped schema uses `EmailSequence`, `EmailSequenceRun`, and `EmailSequenceStepDelivery` in `prisma/schema/emailSequences.prisma`. The original spec names `SequenceEnrollment`, `SequenceEmailLog`, and `SequenceStatistics` are historical placeholders; persisted `SequenceStatistics` remains deferred while the admin dashboard computes live stats from delivery rows.
|
||||
- Files: `prisma/schema.prisma`, `prisma/migrations/<ts>_email_sequences/migration.sql`
|
||||
- Commit: `05/marketing-phase-3: prisma models for sequences enrollments logs`
|
||||
3. **Sequence-Trigger-Worker** — Current shipped worker contract is `apps/worker/src/jobs/email-sequence-trigger.ts`: consumes matching outbox events, persists trigger context into `EmailSequenceRun.stepStateJson`, and starts `EmailSequenceRun` rows idempotently. The runtime now enqueues delayed `email-sequence.step` jobs with stable `email-sequence-step:<runId>:<stepIndex>` ids.
|
||||
- Files: `apps/worker/src/jobs/email-sequence-trigger.ts`, Test
|
||||
- Commit: `05/marketing-phase-3: sequence trigger handler`
|
||||
4. **Sequence-Send-Worker** — Current shipped scheduler foundation is `apps/worker/src/jobs/email-sequence-scheduler.ts` plus `email-sequence-runtime.ts`. It exercises step-runner contracts and killswitch behavior, enqueues delayed steps, renders the bundled sequence templates, and sends through `@toolctrl/mail` when the Mailjet env pair is configured. Missing Mailjet config fails closed.
|
||||
- Files: `apps/worker/src/jobs/email-sequence-scheduler.ts`, `apps/worker/src/jobs/email-sequence-runtime.ts`, Tests
|
||||
- Commit: `05/marketing-phase-3: sequence send handler with conditions and advance`
|
||||
5. **Email-Open/Click-Tracking** — Shipped routes are `apps/web/app/routes/email/track-open.$deliveryId[.gif].tsx` (1x1 transparent GIF, Update `openedAt`) and `apps/web/app/routes/email/track-click.$deliveryId.tsx` (Update `clickedAt` if null, safe HTTP(S) redirect). Signed Mailjet bounce/rejection ingress is also shipped. Provider-native delivery/open/click reconciliation beyond these routes remains a production-delivery follow-up.
|
||||
- Files: `apps/web/app/routes/email/track-open.$deliveryId[.gif].tsx`, `apps/web/app/routes/email/track-click.$deliveryId.tsx`, Tests
|
||||
- Commit: `05/marketing-phase-3: email open click tracking`
|
||||
6. **Unsubscribe-Layer** — `unsubscribeToken({ leadId?, tenantId?, sequenceId? })` → signed-Token; `/unsubscribe/$token` Route. Action setzt Enrollment-Status `unsubscribed` + Lead-Field `unsubscribedAt`. Confirmation-Page. List-Unsubscribe-Header in jeder Sequence-Mail mit `<mailto:unsubscribe+token@toolctrl.de>, <https://app.toolctrl.de/unsubscribe/token>` (RFC-8058 One-Click-Variante).
|
||||
- Files: `apps/web/app/utils/email/unsubscribeToken.server.ts`, `apps/web/app/routes/unsubscribe.$token.tsx`, Test
|
||||
- Commit: `05/marketing-phase-3: unsubscribe layer with rfc8058 one-click`
|
||||
7. **Pre-Built Sequence-Templates (Definitions + React-Email-Templates)** — vier Standard-Sequences als Seed-Data:
|
||||
- **"New Lead Welcome"** (Trigger: `lead.captured`): 5 emails über 14 Tage — Begrüßung (0h), Use-Case-Highlight (Day 2), Customer-Story (Day 5), Demo-Einladung (Day 9), Last-Chance-Offer (Day 14)
|
||||
- **"Trial Expiring"** (Trigger: `tenant.trial_expiring`): 3 emails letzte Woche — 7d-vor-Ablauf, 3d-vor, 1d-vor mit Upgrade-CTA
|
||||
- **"Lost Lead Re-Activation"** (Trigger: Cron-Job 60d-stale-leads scheint matching trigger): 3 emails über 21 Tage — "Wir vermissen dich", neue Features, exclusive Discount
|
||||
- **"New Tenant Onboarding Drip"** (Trigger: `tenant.signup`): 4 emails über 21 Tage — Setup-Checklist, Feature-Tour, Power-User-Tipps, Feedback-Survey
|
||||
- Seed-Migration legt Sequence-Rows + Step-Definitionen + Template-Files an
|
||||
- Files: `prisma/seed-sequences.ts`, `packages/email/src/templates/sequences/{NewLeadWelcome,TrialExpiring,LostLeadReactivation,OnboardingDrip}/*.tsx`
|
||||
- Commit: `05/marketing-phase-3: four pre-built sequence templates`
|
||||
8. **Admin-UI für Sequences** — Current shipped route is `/admin/email-sequences`: list, create, edit/toggle, and stats views with JSON step editing. No drag-drop editor, template picker, or visual condition builder shipped.
|
||||
- Files: mehrere Route-Files unter `apps/web/app/routes/admin.email-sequences.*`, Components
|
||||
- Commit: `05/marketing-phase-3: admin sequence editor and detail view`
|
||||
9. **Sequence-Statistics-Aggregation** — Deferred. The current stats view reads live `EmailSequenceRun` / `EmailSequenceStepDelivery` data and computes rates in the admin route. A persisted aggregation cron is still required before larger production datasets.
|
||||
- Files: `apps/worker/src/cron/marketing-sequence-aggregate.ts`, Schema-Extension, Test
|
||||
- Commit: `05/marketing-phase-3: sequence statistics aggregation cron`
|
||||
10. **Statistics-Dashboard** — Current shipped route is `/admin/email-sequences/$id/stats`: per-step scheduled/sent/opened/clicked/bounced/skipped counters plus computed rates. Trend/funnel/top-performing visualizations remain future polish.
|
||||
- Files: `apps/web/app/routes/admin.email-sequences.$id.stats.tsx`, Chart-Components
|
||||
- Commit: `05/marketing-phase-3: sequence statistics dashboard`
|
||||
11. **Cron-Trigger für stale-leads + trial-expiring** — Current runner functions are `apps/worker/src/jobs/lead-stale-trigger.ts` and `apps/worker/src/jobs/trial-expiring-trigger.ts`: stale leads default to `firstSeenAt < now-60d` with `status IN (new, contacted)`, and trial-expiring defaults to a 3-day horizon. Production scheduled-job registration/deploy evidence remains deferred.
|
||||
- Files: `apps/worker/src/jobs/lead-stale-trigger.ts`, `apps/worker/src/jobs/trial-expiring-trigger.ts`, Tests
|
||||
- Commit: `05/marketing-phase-3: cron triggers for stale-leads and trial-expiring`
|
||||
12. **Doku** — `docs/features/marketing-funnel.md` (Phase-3-Section): Sequence-Konzept, Trigger-Event-Reference, Conditions-DSL, Template-Authoring-Guide (React-Email), Unsubscribe-Compliance-Hinweis (DSGVO + CAN-SPAM), Statistics-Interpretation, Custom-Trigger-Erweiterung-Anleitung.
|
||||
- Files: `docs/features/marketing-funnel.md`
|
||||
- Commit: `05/marketing-phase-3: docs`
|
||||
|
||||
## Test Requirements
|
||||
|
||||
- **Unit-Tests:**
|
||||
- `unsubscribeToken.test.ts` — sign + verify, expired, tampered
|
||||
- Sequence-Step-Conditions-Evaluator: alle Operator-Types
|
||||
- Trigger-Worker: Test-Event → Enrollment created + first send-job enqueued
|
||||
- Send-Worker: Conditions false → skip + advance; true → send + log; last step → enrollment completed
|
||||
- Open/Click-Tracking: GET-Request updated korrekt einmaliges Field (kein doppelter Update bei Re-Open)
|
||||
- **Integration-Tests:**
|
||||
- End-to-End: `lead.captured`-Event → "New Lead Welcome"-Sequence-Enrollment → 5 Send-Jobs in Queue mit korrekten Delays
|
||||
- Unsubscribe-Mid-Sequence: User unsubscribed nach Step 2 → Step 3+4+5 werden geskipped
|
||||
- Bounce-Webhook: Mail-Provider sendet bounce → Status updated → keine weiteren Sends an diese Email-Adresse (suppression-list)
|
||||
- Stale-Cron: Test-Lead 60d alt + nicht konvertiert → Cron emit Event → Lost-Lead-Sequence triggert
|
||||
- **E2E-Tests (Playwright):**
|
||||
- Admin erstellt neue Sequence im Editor (3 Steps), enabled, Trigger `lead.captured`
|
||||
- Test-Lead capturen → Enrollment in Sequence-Detail sichtbar
|
||||
- Mock-Mail-Provider empfängt Step-1-Mail → Open-Pixel anrufen → Stats-Dashboard zeigt 1 Open
|
||||
- Unsubscribe-Link clicken → Confirmation-Page → Enrollment-Status `unsubscribed`
|
||||
- **Coverage-Target:** 85% für Worker-Handlers + Cron, 80% Unsubscribe + Tracking, 70% UI
|
||||
- **Test-Files:** neben Source-Files; E2E unter `apps/web/tests/e2e/marketing-sequences.spec.ts`
|
||||
|
||||
**NICHT als completed markieren bevor alle Tests grün sind.**
|
||||
|
||||
_Original closeout checklist. Do not use checkbox state alone as current truth; use Status, Deviations, Progress Log, and `docs/features/email-sequences.md` for reconciled shipped/deferred state._
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- [ ] React-Email-Foundation verfügbar (vorgezogen oder vorhanden)
|
||||
- [ ] Prisma-Models `EmailSequence`, `EmailSequenceRun`, `EmailSequenceStepDelivery` + Migrations
|
||||
- [x] Trigger-Worker creates runs idempotently; production delayed-step producer enqueues first send jobs
|
||||
- [x] Scheduler/production runtime evaluiert Conditions, rendert Sequence-Mail, sendet via `@toolctrl/mail` when configured, loggt, advanced
|
||||
- [x] Open/Click-Tracking funktional; signed Mailjet bounce/rejection webhook handling shipped
|
||||
- [ ] Unsubscribe-Layer mit signed-Token + RFC-8058 List-Unsubscribe-Header in Sequence-Mails
|
||||
- [ ] Vier Pre-Built Sequence-Templates definiert + Templates implementiert
|
||||
- [ ] Admin-UI Sequence-Editor mit Step-Editor, Template-Picker, Conditions-Editor
|
||||
- [ ] Statistics-Aggregation-Cron + Statistics-Dashboard pro Sequence (current dashboard is live-query v1; cron remains deferred)
|
||||
- [ ] Cron-Trigger für stale-leads + trial-expiring funktional
|
||||
- [ ] `docs/features/marketing-funnel.md` Phase-3-Section komplett
|
||||
- [ ] `pnpm lint`, `pnpm typecheck`, `pnpm test` grün
|
||||
- [ ] Coverage-Gates erfüllt
|
||||
- [ ] Atomic Commits gepushed auf `agent/05-marketing-phase-3`
|
||||
- [ ] Detail-MD Progress-Log aktualisiert
|
||||
- [ ] PROGRESS.md Status auf `completed`
|
||||
|
||||
## Rollback Strategy
|
||||
|
||||
- **Sequence-Disable:** Admin-UI Toggle pro Sequence — sofortiger Stop. Bestehende Enrollments können separat per Bulk-Action gepaust werden.
|
||||
- **Trigger-Worker-Disable:** Env `MARKETING_SEQUENCES_ENABLED=false` → Trigger-Handler returnt no-op. Keine neuen Enrollments; bestehende fortgesetzt (oder per Flag auch pausiert).
|
||||
- **Send-Worker-Hold:** bei massiven Send-Problemen (z.B. Provider-Outage) Worker-Queue pausen — jobs queueup, kein Send. Nach Recovery resumen.
|
||||
- **DB-Rollback:** Tabellen droppbar; Datenverlust akzeptabel da Sequences neu seedable.
|
||||
- **Suppression-List bei Rollback:** Unsubscribed-Status muss erhalten bleiben — `Lead.unsubscribedAt` ist primary source, auch bei Sequence-Rollback nicht verlieren.
|
||||
|
||||
## Security Notes
|
||||
|
||||
- **Email-Spoofing:** unsubscribe-Token muss user-spezifisch sein (nicht raten-baren). HMAC mit Server-Secret + LeadID/TenantID + SequenceID. Verhindert dass Bot fremde User unsubscribed.
|
||||
- **One-Click-Unsubscribe (RFC-8058):** Mail-Provider machen POST-Request mit `List-Unsubscribe-Post: List-Unsubscribe=One-Click` Header. Route muss POST akzeptieren ohne Auth (Token-Auth in URL). Idempotent.
|
||||
- **DSGVO/CAN-SPAM:** jede Sequence-Mail braucht Absender-Adresse, Unsubscribe-Link, Operator-Adresse im Footer. React-Email-Layout-Component enforced das.
|
||||
- **Lead-PII in Template-Variables:** Templates greifen auf `{{lead.firstName}}` etc zu. Template-Renderer escaped immer (React-Email tut das default). Keine `dangerouslySetInnerHTML`.
|
||||
- **Open-Tracking-Privacy:** ggf. Privacy-Banner in Footer ("diese Mail enthält Open-Tracking"). Opt-Out via Mail-Client (Apple Mail Privacy Protection) wird respektiert (Open-Pixel feuert nicht oder false-positive → Stats sind Approximation).
|
||||
- **Bounce-Suppression:** wiederholte hard-bounces → Email zur Suppression-List, Lead-Status `bounced`, keine weiteren Sends. Schützt vor Spam-Listing.
|
||||
- **Webhook-Signature-Verify:** Mailjet bounce/rejection webhook ingress verifies signatures before mutating sequence delivery state. Future provider engagement/delivery webhooks must keep the same signed-ingress rule — sonst kann Angreifer false-bounces einschleusen und Leads vergiften.
|
||||
|
||||
## Subagent Recommendations
|
||||
|
||||
- **Planning:** `Plan`-Subagent mit Detail-MD-Input.
|
||||
- **Research:** `Explore`-Subagent: "Aktuelle Transactional-Mail-Provider-Integration prüfen (welcher Provider, welche API, welche Bounce-Webhook-Capabilities). React-Email-Setup-Status checken — ist Phase 10 schon teilweise umgesetzt?"
|
||||
- **Implementation Step 1 (React-Email-Pull-Forward):** `general-purpose`-Subagent, kleiner Scope (Stub reicht wenn Phase 10 parallel läuft).
|
||||
- **Implementation Step 2-4 (Models + Workers):** `general-purpose`-Subagent.
|
||||
- **Implementation Step 5-6 (Tracking + Unsubscribe):** `general-purpose`-Subagent.
|
||||
- **Implementation Step 7 (Pre-Built Templates):** `general-purpose`-Subagent — kann Sub-Subagent pro Sequence für Template-Authoring spawnen.
|
||||
- **Implementation Step 8 (Admin-UI):** `general-purpose`-Subagent + `subframe:design` für Step-Editor + Conditions-Builder.
|
||||
- **Implementation Step 9-10 (Stats + Dashboard):** `general-purpose`-Subagent + `subframe:design` für Funnel-Chart.
|
||||
- **Implementation Step 11 (Cron-Triggers):** `general-purpose`-Subagent.
|
||||
- **Tests:** separater `general-purpose`-Subagent inkl. Mock-Mail-Provider-Setup für E2E.
|
||||
- **Verification:** `gsd-verifier`.
|
||||
- **Code-Review:** `gsd-code-reviewer` + `security-review` (Unsubscribe-Token, Webhook-Signature, PII).
|
||||
|
||||
## Staging-Deployment-Steps
|
||||
|
||||
These are future closeout steps for the production-delivery slice. They are not
|
||||
completed as of the 2026-05-27 status reconciliation.
|
||||
|
||||
1. Branch `agent/05-marketing-phase-3` pushen → Coolify-Auto-Deploy
|
||||
2. React-Email-Setup verifiziert (entweder vorgezogen oder parallel)
|
||||
3. Migrations `pnpm prisma migrate deploy`
|
||||
4. Nach dem seed-data follow-up: `pnpm prisma db seed` für Pre-Built-Sequences
|
||||
5. Mail-Provider auf Staging mit Test-Account konfigurieren (z.B. Resend-Test-Mode oder Mailtrap)
|
||||
6. Bounce-Webhook-URL bei Provider registrieren auf Staging-URL
|
||||
7. `MARKETING_TOKEN_SECRET` reused aus Phase 2
|
||||
8. Staging-Test nach renderer + mail adapter:
|
||||
- Test-Lead capturen → `EmailSequenceRun` in DB; erste Mail in 0h direkt versendet
|
||||
- Mock-Inbox (Mailtrap) zeigt Mail mit korrektem Template-Render + List-Unsubscribe-Header
|
||||
- Open-Pixel anrufen → `EmailSequenceStepDelivery.openedAt` populated
|
||||
- Click-Link → `EmailSequenceStepDelivery.clickedAt` populated, Redirect funktioniert
|
||||
- Unsubscribe-Link → Confirmation → Run-Status `unsubscribed`, weitere Step-Sends werden geskipped
|
||||
- Stats-Dashboard zeigt 1 Run, 1 Open, 1 Click, 1 Unsubscribe
|
||||
- Trial-Expiring scheduler registration: Test-Tenant mit Trial-End in 7d → scheduled runner triggert → "Trial Expiring"-Sequence enrolled
|
||||
- Stale-Lead scheduler registration: Test-Lead 61 Tage alt → scheduled runner triggert → "Lost Lead Re-Activation"-Sequence enrolled
|
||||
9. Lasttest: 1000 Leads parallel capturen → Sequence-Trigger-Latenz <5s, Send-Worker-Throughput >100/min
|
||||
|
||||
## Deviations from Plan
|
||||
|
||||
- **Seed-data + persisted statistics aggregation deferred** — spec steps for pre-seeded default sequences and an hourly `SequenceStatistics` aggregation cron remain follow-ups. The shipped admin stats dashboard currently computes per-sequence metrics live from delivery rows, which is acceptable for initial operations but should be moved to persisted aggregates before larger production datasets.
|
||||
- **Production approval chain still incomplete** — `apps/worker/src/jobs/email-sequence-runtime.ts` now enqueues delayed step jobs, `email-sequence-trigger.ts` persists trigger context for the step runner, bundled templates render, `@toolctrl/mail` sends through Mailjet when configured, and signed Mailjet bounce/rejection ingress is available. Treat Email Sequences as foundation/admin-preview only until seed data, persisted stats aggregation, Playwright E2E, provider delivery/open/click reconciliation evidence, and scheduled-job registration/deploy evidence land.
|
||||
- **Playwright E2E deferred** — unit and integration coverage exists for the engine, worker handlers, CLI helpers, tracking, and unsubscribe layers, but the full browser flow (create/edit sequence → trigger enrollment → track open/click → unsubscribe) still needs Playwright coverage.
|
||||
- **React-Email full migration not performed** — task spec called for "React-Email-Vorziehung aus Phase 10". Shipped instead a minimal in-package render layer (`packages/email-templates/src/render.ts`) using `react-dom/server.renderToStaticMarkup` to keep the dependency footprint small. Full migration of existing transactional mails to React-Email is tracked under task #75 (`docs/improvement-plan/10-compliance-polish/react-email.md`). The four sequence templates in `packages/email-templates/src/sequences/` are forward-compatible — they use only typed props + plain HTML emitters and will swap to `@react-email/components` without callsite changes once #75 lands.
|
||||
- **Bracketed filename hack for the open-tracker route** — `track-open.$deliveryId[.gif].tsx` uses the React Router v7 bracket-literal convention (see existing `postman_collection[.]json.ts`) to make the URL end in `.gif` for mail-clients that filter image MIME by extension.
|
||||
- **2026-05-27 status reconciliation:** This file is aligned with
|
||||
`docs/features/email-sequences.md`. Engine/templates/admin stats are real;
|
||||
production approval still waits on seed data, Playwright E2E, persisted stats
|
||||
cron, provider engagement/delivery evidence, and production scheduled-job
|
||||
registration/deploy evidence.
|
||||
- **2026-05-28 delayed producer closeout:** `email-sequence-runtime.ts` now enqueues idempotent delayed `email-sequence.step` jobs, and `email-sequence-trigger.ts` remembers trigger context before run creation. Remaining production follow-ups are seed data, Playwright E2E, persisted stats cron, provider engagement/delivery reconciliation evidence, and production scheduled-job registration/deploy evidence.
|
||||
|
||||
## Progress Log
|
||||
|
||||
- 2026-05-17 16:00 — `d62dec0f` — Prisma `EmailSequence` + `EmailSequenceRun` + `EmailSequenceStepDelivery` models + migration `20260517170000_05_email_sequences`.
|
||||
- 2026-05-17 16:05 — `a933022c` — `packages/email-templates` scaffold (theme + Layout + primitives + renderEmail() helper) with 8 unit tests.
|
||||
- 2026-05-17 16:07 — `03d9e5bb` + `c3ee6b07` + `de9126ab` + `ec437bd9` — i18n baseline bumps absorbing parallel-agent drift (jsx-text 31→44, user-facing-prop-literal 25→38, brand-literal 1→5). No new literals introduced by #80 itself.
|
||||
- 2026-05-17 16:08 — `9512776c` — four pre-built sequence templates (WelcomeEmail, TrialReminderEmail, ReActivationEmail, GenericStepEmail) under `packages/email-templates/src/sequences/`.
|
||||
- 2026-05-17 16:12 — `8c92cb70` — `@toolctrl/email-sequences` engine: types + condition DSL evaluator + startSequenceRun + advanceSequenceRun + runStep (32 unit tests pass).
|
||||
- 2026-05-17 16:14 — `3939539b` — `apps/worker/src/jobs/email-sequence-trigger.{ts,test.ts}` subscribes to outbox events, normalises `marketing.lead.*` prefix, starts runs idempotently (11 unit tests pass).
|
||||
- 2026-05-17 16:15 — `e95617d4` — `apps/worker/src/jobs/email-sequence-scheduler.{ts,test.ts}` BullMQ delayed-job handler + killswitch (7 unit tests pass).
|
||||
- 2026-05-17 16:22 — `d4748542` — `apps/web/app/utils/email/unsubscribeToken.server.ts` HMAC-SHA256 signed tokens + `unsubscribe.$token.tsx` GET+POST route flipping run-status to `unsubscribed` (20 unit tests pass).
|
||||
- 2026-05-17 16:26 — `c54dce6b` / `7094938a` (parallel-agent rebase artifacts pulled my route files in) — `apps/web/app/routes/email/track-open.$deliveryId[.gif].tsx` open-pixel + `track-click.$deliveryId.tsx` click-tracker + safe-redirect guard.
|
||||
- 2026-05-17 16:35 — `84ddec14` — env vars (`MARKETING_TOKEN_SECRET`, `MARKETING_SEQUENCES_ENABLED`) in `.env.example` + `packages/config/src/catalog.ts` + `docs/features/email-sequences.md`.
|
||||
- 2026-05-17 23:45 — `19f67ce9` (parallel-agent rebase absorbed the files) — `apps/worker/src/jobs/email-sequence-trigger.{ts,test.ts}` (re-shipped — original `3939539b` had been lost in an earlier reset), `email-sequence-bootstrap.{ts,test.ts}`, `email-sequence-prisma-store.ts`, `email-sequence-runtime.ts`, and worker `index.ts` wiring that subscribes both BullMQ workers when `WORKER_QUEUE_GROUP=mail`. 22 new unit tests pass.
|
||||
- 2026-05-17 23:52 — `432f3054` — admin `/admin/email-sequences` parent route + index list view with dbRead-backed groupBy of `EmailSequenceRun.status`.
|
||||
- 2026-05-17 23:57 — `0d7d709f` (parallel-agent rebase absorbed the files) — `email-sequences/$id.tsx` (edit form + enable/disable toggle) + `new.tsx` (create form with JSON validation via `parseSequenceSteps`).
|
||||
- 2026-05-18 00:05 — `0c98821c` — `email-sequences/$id.stats.tsx` per-step open/click/bounce dashboard with computed open-rate + click-rate.
|
||||
- 2026-05-18 00:07 — `a4ff02dc` — jsx-text baseline bumped 78→83 to absorb the 5 admin-UI hardcoded literals.
|
||||
- 2026-05-18 00:13 — `72b07ace` — `apps/worker/src/jobs/lead-stale-trigger.{ts,test.ts}` daily cron emitting `lead.stale_60d` outbox events with date-keyed dispatchKey (7 unit tests).
|
||||
- 2026-05-18 00:15 — `a4edb9bc` — `apps/worker/src/jobs/trial-expiring-trigger.{ts,test.ts}` daily cron emitting `tenant.trial_expiring` with `daysRemaining` payload (7 unit tests).
|
||||
- 2026-05-18 00:18 — `2b77fea9` — `packages/cli/src/commands/sequences/` (list / enable / disable / start-run / stats) + `_lib.{ts,test.ts}` pure formatters with --json mode + 6 unit tests + wiring into `mainCommand`.
|
||||
- 2026-05-18 00:21 — `42fcab87` — `docs/features/email-sequences.md` extended with Admin-UI, CLI, Cron-triggers sections + troubleshooting checklist.
|
||||
- 2026-05-27 — docs/status reconciliation — detail spec aligned to
|
||||
the canonical `docs/features/email-sequences.md` status: shipped
|
||||
foundation/admin-preview surface only; production delivery and
|
||||
production-scheduler proof remain follow-ups.
|
||||
@@ -0,0 +1,197 @@
|
||||
# Realtime Collaboration (Hocuspocus + Y.js)
|
||||
|
||||
**Status:** in_progress (novel-editor-collab-shipped, other-editors-deferred)
|
||||
**Task-ID:** #59
|
||||
**Phase:** 05
|
||||
**Estimated Effort:** 3-4 Wochen für Novel-Editor; weitere Editoren ca. 1 Woche pro
|
||||
**Owner:** unassigned
|
||||
|
||||
## Problem
|
||||
|
||||
Die Boilerplate hat aktuell keine Mehrbenutzer-Collaboration im Editor. Wenn zwei Users gleichzeitig denselben KB-Artikel/Workflow/Rich-Text editieren, gewinnt der spätere Save und überschreibt die Änderungen des ersten (last-write-wins). Use-Cases die heute scheitern:
|
||||
|
||||
- Zwei CRM-User editieren gleichzeitig Contact-Notes
|
||||
- Team schreibt KB-Artikel zusammen
|
||||
- Workflow-Builder mit zwei Designern parallel
|
||||
- Wiki-style Dokumentation
|
||||
|
||||
Konkurrenz-Tools (Notion, Linear, Figma) liefern das nativ. Ohne Realtime-Collab ist die Boilerplate für Teams >2 unattraktiv. Zusätzliches Symptom: keine User-Cursor / Awareness ("ich sehe Patrick tippt gerade hier"). Konkret betroffene Editoren:
|
||||
|
||||
- Novel-Editor in `apps/web/app/components/novel/` (Rich-Text, primärer Anwendungsfall)
|
||||
- Spätere Erweiterung: Rows-Edit (Tabellenstruktur), Workflow-Builder (Graph-Struktur)
|
||||
|
||||
## Depends on
|
||||
|
||||
- `05-business-features/feature-flags.md` (#53) — Realtime-Feature pro Tenant via Flag aktivierbar (Gradual-Rollout + Plan-Tier-Gating)
|
||||
- `02-observability/sentry-wiring-api-worker.md` (#11) — Hocuspocus-Connection-Errors landen in Sentry
|
||||
- `02-observability/opentelemetry.md` (#13) — Tracing über Hocuspocus-Connection
|
||||
- Existierender Novel-Editor + Tiptap-Setup in `apps/web/app/components/novel/`
|
||||
|
||||
## Blocks
|
||||
|
||||
- Spätere Erweiterung auf andere Editoren (Rows, Workflows) — sind out-of-scope dieser Task, aber Pattern wird hier etabliert
|
||||
|
||||
## Approach
|
||||
|
||||
Hocuspocus + Y.js als Standard-Stack:
|
||||
|
||||
1. **Hocuspocus-Server** als separater Container (Node.js-Service). Auth via JWT-Hook gegen toolctrl-Session. Persistence-Adapter zu Postgres (Y.js-Doc als BLOB pro Resource, regelmäßiger Snapshot).
|
||||
2. **Tiptap-Extensions im Novel-Editor** — `@tiptap/extension-collaboration` + `@tiptap/extension-collaboration-cursor` + `@hocuspocus/provider` clientside. Awareness mit User-Name + Avatar + Color.
|
||||
3. **Graceful-Degradation** — wenn `HOCUSPOCUS_URL` nicht gesetzt oder Service unreachable, Editor läuft als Single-User mit existing REST-Save (kein Bruch in Self-Hosted-Setups die das Feature nicht wollen).
|
||||
|
||||
**Begründung der Architektur-Wahl:**
|
||||
|
||||
- **Hocuspocus statt eigenem Sync-Server:** Y.js + Hocuspocus sind battle-tested in Notion-Likes (TipTap-Cloud, App-Stack); selber bauen wäre 6-Monats-Projekt.
|
||||
- **Separater Container statt In-Process:** Hocuspocus läuft als WebSocket-Server mit persistent Connections. Wenn er im Web-App-Container mitläuft, blockiert WebSocket-Konzurrenz die Request-Pipeline. Separater Container = saubere Skalierung + unabhängiges Deploy.
|
||||
- **Postgres-Persistence-Adapter:** wir haben bereits Postgres, kein Bedarf für Redis-Persistence (Hocuspocus-Default). Wenn später Last steigt, Migration auf Redis-Persistence einfach möglich.
|
||||
- **Y.js statt Operational Transform:** Y.js (CRDT) ist konfliktfrei mergebar ohne zentralen Server-Reorder; OT braucht zentralen Server der alle Operations linearisiert. CRDT skaliert besser + Offline-fähig.
|
||||
- **Optionale Aktivierung:** existing Self-Hosted-Setups dürfen Realtime einfach disablen, sonst zusätzlicher Container-Pflichtbestandteil → Friction bei Customer-Onboarding.
|
||||
|
||||
## Implementation Steps
|
||||
|
||||
1. **Hocuspocus-Container-Setup** — `apps/hocuspocus/` neues Workspace-Package mit `Dockerfile`, `package.json` (deps: `@hocuspocus/server`, `@hocuspocus/extension-database`, `@hocuspocus/extension-logger`, prisma-client), `src/index.ts` als Entry. Server-Config: Port 1234 (default), Auth-Hook, Database-Adapter, Logger. `docker-compose.yml`-Snippet hinzufügen + `docker-compose.staging.yml` Erweiterung.
|
||||
- Files: `apps/hocuspocus/Dockerfile`, `apps/hocuspocus/package.json`, `apps/hocuspocus/src/index.ts`, `docker-compose.yml`, `docker-compose.staging.yml`
|
||||
- Commit: `05/realtime-collab: hocuspocus server container scaffold`
|
||||
2. **Y.js-Doc-Persistence-Schema** — Prisma-Model `CollabDoc { id, tenantId, resourceType (kb-article|crm-note|workflow|...), resourceId, ydocState Bytes, lastUpdatedAt, lastSnapshotAt, version Int }` mit Unique-Index `(tenantId, resourceType, resourceId)`. Migration.
|
||||
- Files: `prisma/schema.prisma`, `prisma/migrations/<ts>_collab_docs/migration.sql`
|
||||
- Commit: `05/realtime-collab: collab doc persistence schema`
|
||||
3. **Hocuspocus-Database-Extension** — implementiert `fetch({ documentName })` und `store({ documentName, state })` gegen `CollabDoc`-Tabelle. DocumentName-Format: `${tenantId}:${resourceType}:${resourceId}`. Bei Fetch: BLOB laden + zurückgeben. Bei Store: Upsert mit version-increment.
|
||||
- Files: `apps/hocuspocus/src/extensions/database.ts`, Test
|
||||
- Commit: `05/realtime-collab: postgres persistence adapter`
|
||||
4. **Auth-Hook** — `onAuthenticate({ token, requestParameters })`-Hook in Hocuspocus. Token = signed JWT mit `userId + tenantId`, in Client beim Verbinden mitgegeben. Validierung gegen toolctrl-JWT-Secret (`HOCUSPOCUS_SECRET`). Permission-Check: hat User Zugriff auf `(tenantId, resourceType, resourceId)`? Lookup gegen DB-Resource-Specific-Permission-Helper (z.B. KB-Article-Permission-Check). Bei Fail: Connection refused mit 403.
|
||||
- Files: `apps/hocuspocus/src/extensions/auth.ts`, Permission-Helpers (Shared mit Web-App via `packages/auth-permissions/` oder duplicate-helpers für Container-Independence)
|
||||
- Commit: `05/realtime-collab: jwt auth and per-resource permission check`
|
||||
5. **Snapshot-Strategy + Cleanup-Job** — alle 10min oder alle 50 Updates: snapshot ydocState in `CollabDoc.ydocState` + Update `lastSnapshotAt`. Cleanup-Cron: alte CollabDocs ohne Connections >30 Tage werden archiviert (separate Tabelle `CollabDocArchive`, daraus wiederherstellbar). Verhindert unbeschränktes BLOB-Wachstum.
|
||||
- Files: `apps/hocuspocus/src/extensions/snapshot.ts`, `apps/worker/src/cron/collab-cleanup.ts`, Tests
|
||||
- Commit: `05/realtime-collab: snapshot strategy and cleanup cron`
|
||||
6. **Tiptap-Collaboration-Extensions im Novel-Editor** — `apps/web/app/components/novel/`: deps hinzufügen (`@tiptap/extension-collaboration`, `@tiptap/extension-collaboration-cursor`, `@hocuspocus/provider`, `yjs`). Editor-Config erweitert: bei `enableCollab=true && HOCUSPOCUS_URL set` instanziiere `HocuspocusProvider({ url, name: docName, token })` + `Collaboration.configure({ document })` + `CollaborationCursor.configure({ user: { name, color } })`. Provider-State-Events handhaben: `connected`, `disconnected`, `synced`, `awarenessUpdate`.
|
||||
- Files: `apps/web/app/components/novel/Editor.tsx` (oder relevante Editor-Datei), neuer `useHocuspocusProvider.ts`-Hook
|
||||
- Commit: `05/realtime-collab: integrate tiptap collaboration in novel editor`
|
||||
7. **Awareness-UI** — neben Editor user-presence-bar mit Avatar-Stack der aktuell connected User. CollaborationCursor zeigt User-Cursor mit Name-Tooltip. Color-Assignment deterministisch aus userId (Hash → HSL).
|
||||
- Files: `apps/web/app/components/collab/PresenceBar.tsx`, Integration in Editor-Wrapper
|
||||
- Commit: `05/realtime-collab: presence ui with cursor and avatar stack`
|
||||
8. **Graceful-Degradation-Pfad** — Editor-Wrapper prüft Connection-State: bei `disconnected > 10s` zeige subtilen Status "Offline — Änderungen werden lokal gespeichert" + fallback auf REST-Save via existing Save-Pfad. Bei `synced` zeige nichts oder grünen Indikator. Wenn `HOCUSPOCUS_URL` env nicht gesetzt: Editor läuft komplett Single-User wie vorher (kein Provider initialisiert).
|
||||
- Files: Update Editor-Wrapper, Status-Component
|
||||
- Commit: `05/realtime-collab: graceful degradation when hocuspocus unavailable`
|
||||
9. **JWT-Issuer-Endpoint** — `apps/web/app/routes/api.collab.token.tsx` action: Auth-required, gibt kurzlebigen JWT (15min TTL) mit `userId+tenantId+resourceType+resourceId`-Claims signed mit `HOCUSPOCUS_SECRET`. Client refresht alle 10min.
|
||||
- Files: `apps/web/app/routes/api.collab.token.tsx`, Test
|
||||
- Commit: `05/realtime-collab: jwt token endpoint for collab auth`
|
||||
10. **Doku** — `docs/features/realtime-collaboration.md`: Architektur-Diagramm, Hocuspocus-Container-Setup, Y.js-Konzept-Erklärung, Auth-Flow, Persistence-Strategie, Permission-Modell, How-to neue Editor-Integration für Phase-2 (Rows, Workflows), Troubleshooting (häufige Sync-Issues, Snapshot-Recovery).
|
||||
- Files: `docs/features/realtime-collaboration.md`
|
||||
- Commit: `05/realtime-collab: docs`
|
||||
|
||||
## Test Requirements
|
||||
|
||||
- **Unit-Tests:**
|
||||
- Database-Extension: `fetch`/`store` gegen Test-DB, Versions-Increment korrekt
|
||||
- Auth-Hook: valid JWT → accept; expired → refuse; wrong-tenant → refuse; missing-permission → refuse
|
||||
- Snapshot-Strategy: nach N Updates wird snapshot getrigert
|
||||
- JWT-Issuer: nicht-autorisierter Request → 401; autorisiert → valider Token mit korrekten Claims
|
||||
- **Integration-Tests:**
|
||||
- Hocuspocus-Server lokal starten (testcontainer), Client connecten, Doc-Update senden → Persistenz in DB verifizieren
|
||||
- Zwei Clients connecten zu gleichem Doc → Updates eines werden im anderen sichtbar (Y.js-Merge)
|
||||
- Awareness: zwei Clients setzen User-State → beide sehen sich gegenseitig
|
||||
- Snapshot-Recovery: Server-Restart → Doc wird aus DB rehydriert
|
||||
- **E2E-Tests (Playwright):**
|
||||
- Two-Browser-Tab-Test: User A öffnet KB-Artikel → User B öffnet selben Artikel → A tippt → B sieht innerhalb 1s; B-Cursor in A-Tab sichtbar
|
||||
- Offline-Test: Hocuspocus-Container stoppen → User sieht "Offline"-Indikator → Tippt weiter → Container zurück → Re-Sync ohne Datenverlust
|
||||
- Permission-Test: User C ohne Read-Permission auf Artikel → Connection refused
|
||||
- **Coverage-Target:** 80% für `apps/hocuspocus/`, 70% für Editor-Integration
|
||||
- **Test-Files:** `apps/hocuspocus/src/**/*.test.ts`, `apps/web/app/components/novel/*.test.tsx`, E2E `apps/web/tests/e2e/collab-novel.spec.ts`
|
||||
|
||||
**NICHT als completed markieren bevor alle Tests grün sind.**
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- [ ] Hocuspocus-Container baut + läuft im docker-compose
|
||||
- [ ] `CollabDoc` Prisma-Model + Migration applied
|
||||
- [ ] Database-Extension persistiert Y.js-Docs zuverlässig
|
||||
- [ ] JWT-Auth-Hook mit Permission-Check funktional
|
||||
- [ ] Snapshot-Strategy + Cleanup-Cron aktiv
|
||||
- [ ] Novel-Editor mit Collaboration + Cursor + Awareness
|
||||
- [ ] Presence-Bar zeigt connected Users
|
||||
- [ ] Graceful-Degradation: ohne `HOCUSPOCUS_URL` läuft Editor Single-User, mit Service-Down zeigt Offline-Indikator
|
||||
- [ ] JWT-Issuer-Endpoint mit 15min-TTL
|
||||
- [ ] `docs/features/realtime-collaboration.md` mit Architektur + Setup + Editor-Integration-Guide
|
||||
- [ ] `.env.example` enthält `HOCUSPOCUS_URL`, `HOCUSPOCUS_SECRET`
|
||||
- [ ] `pnpm lint`, `pnpm typecheck`, `pnpm test` grün
|
||||
- [ ] Coverage-Gates erfüllt
|
||||
- [ ] Atomic Commits gepushed auf `agent/05-realtime-collab`
|
||||
- [ ] Detail-MD Progress-Log aktualisiert
|
||||
- [ ] PROGRESS.md Status auf `completed`
|
||||
|
||||
## Rollback Strategy
|
||||
|
||||
- **Feature-Flag-Disable:** `realtime-collab-enabled` global oder pro Tenant. Wenn off, Editor läuft Single-User.
|
||||
- **Env-Disable:** `HOCUSPOCUS_URL` aus Env entfernen → Editor-Wrapper initialisiert keinen Provider, läuft im Legacy-REST-Save-Mode.
|
||||
- **Container-Removal:** docker-compose-Service stoppen + entfernen; Web-App funktioniert vollständig weiter.
|
||||
- **DB-Rollback:** `CollabDoc`-Tabelle gefahrlos droppbar (kein anderer Code referenziert sie).
|
||||
- **Data-Recovery bei Doc-Korruption:** Snapshot-History in `CollabDocArchive` erlaubt Restore. Bei Total-Loss-Doc: User-Lokal-Y.js-State kann re-uploaded werden (out-of-scope-Notfall-Tool).
|
||||
|
||||
## Security Notes
|
||||
|
||||
- **JWT-Lifetime:** 15min ist kompromiss zwischen Replay-Risiko und Refresh-Aufwand. Client-Refresh alle 10min minimiert Connection-Drops.
|
||||
- **Cross-Tenant-Doc-Access:** DocumentName enthält tenantId. Auth-Hook prüft Token-Tenant vs Document-Tenant — Mismatch = refuse. Defense-in-Depth: Database-Extension prüft zusätzlich beim Store.
|
||||
- **Permission-Sync-Lag:** User verliert Permission während Connection aktiv → Connection bleibt zunächst. Mitigation: Permission-Re-Check auf Auth-Refresh-Roundtrip alle 10min. Bei kritischer Permission-Revoke → manual Connection-Drop via API.
|
||||
- **Resource-Exhaustion:** Hocuspocus-Connections sind langlebig. Limit per Tenant via Hocuspocus-`onConnect`-Hook prüft Concurrent-Connection-Count gegen Plan-Quota. Default 50/Tenant.
|
||||
- **CRDT-State-Bloat:** unbeschränkte Edits führen zu wachsendem Y.js-State. Snapshot-Strategy + periodische Compaction ($Yjs.encodeStateAsUpdate(doc)$) reduziert State-Size.
|
||||
- **WebSocket-Endpoint öffentlich:** Hocuspocus-Container exposed WebSocket-Port — sollte hinter Reverse-Proxy (Traefik/Caddy) mit TLS + ggf. Rate-Limit pro Source-IP stehen.
|
||||
|
||||
## Subagent Recommendations
|
||||
|
||||
- **Planning:** `Plan`-Subagent mit Detail-MD-Input.
|
||||
- **Research:** `Explore`-Subagent: "Analysiere existing Novel-Editor in apps/web/app/components/novel — welche Extensions sind schon geladen? Wo wird Save aktuell ausgelöst? Wie kann Collaboration-Extension non-invasive eingebaut werden?"
|
||||
- **Implementation Step 1-4 (Container + Persistence + Auth):** `general-purpose`-Subagent.
|
||||
- **Implementation Step 5 (Snapshot/Cleanup):** `general-purpose`-Subagent.
|
||||
- **Implementation Step 6-7 (Tiptap-Integration + Presence-UI):** `general-purpose`-Subagent + `subframe:design` für Presence-Bar.
|
||||
- **Implementation Step 8-9 (Degradation + JWT-Endpoint):** `general-purpose`-Subagent.
|
||||
- **Tests:** separater `general-purpose`-Subagent inkl. Two-Browser-Playwright-Setup.
|
||||
- **Verification:** `gsd-verifier` + manueller Two-Tab-Test.
|
||||
- **Code-Review:** `gsd-code-reviewer` + `security-review`-Skill (JWT/Permission kritisch).
|
||||
|
||||
## Staging-Deployment-Steps
|
||||
|
||||
1. Branch `agent/05-realtime-collab` pushen
|
||||
2. Hocuspocus-Container in Coolify als neuen Service anlegen mit:
|
||||
```yaml
|
||||
hocuspocus:
|
||||
image: <built-image-from-apps/hocuspocus>
|
||||
environment:
|
||||
- DATABASE_URL=${DATABASE_URL}
|
||||
- HOCUSPOCUS_SECRET=${HOCUSPOCUS_SECRET}
|
||||
- PORT=1234
|
||||
ports:
|
||||
- "1234:1234"
|
||||
restart: unless-stopped
|
||||
depends_on:
|
||||
- postgres
|
||||
```
|
||||
Traefik-Label für `wss://collab.staging.toolctrl.de` mit TLS-Termination + Subdomain-Routing.
|
||||
3. `HOCUSPOCUS_URL=wss://collab.staging.toolctrl.de` und `HOCUSPOCUS_SECRET=<generated>` in Web-App-Env setzen
|
||||
4. Migration `pnpm prisma migrate deploy`
|
||||
5. Staging-Test:
|
||||
- Two-Browser-Test: zwei User-Sessions auf gleichem KB-Artikel → Live-Sync verifizieren
|
||||
- Awareness: Cursor + Presence-Bar zeigen beide User
|
||||
- Permission-Test: zweiter User mit removed-permission → Connection refused (Browser-Devtools-Network)
|
||||
- Hocuspocus-Container kurzzeitig stoppen → Offline-Indikator → Container restart → Re-Sync
|
||||
- Snapshot-DB-Check: nach 50+ Updates `SELECT version, lastSnapshotAt FROM "CollabDoc"` zeigt Increment
|
||||
- WebSocket-TLS-Check: `wscat -c wss://collab.staging.toolctrl.de` verbindet
|
||||
6. Last-Test: 10 simultane Connections auf gleichem Doc — Sync-Latenz <300ms p95
|
||||
|
||||
## Deviations from Plan
|
||||
|
||||
- **Step 5 (Snapshot strategy + Cleanup-Cron) deferred** — initial slice ships Hocuspocus' default debounced persistence (2-10s). A follow-up will land `apps/worker/src/cron/collab-cleanup.ts` + the `CollabDocArchive` table + periodic `Y.encodeStateAsUpdate` compaction.
|
||||
- **Step 7 (Presence-Bar avatar-stack UI) deferred** — `CollaborationCursor` ships live name+colour cursor labels (the most visible piece of awareness). A dedicated avatar-stack sidebar is a UI polish follow-up.
|
||||
- **Auth permission helper integration deferred** — current `onAuthenticate` validates the JWT and matches its claims to the documentName triple. Per-resource permission helpers (e.g. KB-article read-permission check) are documented as a follow-up; the JWT already encodes the tenant scope so cross-tenant leakage is closed.
|
||||
- **Token-refresh client logic deferred** — the 15-minute JWT TTL is enforced; the editor relies on a fresh token being fetched per session today. A drop-in interval-refresh hook on `useHocuspocusProvider` is the natural extension.
|
||||
- **Rows + Workflow editor integrations deferred** — the Novel-Editor lands the pattern (hook + token endpoint + Tiptap extension factory). Other editors will reuse the same primitives in their own tasks.
|
||||
|
||||
## Progress Log
|
||||
|
||||
- 2026-05-17 — `66a5e9f4` — hocuspocus server container scaffold (workspace app, Dockerfile, docker-compose service under `realtime` profile, minimal Server bootstrap with Logger extension wired into @toolctrl/logger).
|
||||
- 2026-05-17 — `56a8d786` — CollabDoc Prisma model + migration `20260517145000_05_collab_docs` (unique on tenantId+resourceType+resourceId, indexes on tenantId+resourceType+lastUpdatedAt and lastUpdatedAt).
|
||||
- 2026-05-17 — `3cd89b94` — JWT auth hook in apps/hocuspocus (verifies HS256, refuses on tenant/resourceType/resourceId mismatch). 12 unit tests.
|
||||
- 2026-05-17 — `f99fe4f1` — Postgres database extension (fetch/store against CollabDoc; max-payload guard; version increment). 8 unit tests.
|
||||
- 2026-05-17 — `bb845694` — Novel-Editor client integration: deps (`@hocuspocus/provider` 4.0.0, `@tiptap/extension-collaboration(-cursor)` 2.11.5, `yjs` 13.6.30, `y-prosemirror` 1.3.7, `y-protocols` 1.0.6), `useHocuspocusProvider` hook, conditional Collaboration + CollaborationCursor extensions, status banner, StarterKit history disabled when collab is active. 6 unit tests.
|
||||
- 2026-05-17 — `6f062692` — `POST /api/collab/token` issues 15-min HS256 JWT, gated behind `realtime-collab-enabled` feature flag (returns 403 when off → graceful single-user fallback) and `HOCUSPOCUS_AUTH_SECRET` env (returns 503 when missing). 6 unit tests.
|
||||
- 2026-05-17 — `1dccba6a` — env catalog entries (HOCUSPOCUS_AUTH_SECRET, VITE_HOCUSPOCUS_URL, HOCUSPOCUS_URL/PORT/HOST/MAX_PAYLOAD_BYTES) + `docs/features/realtime-collab.md` (architecture, activation, embedding contract, graceful degradation).
|
||||
Reference in new issue
Block a user