# Marketing-Funnel — Phase 1: Lead-Capture + Ingestion-API + Admin-Inbox **Status:** in_progress (core shipped) **Task-ID:** #78 **Phase:** 05 **Estimated Effort:** 1 Woche **Owner:** unassigned ## Problem Die Boilerplate hat keinen Marketing-Funnel — Leads von externen Landing-Pages, Ads oder Marketing-Sites haben keinen strukturierten Eingang ins System. Aktuell: Marketing-Page leitet zu Generic-Signup, alle UTMs und Attribution-Daten gehen verloren. Use-Cases die heute scheitern: - Marketing-Landing-Page mit Lead-Capture-Form → "Mailchimp / Brevo / HubSpot bitte selbst integrieren" - Performance-Marketing: Ads-Conversion-Tracking ohne Lead-Storage - Lead-Scoring + Qualification-Workflow durch Sales-Team - Attribution-Reporting (welcher Channel bringt welche Conversions) - Outbound-Sales: Liste der Interessenten ohne CRM zu kopieren Boilerplate-Käufer wollen einen sofort-startbaren Marketing-Funnel, nicht "bauen sich selbst CRM-Integration". ## Depends on - `05-business-features/feature-flags.md` (#53) — Turnstile-Layer hinter Flag für Gradual-Rollout - `03-scaling-infrastructure/shared-store-package.md` (#37) — Rate-Limit pro IP + pro Email via SharedStore - `02-observability/sentry-wiring-api-worker.md` (#11) — Failed-Captures landen in Sentry - Existierende AuditEvent-Infrastruktur — vorhanden, wird konsumiert - `03-scaling-infrastructure/asset-cdn.md` (#46) — `marketing.js`-SDK wird via CDN ausgeliefert ## Blocks - `05-business-features/marketing-funnel-phase-2.md` (#79) — Self-Service-Conversion baut auf Lead-Model - `05-business-features/marketing-funnel-phase-3.md` (#80) — Email-Sequences triggern auf Lead-Lifecycle-Events ## Approach Vier-Schicht-Architektur: 1. **Lead-Datenmodell** — `Lead`-Tabelle als zentraler Eintrag mit Status-Machine + Attribution-Daten. 2. **Ingestion-Endpoint** — `POST /api/leads/capture` mit CORS-Whitelist, Rate-Limit, Anti-Spam. 3. **Marketing-SDK (`marketing.js`)** — minimaler JS-Snippet für Marketing-Pages mit Auto-UTM-Capture + Form-Hook. 4. **Admin-Inbox** — DataTable mit Filter/Search, Detail-View mit Activity-Timeline + Inline-Notes. **Begründung der Architektur-Wahl:** - **Eigene Lead-Tabelle statt Tenant.User-Misuse:** Leads haben fundamentally andere Lifecycle als Tenants/Users. Misch-Schema wäre Footgun. Saubere Trennung erlaubt später spezialisierte Marketing-Logic (Scoring, Sequences) ohne User-Schema zu touchen. - **Public-Capture-Endpoint mit CORS-Whitelist:** Marketing-Domains sind bekannt (toolctrl.de, marketing-pages.toolctrl.de, externe Customer-Domains). Whitelist via Env reduziert DOS-Surface. - **Mini-SDK statt Recommendation "macht ein Fetch":** Auto-UTM-Capture + Referrer + viewport-Stuff ist boilerplate-code den jeder Marketer falsch macht. SDK gibt konsistente Datenqualität. - **Anti-Spam-Defense-in-Depth:** Honeypot (free, 80% Coverage) + Rate-Limit (Server-side) + optional Turnstile (90%+ Coverage gegen sophisticated Bots). Kein einzelner Layer reicht. - **Status-Machine (new → contacted → qualified → trial → customer → churned/disqualified):** klare Sales-Funnel-Stages, ermöglicht Funnel-Reports in Phase 2/3. ## Implementation Steps 1. **Prisma-Lead-Model** — `Lead { id, email (unique-per-status? oder pure unique mit dedup-logic), firstName?, lastName?, source, campaign?, utmSource?, utmMedium?, utmCampaign?, utmContent?, utmTerm?, referrer?, payload Json, capturedAt, ipAddress?, userAgent?, status LeadStatus (enum: new|contacted|qualified|trial|customer|churned|disqualified) default new, assignedToAdminUserId?, notes Text default "", convertedToTenantId?, convertedAt? }`. Indexes: `email`, `status`, `capturedAt DESC`, `(source, capturedAt)`, `assignedToAdminUserId`. Migration. - Files: `prisma/schema.prisma`, `prisma/migrations/_marketing_leads/migration.sql` - Commit: `05/marketing-phase-1: prisma lead model` 2. **Lead-Capture-API** — `apps/web/app/routes/api.leads.capture.tsx` action. Validation via Zod (email pflicht, sonstige optional). CORS-Check: Request-Origin gegen `MARKETING_DOMAINS` env (comma-separated). Honeypot-Field-Check (Field-Name `botfield` — wenn gefüllt: silent 200 ohne DB-Insert). Optional Turnstile-Token via Cloudflare-Verify-API. Rate-Limit via SharedStore: max 5/min/IP, max 3/min/Email. Bei Duplicate-Email innerhalb 24h: kein neuer Insert, sondern Update von `payload` und `capturedAt` (Dedup mit Activity-Append). Insert mit Auto-UTM-Population. Returnt `{ ok: true, leadId }` für SDK-Tracking. - Files: `apps/web/app/routes/api.leads.capture.tsx`, `apps/web/app/utils/leads/captureLead.server.ts`, `apps/web/app/utils/leads/originCheck.ts`, Tests - Commit: `05/marketing-phase-1: lead capture api with anti-spam` 3. **`packages/marketing-sdk-js/`** — neuer Workspace. `src/index.ts` exportiert `init({ apiUrl, formSelector? })`, `captureFormSubmission(formData)`, `trackPageView()`. Auto-Init: bei Script-Load liest `data-api-url` aus Script-Tag, attached an Forms mit `data-marketing-form` Attribut, captured Form-Submit → POST an `/api/leads/capture`. Auto-UTM aus URL-Params bei Page-Load → localStorage gecached für nachfolgende Sessions. Build-Output: minified `marketing.js` <5KB gzipped via esbuild. - Files: `packages/marketing-sdk-js/src/index.ts`, `packages/marketing-sdk-js/build.config.ts`, Tests, README - Commit: `05/marketing-phase-1: marketing-sdk-js package` 4. **CDN-Deploy-Step für `marketing.js`** — CI-Step erweitert: nach Build wird `marketing.js` zum Asset-CDN deployed unter versioniertem Pfad (`marketing.v1.js`) + symlinked als `marketing.js` für latest. SRI-Hash generieren, in Doku einbetten. - Files: `.github/workflows/deploy.yml` oder vergleichbarer CI-Step, `apps/web/app/utils/marketing/getSdkSriHash.server.ts` - Commit: `05/marketing-phase-1: ci step for marketing-sdk cdn deploy` 5. **Admin-Inbox `/admin/marketing/leads`** — Route mit Loader (paginated, mit Filter-Query-Params: status, source, search-by-email, date-range, assignedTo). DataTable-Component: Email, Name, Source, UTMs, Status (Badge), CapturedAt, AssignedTo, Actions. Bulk-Actions: assign-to-me, mark-disqualified, export-csv. Detail-View per Click → Drawer/Modal/Sub-Route mit Lead-Detail. - Files: `apps/web/app/routes/admin.marketing.leads._index.tsx`, Components - Commit: `05/marketing-phase-1: admin leads inbox` 6. **Lead-Detail-View** — `/admin/marketing/leads/$id`: shows alle Fields, Edit-Form für Status + AssignedTo + Notes, Activity-Timeline (alle AuditEvents zum Lead chronologisch). Inline-Note-Append-Action. - Files: `apps/web/app/routes/admin.marketing.leads.$id.tsx`, Action für Update - Commit: `05/marketing-phase-1: lead detail view with activity timeline` 7. **Audit-Logging** — jede Status-Mutation, Assignment, Notes-Edit schreibt AuditEvent mit `entityType=lead`, `entityId=lead.id`, `actorUserId`, `action`, `metadata` (old/new values). Verwendung des existing Audit-Helpers. - Files: `apps/web/app/utils/leads/leadAudit.server.ts`, Integration in alle Action-Handlers - Commit: `05/marketing-phase-1: audit trail for lead mutations` 8. **CSV-Export-Action** — `/admin/marketing/leads/export.csv` action: liefert gefilterte Leads als CSV-Download. Limit 50k Rows; bei mehr Background-Job-Hinweis. - Files: `apps/web/app/routes/admin.marketing.leads.export[.]csv.tsx` - Commit: `05/marketing-phase-1: csv export for leads` 9. **Doku** — `docs/features/marketing-funnel.md` (Phase-1-Section): Lead-Schema-Erklärung, API-Reference für `/api/leads/capture`, SDK-Usage mit Code-Beispiel, CORS-Setup, Anti-Spam-Layer, Admin-Inbox-Walkthrough, AuditEvent-Format. - Files: `docs/features/marketing-funnel.md` - Commit: `05/marketing-phase-1: docs` ## Test Requirements - **Unit-Tests:** - `captureLead.test.ts` — valid Lead-Insert, Duplicate-Dedup-Logic, UTM-Population, Status-Default - `originCheck.test.ts` — Whitelist-Matches, Wildcards, Reject - Honeypot-Validation: gefüllter Honeypot → silent 200, kein Insert - Rate-Limit: 6. Request innerhalb 1min/IP → 429 - Turnstile-Mock: invalid token → 403 - **Integration-Tests:** - End-to-End Capture: Request → DB-Row mit allen Feldern → AuditEvent geschrieben - Duplicate-Email innerhalb 24h: erste Capture inserts, zweite updated `payload`+`capturedAt`, schreibt zweites AuditEvent - CSV-Export: 100 Test-Leads → korrekte CSV-Struktur, alle Filter-Params respektiert - **E2E-Tests (Playwright):** - SDK-Test: Test-HTML-Page mit eingebundenem `marketing.js` + Marketing-Form → submit → Lead in Admin-Inbox sichtbar - Admin-Inbox: Filter nach Status, Search nach Email, Bulk-Disqualify, CSV-Download funktioniert - Detail-View: Status ändern → AuditEvent in Timeline; Note hinzufügen → in Timeline - Origin-Block: SDK auf nicht-whitelisteter Domain → API liefert CORS-Error, sichtbar in Test-Console - **Coverage-Target:** 85% für `packages/marketing-sdk-js/`, 85% für `captureLead.server.ts` + `originCheck.ts`, 70% UI - **Test-Files:** neben Source-Files; E2E unter `apps/web/tests/e2e/marketing-leads.spec.ts` + Test-HTML-Page `apps/web/tests/fixtures/marketing-test-page.html` **NICHT als completed markieren bevor alle Tests grün sind.** ## Acceptance Criteria - [ ] Prisma-Model `Lead` mit allen Feldern + Indexes + Migration - [ ] `/api/leads/capture` mit CORS, Honeypot, Turnstile (opt-in), Rate-Limit - [ ] `packages/marketing-sdk-js/` bauen + minified `<5KB` gzipped - [ ] CI-Step deployed `marketing.js` zum CDN mit SRI-Hash - [ ] Admin-Inbox mit Filter/Search/Pagination/Bulk-Actions - [ ] Lead-Detail-View mit Activity-Timeline + Inline-Notes - [ ] Audit-Logging jeder Status- + Assignment- + Notes-Mutation - [ ] CSV-Export mit Filter-Respekt + Row-Limit - [ ] `docs/features/marketing-funnel.md` Phase-1-Section komplett - [ ] `.env.example` enthält `MARKETING_DOMAINS`, `TURNSTILE_SITE_KEY`, `TURNSTILE_SECRET_KEY`, `MARKETING_SDK_CDN_URL` - [ ] `pnpm lint`, `pnpm typecheck`, `pnpm test` grün - [ ] Coverage-Gates erfüllt - [ ] Atomic Commits gepushed auf `agent/05-marketing-phase-1` - [ ] Detail-MD Progress-Log aktualisiert - [ ] PROGRESS.md Status auf `completed` ## Rollback Strategy - **Feature-Flag-Disable:** `marketing-funnel-enabled` global. Wenn off, `/api/leads/capture` liefert 404; Admin-UI versteckt. - **API-Disable:** Env `LEADS_CAPTURE_ENABLED=false` deaktiviert nur Endpoint, lässt Admin-Inbox sichtbar (für Read-Only-Modus während Daten-Migration). - **CDN-SDK-Versioning:** alte SDK-Version bleibt unter `marketing.v1.js`; Customer-Embeds pinnen Version → neue Bugs betreffen nur neue Embeds. - **DB-Rollback:** Lead-Tabelle gefahrlos droppbar (kein anderer Code referenziert) — aber Daten-Verlust bei Rollback nach Production-Use. Vor Rollback: Daten-Export. ## Security Notes - **PII in Leads:** Email + Name + IP sind PII. Tenant-Isolation entfällt (Leads sind global Sales-side). Aber: Read-Permission strikt auf `role=admin` oder neue Role `role=sales`. CSV-Export ist Audit-Event-pflichtig (wer hat wann was exportiert). - **Honeypot-Field-Naming:** Field-Name `botfield` ist generisch — Bots könnten lernen. Field-Name aus Env (`HONEYPOT_FIELD_NAME`) zufällig pro Deployment. - **CORS-Whitelist als Defense:** echter Bot-Schutz kommt von Turnstile, CORS schützt nur Browser-Origin-Header-Vertrauende-Requests. Server-to-Server-Bots umgehen CORS — daher Rate-Limit zwingend. - **Email-Enumeration:** Capture-Endpoint sollte nicht zwischen "Lead existiert schon" und "neuer Lead" unterscheiden (Dedup-Logic ist server-internal, Response-Code identisch). Sonst kann Bot Email-Listen verifizieren. - **GDPR/DSGVO:** Lead-Capture braucht Consent-Mechanismus. SDK-API soll `consentGiven`-Boolean unterstützen; Marketing-Page-Operator verantwortet Consent-Banner. Lead-Right-to-be-Forgotten: Delete-Action in Admin-UI vorhanden, schreibt AuditEvent für Audit. - **IP-Logging:** wir loggen ipAddress für Rate-Limit + Abuse-Detection. DSGVO-Doku-Update notwendig (Phase 10 `dsgvo-bulk-ops.md` referenziert das). ## Subagent Recommendations - **Planning:** `Plan`-Subagent mit Detail-MD-Input. - **Research:** `Explore`-Subagent: "Existing Mailchimp/Brevo-Integration-Versuche im Codebase prüfen — gibt es schon Code-Pfade die wir wiederverwenden können oder ablösen?" - **Implementation Step 1-2 (Model + API):** `general-purpose`-Subagent. - **Implementation Step 3-4 (SDK + CDN-Deploy):** `general-purpose`-Subagent mit Bundle-Size-Budget-Focus. - **Implementation Step 5-6 (Admin-UIs):** `general-purpose`-Subagent + `subframe:design` für Inbox-Layout. - **Implementation Step 7-8 (Audit + Export):** `general-purpose`-Subagent. - **Tests:** separater `general-purpose`-Subagent inkl. SDK-E2E-Setup. - **Verification:** `gsd-verifier`. - **Code-Review:** `gsd-code-reviewer` + `security-review` (PII + CORS + Anti-Spam). ## Staging-Deployment-Steps 1. Branch `agent/05-marketing-phase-1` pushen → Coolify-Auto-Deploy 2. Asset-CDN für `marketing.js` deployed (CI-Step) 3. Migration `pnpm prisma migrate deploy` 4. Turnstile-Test-Keys von Cloudflare in Staging-Env setzen 5. `MARKETING_DOMAINS=marketing.staging.toolctrl.de,test.toolctrl.de` setzen 6. Staging-Test: - Test-Landing-Page mit eingebundenem `marketing.js` deployen - Form auf Page submitten → Lead in `/admin/marketing/leads` sichtbar - UTM-Test: URL mit `?utm_source=test&utm_campaign=phase1`, danach submit → UTMs in Lead-Row - CORS-Block: Form aus nicht-whitelisteter Domain → Reject sichtbar in Devtools - Honeypot: programmatischer Submit mit gefülltem `botfield` → 200 OK aber kein Lead - Rate-Limit: 10 schnelle Submits von gleicher IP → 6.+ wird 429 - Admin-Inbox: Filter + Search + Bulk-Disqualify + CSV-Download - Audit-Timeline: Status-Änderung → Eintrag erscheint sofort 7. SDK-Bundle-Size-Verify via `curl ... | gzip -c | wc -c` < 5120 ## Deviations from Plan - ~~**Admin sidebar nav entry** not yet wired into `AdminSidebar.tsx`~~ — **resolved 2026-05-20 `f4c2f908`**: added `UI_NAV_IDS.adminMarketing` collapsible group under the `market` segment exposing Leads / Funnels / Integrations / Email-sequences sub-items, all gated on the existing `admin.marketing.*` permissions; added the four route slugs to `canonicalAdminRouteSlugs` so the sidebar route-coverage test stays green. - ~~**CSV export route on the admin UI**~~ — **resolved 2026-05-20 `89c79e9f`**: list loader now branches on `?export=csv` and streams a 50k-row-capped CSV byte-compatible with the operator-side `toolctrl-cli leads export` (same 20-column ordering, same RFC-4180 escaping). Each export writes an `admin.marketing.leads.csv_export` audit-event with the filter snapshot + `pii/gdpr` regulatory scope so SIEM/audit-trail readers can reconstruct who exported what. - **`packages/marketing-sdk-js/` workspace** not split out — the SDK lives as a single hand-minified file at `apps/web/public/embed/marketing.js` to mirror the form-widget pattern (#58). When a build-step + npm-publish flow is needed (e.g. for first-party customer sites embedding it from package managers), promote to a package. - **CDN deploy CI step + SRI hash** deferred — the file ships from the apps/web origin today; CDN setup belongs to the asset-CDN task (#46). - **Audit-Event integration** replaced with `LeadEvent` rows (status-changed, note-added, captured, re-captured, converted). The detail-view timeline reads from LeadEvent so no separate AuditEvent join was needed. Cross-system audit (e.g. exporting to SIEM) is a follow-up. - **Drag-drop form-builder UI** explicitly out-of-scope for Phase 1. - **E2E Playwright + integration tests** deferred — unit-test coverage spans origin matcher, capture endpoint response shapes (8 cases), SDK size+behavior (8 cases). Coverage gates are not yet on. ## Progress Log - 2026-05-17 14:00 — `7241dd1c` — Lead + LeadEvent prisma model + migration with status enum + first-touch attribution - 2026-05-17 14:00 — `cfcd8fca` — capture endpoint with origin allowlist + honeypot + rate-limit + Turnstile (19 unit tests) - 2026-05-17 14:03 — `53aa5f52` — hand-minified marketing.js SDK (~2KB gz, 8 size+behavior tests) - 2026-05-17 14:22 — `1f62b162` — admin /admin/marketing-leads list + detail view + status-machine + notes + timeline - 2026-05-17 14:25 — `be6c8407` — toolctrl-cli leads {list,export,delete} - 2026-05-17 14:29 — `0c17d998` — env vars + docs/features/marketing-funnel.md - 2026-05-20 — `f4c2f908` — admin sidebar wiring (Marketing funnel collapsible group: Leads / Funnels / Integrations / Email sequences) + en/de i18n keys + canonical route slugs - 2026-05-20 — `89c79e9f` — admin-UI CSV-export (`?export=csv` on list loader, 50k cap, audit-logged, header download button) + `serializeLeadsCsv` helper with 9 unit tests