17 KiB
17 KiB
Marketing-Funnel — Phase 1: Lead-Capture + Ingestion-API + Admin-Inbox
Status: in_progress (core shipped) Task-ID: #78 Phase: 05 Estimated Effort: 1 Woche Owner: unassigned
Problem
Die Boilerplate hat keinen Marketing-Funnel — Leads von externen Landing-Pages, Ads oder Marketing-Sites haben keinen strukturierten Eingang ins System. Aktuell: Marketing-Page leitet zu Generic-Signup, alle UTMs und Attribution-Daten gehen verloren. Use-Cases die heute scheitern:
- Marketing-Landing-Page mit Lead-Capture-Form → "Mailchimp / Brevo / HubSpot bitte selbst integrieren"
- Performance-Marketing: Ads-Conversion-Tracking ohne Lead-Storage
- Lead-Scoring + Qualification-Workflow durch Sales-Team
- Attribution-Reporting (welcher Channel bringt welche Conversions)
- Outbound-Sales: Liste der Interessenten ohne CRM zu kopieren
Boilerplate-Käufer wollen einen sofort-startbaren Marketing-Funnel, nicht "bauen sich selbst CRM-Integration".
Depends on
05-business-features/feature-flags.md(#53) — Turnstile-Layer hinter Flag für Gradual-Rollout03-scaling-infrastructure/shared-store-package.md(#37) — Rate-Limit pro IP + pro Email via SharedStore02-observability/sentry-wiring-api-worker.md(#11) — Failed-Captures landen in Sentry- Existierende AuditEvent-Infrastruktur — vorhanden, wird konsumiert
03-scaling-infrastructure/asset-cdn.md(#46) —marketing.js-SDK wird via CDN ausgeliefert
Blocks
05-business-features/marketing-funnel-phase-2.md(#79) — Self-Service-Conversion baut auf Lead-Model05-business-features/marketing-funnel-phase-3.md(#80) — Email-Sequences triggern auf Lead-Lifecycle-Events
Approach
Vier-Schicht-Architektur:
- Lead-Datenmodell —
Lead-Tabelle als zentraler Eintrag mit Status-Machine + Attribution-Daten. - Ingestion-Endpoint —
POST /api/leads/capturemit CORS-Whitelist, Rate-Limit, Anti-Spam. - Marketing-SDK (
marketing.js) — minimaler JS-Snippet für Marketing-Pages mit Auto-UTM-Capture + Form-Hook. - Admin-Inbox — DataTable mit Filter/Search, Detail-View mit Activity-Timeline + Inline-Notes.
Begründung der Architektur-Wahl:
- Eigene Lead-Tabelle statt Tenant.User-Misuse: Leads haben fundamentally andere Lifecycle als Tenants/Users. Misch-Schema wäre Footgun. Saubere Trennung erlaubt später spezialisierte Marketing-Logic (Scoring, Sequences) ohne User-Schema zu touchen.
- Public-Capture-Endpoint mit CORS-Whitelist: Marketing-Domains sind bekannt (toolctrl.de, marketing-pages.toolctrl.de, externe Customer-Domains). Whitelist via Env reduziert DOS-Surface.
- Mini-SDK statt Recommendation "macht ein Fetch": Auto-UTM-Capture + Referrer + viewport-Stuff ist boilerplate-code den jeder Marketer falsch macht. SDK gibt konsistente Datenqualität.
- Anti-Spam-Defense-in-Depth: Honeypot (free, 80% Coverage) + Rate-Limit (Server-side) + optional Turnstile (90%+ Coverage gegen sophisticated Bots). Kein einzelner Layer reicht.
- Status-Machine (new → contacted → qualified → trial → customer → churned/disqualified): klare Sales-Funnel-Stages, ermöglicht Funnel-Reports in Phase 2/3.
Implementation Steps
- Prisma-Lead-Model —
Lead { id, email (unique-per-status? oder pure unique mit dedup-logic), firstName?, lastName?, source, campaign?, utmSource?, utmMedium?, utmCampaign?, utmContent?, utmTerm?, referrer?, payload Json, capturedAt, ipAddress?, userAgent?, status LeadStatus (enum: new|contacted|qualified|trial|customer|churned|disqualified) default new, assignedToAdminUserId?, notes Text default "", convertedToTenantId?, convertedAt? }. Indexes:email,status,capturedAt DESC,(source, capturedAt),assignedToAdminUserId. Migration.- Files:
prisma/schema.prisma,prisma/migrations/<ts>_marketing_leads/migration.sql - Commit:
05/marketing-phase-1: prisma lead model
- Files:
- Lead-Capture-API —
apps/web/app/routes/api.leads.capture.tsxaction. Validation via Zod (email pflicht, sonstige optional). CORS-Check: Request-Origin gegenMARKETING_DOMAINSenv (comma-separated). Honeypot-Field-Check (Field-Namebotfield— wenn gefüllt: silent 200 ohne DB-Insert). Optional Turnstile-Token via Cloudflare-Verify-API. Rate-Limit via SharedStore: max 5/min/IP, max 3/min/Email. Bei Duplicate-Email innerhalb 24h: kein neuer Insert, sondern Update vonpayloadundcapturedAt(Dedup mit Activity-Append). Insert mit Auto-UTM-Population. Returnt{ ok: true, leadId }für SDK-Tracking.- Files:
apps/web/app/routes/api.leads.capture.tsx,apps/web/app/utils/leads/captureLead.server.ts,apps/web/app/utils/leads/originCheck.ts, Tests - Commit:
05/marketing-phase-1: lead capture api with anti-spam
- Files:
packages/marketing-sdk-js/— neuer Workspace.src/index.tsexportiertinit({ apiUrl, formSelector? }),captureFormSubmission(formData),trackPageView(). Auto-Init: bei Script-Load liestdata-api-urlaus Script-Tag, attached an Forms mitdata-marketing-formAttribut, captured Form-Submit → POST an/api/leads/capture. Auto-UTM aus URL-Params bei Page-Load → localStorage gecached für nachfolgende Sessions. Build-Output: minifiedmarketing.js<5KB gzipped via esbuild.- Files:
packages/marketing-sdk-js/src/index.ts,packages/marketing-sdk-js/build.config.ts, Tests, README - Commit:
05/marketing-phase-1: marketing-sdk-js package
- Files:
- CDN-Deploy-Step für
marketing.js— CI-Step erweitert: nach Build wirdmarketing.jszum Asset-CDN deployed unter versioniertem Pfad (marketing.v1.js) + symlinked alsmarketing.jsfür latest. SRI-Hash generieren, in Doku einbetten.- Files:
.github/workflows/deploy.ymloder vergleichbarer CI-Step,apps/web/app/utils/marketing/getSdkSriHash.server.ts - Commit:
05/marketing-phase-1: ci step for marketing-sdk cdn deploy
- Files:
- Admin-Inbox
/admin/marketing/leads— Route mit Loader (paginated, mit Filter-Query-Params: status, source, search-by-email, date-range, assignedTo). DataTable-Component: Email, Name, Source, UTMs, Status (Badge), CapturedAt, AssignedTo, Actions. Bulk-Actions: assign-to-me, mark-disqualified, export-csv. Detail-View per Click → Drawer/Modal/Sub-Route mit Lead-Detail.- Files:
apps/web/app/routes/admin.marketing.leads._index.tsx, Components - Commit:
05/marketing-phase-1: admin leads inbox
- Files:
- Lead-Detail-View —
/admin/marketing/leads/$id: shows alle Fields, Edit-Form für Status + AssignedTo + Notes, Activity-Timeline (alle AuditEvents zum Lead chronologisch). Inline-Note-Append-Action.- Files:
apps/web/app/routes/admin.marketing.leads.$id.tsx, Action für Update - Commit:
05/marketing-phase-1: lead detail view with activity timeline
- Files:
- Audit-Logging — jede Status-Mutation, Assignment, Notes-Edit schreibt AuditEvent mit
entityType=lead,entityId=lead.id,actorUserId,action,metadata(old/new values). Verwendung des existing Audit-Helpers.- Files:
apps/web/app/utils/leads/leadAudit.server.ts, Integration in alle Action-Handlers - Commit:
05/marketing-phase-1: audit trail for lead mutations
- Files:
- CSV-Export-Action —
/admin/marketing/leads/export.csvaction: liefert gefilterte Leads als CSV-Download. Limit 50k Rows; bei mehr Background-Job-Hinweis.- Files:
apps/web/app/routes/admin.marketing.leads.export[.]csv.tsx - Commit:
05/marketing-phase-1: csv export for leads
- Files:
- Doku —
docs/features/marketing-funnel.md(Phase-1-Section): Lead-Schema-Erklärung, API-Reference für/api/leads/capture, SDK-Usage mit Code-Beispiel, CORS-Setup, Anti-Spam-Layer, Admin-Inbox-Walkthrough, AuditEvent-Format.- Files:
docs/features/marketing-funnel.md - Commit:
05/marketing-phase-1: docs
- Files:
Test Requirements
- Unit-Tests:
captureLead.test.ts— valid Lead-Insert, Duplicate-Dedup-Logic, UTM-Population, Status-DefaultoriginCheck.test.ts— Whitelist-Matches, Wildcards, Reject- Honeypot-Validation: gefüllter Honeypot → silent 200, kein Insert
- Rate-Limit: 6. Request innerhalb 1min/IP → 429
- Turnstile-Mock: invalid token → 403
- Integration-Tests:
- End-to-End Capture: Request → DB-Row mit allen Feldern → AuditEvent geschrieben
- Duplicate-Email innerhalb 24h: erste Capture inserts, zweite updated
payload+capturedAt, schreibt zweites AuditEvent - CSV-Export: 100 Test-Leads → korrekte CSV-Struktur, alle Filter-Params respektiert
- E2E-Tests (Playwright):
- SDK-Test: Test-HTML-Page mit eingebundenem
marketing.js+ Marketing-Form → submit → Lead in Admin-Inbox sichtbar - Admin-Inbox: Filter nach Status, Search nach Email, Bulk-Disqualify, CSV-Download funktioniert
- Detail-View: Status ändern → AuditEvent in Timeline; Note hinzufügen → in Timeline
- Origin-Block: SDK auf nicht-whitelisteter Domain → API liefert CORS-Error, sichtbar in Test-Console
- SDK-Test: Test-HTML-Page mit eingebundenem
- Coverage-Target: 85% für
packages/marketing-sdk-js/, 85% fürcaptureLead.server.ts+originCheck.ts, 70% UI - Test-Files: neben Source-Files; E2E unter
apps/web/tests/e2e/marketing-leads.spec.ts+ Test-HTML-Pageapps/web/tests/fixtures/marketing-test-page.html
NICHT als completed markieren bevor alle Tests grün sind.
Acceptance Criteria
- Prisma-Model
Leadmit allen Feldern + Indexes + Migration /api/leads/capturemit CORS, Honeypot, Turnstile (opt-in), Rate-Limitpackages/marketing-sdk-js/bauen + minified<5KBgzipped- CI-Step deployed
marketing.jszum CDN mit SRI-Hash - Admin-Inbox mit Filter/Search/Pagination/Bulk-Actions
- Lead-Detail-View mit Activity-Timeline + Inline-Notes
- Audit-Logging jeder Status- + Assignment- + Notes-Mutation
- CSV-Export mit Filter-Respekt + Row-Limit
docs/features/marketing-funnel.mdPhase-1-Section komplett.env.exampleenthältMARKETING_DOMAINS,TURNSTILE_SITE_KEY,TURNSTILE_SECRET_KEY,MARKETING_SDK_CDN_URLpnpm lint,pnpm typecheck,pnpm testgrün- Coverage-Gates erfüllt
- Atomic Commits gepushed auf
agent/05-marketing-phase-1 - Detail-MD Progress-Log aktualisiert
- PROGRESS.md Status auf
completed
Rollback Strategy
- Feature-Flag-Disable:
marketing-funnel-enabledglobal. Wenn off,/api/leads/captureliefert 404; Admin-UI versteckt. - API-Disable: Env
LEADS_CAPTURE_ENABLED=falsedeaktiviert nur Endpoint, lässt Admin-Inbox sichtbar (für Read-Only-Modus während Daten-Migration). - CDN-SDK-Versioning: alte SDK-Version bleibt unter
marketing.v1.js; Customer-Embeds pinnen Version → neue Bugs betreffen nur neue Embeds. - DB-Rollback: Lead-Tabelle gefahrlos droppbar (kein anderer Code referenziert) — aber Daten-Verlust bei Rollback nach Production-Use. Vor Rollback: Daten-Export.
Security Notes
- PII in Leads: Email + Name + IP sind PII. Tenant-Isolation entfällt (Leads sind global Sales-side). Aber: Read-Permission strikt auf
role=adminoder neue Rolerole=sales. CSV-Export ist Audit-Event-pflichtig (wer hat wann was exportiert). - Honeypot-Field-Naming: Field-Name
botfieldist generisch — Bots könnten lernen. Field-Name aus Env (HONEYPOT_FIELD_NAME) zufällig pro Deployment. - CORS-Whitelist als Defense: echter Bot-Schutz kommt von Turnstile, CORS schützt nur Browser-Origin-Header-Vertrauende-Requests. Server-to-Server-Bots umgehen CORS — daher Rate-Limit zwingend.
- Email-Enumeration: Capture-Endpoint sollte nicht zwischen "Lead existiert schon" und "neuer Lead" unterscheiden (Dedup-Logic ist server-internal, Response-Code identisch). Sonst kann Bot Email-Listen verifizieren.
- GDPR/DSGVO: Lead-Capture braucht Consent-Mechanismus. SDK-API soll
consentGiven-Boolean unterstützen; Marketing-Page-Operator verantwortet Consent-Banner. Lead-Right-to-be-Forgotten: Delete-Action in Admin-UI vorhanden, schreibt AuditEvent für Audit. - IP-Logging: wir loggen ipAddress für Rate-Limit + Abuse-Detection. DSGVO-Doku-Update notwendig (Phase 10
dsgvo-bulk-ops.mdreferenziert das).
Subagent Recommendations
- Planning:
Plan-Subagent mit Detail-MD-Input. - Research:
Explore-Subagent: "Existing Mailchimp/Brevo-Integration-Versuche im Codebase prüfen — gibt es schon Code-Pfade die wir wiederverwenden können oder ablösen?" - Implementation Step 1-2 (Model + API):
general-purpose-Subagent. - Implementation Step 3-4 (SDK + CDN-Deploy):
general-purpose-Subagent mit Bundle-Size-Budget-Focus. - Implementation Step 5-6 (Admin-UIs):
general-purpose-Subagent +subframe:designfür Inbox-Layout. - Implementation Step 7-8 (Audit + Export):
general-purpose-Subagent. - Tests: separater
general-purpose-Subagent inkl. SDK-E2E-Setup. - Verification:
gsd-verifier. - Code-Review:
gsd-code-reviewer+security-review(PII + CORS + Anti-Spam).
Staging-Deployment-Steps
- Branch
agent/05-marketing-phase-1pushen → Coolify-Auto-Deploy - Asset-CDN für
marketing.jsdeployed (CI-Step) - Migration
pnpm prisma migrate deploy - Turnstile-Test-Keys von Cloudflare in Staging-Env setzen
MARKETING_DOMAINS=marketing.staging.toolctrl.de,test.toolctrl.desetzen- Staging-Test:
- Test-Landing-Page mit eingebundenem
marketing.jsdeployen - Form auf Page submitten → Lead in
/admin/marketing/leadssichtbar - UTM-Test: URL mit
?utm_source=test&utm_campaign=phase1, danach submit → UTMs in Lead-Row - CORS-Block: Form aus nicht-whitelisteter Domain → Reject sichtbar in Devtools
- Honeypot: programmatischer Submit mit gefülltem
botfield→ 200 OK aber kein Lead - Rate-Limit: 10 schnelle Submits von gleicher IP → 6.+ wird 429
- Admin-Inbox: Filter + Search + Bulk-Disqualify + CSV-Download
- Audit-Timeline: Status-Änderung → Eintrag erscheint sofort
- Test-Landing-Page mit eingebundenem
- SDK-Bundle-Size-Verify via
curl ... | gzip -c | wc -c< 5120
Deviations from Plan
Admin sidebar nav entry not yet wired into— resolved 2026-05-20AdminSidebar.tsxf4c2f908: addedUI_NAV_IDS.adminMarketingcollapsible group under themarketsegment exposing Leads / Funnels / Integrations / Email-sequences sub-items, all gated on the existingadmin.marketing.*permissions; added the four route slugs tocanonicalAdminRouteSlugsso the sidebar route-coverage test stays green.CSV export route on the admin UI— resolved 2026-05-2089c79e9f: list loader now branches on?export=csvand streams a 50k-row-capped CSV byte-compatible with the operator-sidetoolctrl-cli leads export(same 20-column ordering, same RFC-4180 escaping). Each export writes anadmin.marketing.leads.csv_exportaudit-event with the filter snapshot +pii/gdprregulatory scope so SIEM/audit-trail readers can reconstruct who exported what.packages/marketing-sdk-js/workspace not split out — the SDK lives as a single hand-minified file atapps/web/public/embed/marketing.jsto mirror the form-widget pattern (#58). When a build-step + npm-publish flow is needed (e.g. for first-party customer sites embedding it from package managers), promote to a package.- CDN deploy CI step + SRI hash deferred — the file ships from the apps/web origin today; CDN setup belongs to the asset-CDN task (#46).
- Audit-Event integration replaced with
LeadEventrows (status-changed, note-added, captured, re-captured, converted). The detail-view timeline reads from LeadEvent so no separate AuditEvent join was needed. Cross-system audit (e.g. exporting to SIEM) is a follow-up. - Drag-drop form-builder UI explicitly out-of-scope for Phase 1.
- E2E Playwright + integration tests deferred — unit-test coverage spans origin matcher, capture endpoint response shapes (8 cases), SDK size+behavior (8 cases). Coverage gates are not yet on.
Progress Log
- 2026-05-17 14:00 —
7241dd1c— Lead + LeadEvent prisma model + migration with status enum + first-touch attribution - 2026-05-17 14:00 —
cfcd8fca— capture endpoint with origin allowlist + honeypot + rate-limit + Turnstile (19 unit tests) - 2026-05-17 14:03 —
53aa5f52— hand-minified marketing.js SDK (~2KB gz, 8 size+behavior tests) - 2026-05-17 14:22 —
1f62b162— admin /admin/marketing-leads list + detail view + status-machine + notes + timeline - 2026-05-17 14:25 —
be6c8407— toolctrl-cli leads {list,export,delete} - 2026-05-17 14:29 —
0c17d998— env vars + docs/features/marketing-funnel.md - 2026-05-20 —
f4c2f908— admin sidebar wiring (Marketing funnel collapsible group: Leads / Funnels / Integrations / Email sequences) + en/de i18n keys + canonical route slugs - 2026-05-20 —
89c79e9f— admin-UI CSV-export (?export=csvon list loader, 50k cap, audit-logged, header download button) +serializeLeadsCsvhelper with 9 unit tests