# Marketing-Funnel Overview This page is the canonical overview for the shipped lead-capture and self-service conversion foundation. Email sequences are covered separately in `docs/features/email-sequences.md`. This page also keeps the marketing-outbound boundary explicit: the `packages/marketing-outbound` adapters and `apps/worker/src/jobs/marketing-outbound.ts` handler are a shipped general handler foundation, but the general subscriber is not bootstrapped from `apps/worker/src/index.ts`. That foundation is separate from the `email-marketing` CRM-contact campaign plugin candidate in `docs/feature-decisions/email-marketing-decision-report.md`. The marketing-funnel ships a four-layer pipeline that lets external landing pages, ads, and blog CTAs feed structured leads into the operator-side admin inbox without an external CRM. | Layer | Component | |------------------|------------------------------------------------------------| | 1. Data model | `Lead` + `LeadEvent` Prisma tables (`prisma/schema/marketingLeads.prisma`) | | 2. Ingestion API | `POST /api/leads/capture` (`apps/web/app/routes/api.leads.capture.tsx`) | | 3. SDK | `apps/web/public/embed/marketing.js` (origin-served mini-snippet; CDN-ready while A7/B2 remains backlog) | | 4. Admin UI | `/admin/marketing-leads` (list) + `/admin/marketing-leads/:id` (detail) | ## Status machine Leads move through seven states: ``` new → contacted → qualified → trial → customer ↘ churned ↘ disqualified ``` Transitions are recorded as `LeadEvent` rows of type `status-changed` for the activity-timeline in the detail-view. Operators move leads forward (or to a terminal state) from the detail page. The `score`, `notes`, and `convertedAt` columns are operator-managed. ## Attribution Every Lead carries first-touch attribution columns populated by the SDK on the initial visit and persisted in localStorage so subsequent page visits attribute to the same source: | Column | Source | |---------------|-----------------------------------------------------| | `utmSource` | `?utm_source=…` | | `utmMedium` | `?utm_medium=…` | | `utmCampaign` | `?utm_campaign=…` | | `utmTerm` | `?utm_term=…` | | `utmContent` | `?utm_content=…` | | `gclid` | `?gclid=…` (Google Ads click identifier) | | `fbclid` | `?fbclid=…` (Meta Ads click identifier) | | `referrer` | `document.referrer` | | `landingPage` | `location.href` at the first visit | | `firstSeenAt` | `new Date()` on initial insert (never overwritten) | | `lastTouchAt` | `new Date()` on every re-capture | `source` is operator-supplied (via the form's `data-source` attribute or the SDK call's `source` field) and identifies the surface — `"landing-pro"`, `"blog-cta"`, `"webinar-form"` — distinct from the paid-channel `utmSource`. ## Anti-spam (defence in depth) The capture endpoint stacks four layers, each weakening different bot classes; none of them is sufficient alone. 1. **Origin allowlist** — `MARKETING_ALLOWED_ORIGINS` (CSV with `*.host` wildcards). 403 on miss. Defeats browser-based form-spam from non-whitelisted pages; can be spoofed by server-to-server bots. 2. **Honeypot** — the SDK injects a hidden `website_url` field. When the server sees it filled, it responds with a silent 204 and inserts nothing. 80%+ of crude form-fillers fall here. 3. **Sliding-window rate-limit** — 5/min per IP (configurable via `MARKETING_RATE_LIMIT_PER_MIN`) and 1/min per email. The per-email limit doubles as a re-capture dedupe so the LeadEvent timeline doesn't flood. 4. **Cloudflare Turnstile** (optional) — gated behind the `marketing-funnel-turnstile` feature flag. When enabled, callers must include a valid `turnstileToken`; verification hits the Cloudflare API. Successful responses are intentionally identical for new and duplicate emails (empty body, 200 OK) so bots cannot enumerate the customer list. ## API contract ### `POST /api/leads/capture` Public, CORS-enabled. Accepts `application/json` or `application/x-www-form-urlencoded`. Body: ```json { "email": "alice@example.com", "firstName": "Alice", "lastName": "Liddell", "phone": "+44 …", "company": "Acme", "source": "landing-pro", "attribution": { "utmSource": "google", "utmCampaign": "spring-launch", "gclid": "EAIaIQ…", "referrer": "https://google.com/", "landingPage": "https://example.com/landing" }, "turnstileToken": "0.OOOO…", "website_url": "" } ``` Responses: | Status | Meaning | |--------|--------------------------------------------------| | 200 | Captured (or silently de-duplicated). Empty body.| | 204 | Honeypot triggered or per-email rate-limit hit. | | 400 | Invalid payload shape. | | 403 | Origin not allowlisted or Turnstile failed. | | 413 | Body exceeds 32 KiB. | | 429 | Per-IP rate-limit exceeded. | ### `OPTIONS /api/leads/capture` CORS preflight. Echoes the matched origin in `Access-Control-Allow-Origin`. ## SDK (`marketing.js`) Drop the script tag onto every marketing page: ```html ``` The script self-bootstraps on `DOMContentLoaded`: - Reads `?utm_source/medium/campaign/term/content`, `?gclid`, `?fbclid` from the URL and merges them with the stored first-touch attribution. - Persists merged attribution to `localStorage["__tcm_attr"]` so multi-page flows hit the same lead source. - Auto-binds any `
` on the page — submitting POSTs to `/api/leads/capture` instead of the form's action. Imperative API exposed as `window.toolctrlMarketing`: ```js toolctrlMarketing.captureLead({ email: "alice@example.com", firstName: "Alice", source: "blog-cta", }); toolctrlMarketing.bindFormById("lead-form"); toolctrlMarketing.bindFormsByAttr(); toolctrlMarketing.getAttribution(); // → merged first-touch object toolctrlMarketing.resetAttribution(); // wipes localStorage cache ``` Custom events on bound forms: - `tcm:captured` — `{ ok: boolean, status: number }` - `tcm:error` — `{ error: string }` The SDK is hand-minified to stay under the 3 KB gzipped budget (currently ~2 KB) and ships from `apps/web/public/embed/marketing.js`. ## Admin UI - **List** at `/admin/marketing-leads` — filter by status (seven badge shortcuts), source, or search by email/name/company. Paginated 25 per page. Permission: `admin.marketing.leads.view`. - **Detail** at `/admin/marketing-leads/:id` — profile + attribution panels, status transition buttons (every state reachable via one click), inline notes editor with history, convert-to-tenant action, and the LeadEvent activity timeline. Permission: `admin.marketing.leads.manage`. ## CLI ``` toolctrl-cli leads list [--status new] [--limit 50] toolctrl-cli leads export [--out leads.csv] [--status qualified] toolctrl-cli leads delete --id # DSGVO right-to-be-forgotten ``` ## Configuration | Env | Default | Purpose | |--------------------------------|---------|-------------------------------------------------------------| | `MARKETING_ALLOWED_ORIGINS` | (empty) | CSV of allowed origins for capture endpoint. Fail-closed. | | `MARKETING_RATE_LIMIT_PER_MIN` | 5 | Per-IP sliding-window submission limit. | | `TURNSTILE_SITE_KEY` | — | Cloudflare Turnstile site key (shared with form-widget). | | `TURNSTILE_SECRET_KEY` | — | Cloudflare Turnstile secret (shared with form-widget). | The `marketing-funnel-turnstile` feature flag must be enabled for Turnstile to actually gate submissions; the secret alone is not enough. ## Rollback - `MARKETING_ALLOWED_ORIGINS=` empties the allowlist → all browser captures are rejected with 403 while leaving the admin UI usable for triage. - Drop the `Lead` + `LeadEvent` tables to wipe captured data (no other tables reference them today). Export with `toolctrl-cli leads export` first. --- # Phase 2 — Self-Service Conversion + Outbound Webhooks + Attribution Phase 2 (task #79) extends Phase 1's inbox with three independently-shippable layers: 1. **Self-service conversion deep-links** — a signed lead-token in the URL pre-fills the registration form. The normal registration service creates the User + Tenant, then records the Lead conversion against that Tenant: `Lead.convertedToTenantId`, a `TenantAcquisition` snapshot, and the marketing outbox event are written in one transaction. 2. **Outbound webhook foundation** — six pluggable adapters (Slack, Discord, Mailchimp, HubSpot, Brevo, custom HTTPS) plus the `apps/worker/src/jobs/marketing-outbound.ts` handler contract. General marketing integration delivery is still a production follow-up because that handler is not currently bootstrapped from `apps/worker/src/index.ts`. 3. **Attribution reports** — `/admin/marketing-funnels` aggregates leads + acquisitions by source, campaign, utm_source, conversionPath. Mirror via `toolctrl-cli marketing funnel|attribution`. ## Signed lead-token `signLeadToken({ leadId, planId, expiresInDays = 7 })` produces a token of the shape `v1..`. The HMAC key is `LEAD_CONVERSION_SECRET` (min 32 bytes). `verifyLeadToken(token)` returns the payload when the signature matches AND the token has not expired — otherwise `null`. Both functions fail-closed when the secret env is missing. Email pre-filled from a Lead is rendered read-only on the register form to prevent lead-hijacking ("I use your token but with my email"). The signed token is also submitted as a hidden field and verified again in the register action. If email verification is enabled, the verified conversion context is stored on the pending `Registration` row and replayed from `/verify/:token` after the visitor sets a password. To change the email the visitor must abandon the link and start a fresh signup. ## Atomic conversion service `convertLeadToTenant({ leadId, tenant: { name, slug, inboundAddress }, ... })` runs inside a single Prisma `$transaction`: 1. `tenant.create` (slug + inbound address pre-resolved by caller). 2. `lead.update` → `status` advances to `trial`/`customer`, `convertedToTenantId` + `convertedAt` populated. 3. `leadEvent.create` of type `converted` with full conversion payload. 4. `tenantAcquisition.create` snapshot copying UTM/referrer/landingPage from the Lead. 5. `publishOutboxEvent` with type `marketing.lead.convertedtotrial` or `marketing.lead.convertedtocustomer` and dispatchKey `lead-convert:` (idempotent on retry). Already-converted leads (`convertedToTenantId !== null`) short-circuit and return the existing tenant. For the self-service register flow, the already-created Tenant is linked via `recordExistingTenantLeadConversion({ leadId, tenantId, ... })`. It reuses the same conversion event/outbox payload and upserts the `TenantAcquisition` row instead of creating a second Tenant. ## Admin manual override `/admin/marketing-leads/` exposes a "Create Tenant" form. Operator can override the tenant name + supply an optional planId. The action calls `convertLeadToTenant({ conversionPath: "admin-override", actorId })`. A native confirm dialog guards accidental clicks. ## Outbound webhook adapters `packages/marketing-outbound` ships six adapters under `src/adapters/*.ts`: | Adapter | Config keys (encrypted at rest) | Notes | |---------|----------------------------------------------------------------|-------| | slack | `webhookUrl`, `channel?` | Block-Kit message. | | discord | `webhookUrl`, `username?` | Embed with color-coded event. | | mailchimp | `apiKey`, `listId`, `status?`, `tags?` | PUT to lists/{id}/members/{md5(email)}. | | hubspot | `accessToken` | POST /crm/v3/objects/contacts; 409 = idempotent. | | brevo | `apiKey`, `listIds?` | POST /v3/contacts with `updateEnabled: true`. | | webhook | `url`, `signingSecret?`, `headers?` | POSTs canonical envelope; optional HMAC-SHA256 sig. | All adapters share a single `adapterFetch` helper that enforces: - HTTPS only (rejects http: / localhost / RFC1918 / 169.254.* / `metadata.google.internal` / `.internal` / `.local` / `.lan`). - 10 s default timeout (AbortController). - Returns structured `AdapterResult { ok, status?, errorTag? }` so the worker can categorise failures consistently. The `ctx.maskPii.email` toggle masks the local part of the email (`patrick@example.com → ***@example.com`) before any payload leaves the host. Use it for data-egress-restricted tenants. ## Apps/worker handler status `apps/worker/src/jobs/marketing-outbound.ts` defines a pure handler contract for marketing integration delivery, but it does not own or subscribe to `outbox.marketing`. That queue is currently used by the email-sequence trigger subscriber when `MARKETING_SEQUENCES_ENABLED=true`; this does not imply Slack/Discord/Mailchimp/HubSpot/Brevo/custom webhook delivery is active. Treat the adapters, handler, admin UI, and env flag as a shipped foundation until a bootstrap module wires a distinct general marketing integration subscriber into the worker runtime with explicit ownership tests. The handler accepts only these event types today: `marketing.lead.captured`, `marketing.lead.qualified`, `marketing.lead.convertedtotrial`, and `marketing.lead.convertedtocustomer`. New `marketing.lead.*` events require an update to `MARKETING_LEAD_EVENT_TYPES` plus tests. For each accepted event: 1. Parse + validate envelope (`parseMarketingOutboundJob`). 2. Bail out when `MARKETING_WEBHOOKS_ENABLED=false`. 3. Fetch every enabled `MarketingIntegration` whose `triggerEvents` includes the eventType. 4. For each integration: decrypt config, resolve adapter via `resolveAdapter(type)`, call `adapter.send`. One integration's failure does not affect siblings. 5. Persist diagnostics (`lastDeliveredAt`, `lastError`) on the integration row so the admin UI can surface them. ## Admin integrations UI `/admin/marketing-integrations` (Phase 05 #79). CRUD list with: - Add form (type picker + name + trigger-events multi-select + config JSON textarea). - Per-integration enable/disable toggle. - Delete with native confirm dialog. - Last-delivered / last-error display. - Config display masked (`***` for `webhookUrl`, `apiKey`, `accessToken`, `signingSecret`). Secrets are encrypted with the existing AES-256-GCM CryptoApi helper (`apps/web/app/utils/api/.server/CryptoApi.ts`) keyed off `CRYPTO_SECRET`. ## Attribution reports `/admin/marketing-funnels` (Phase 05 #79). Top-line cards (leads captured, conversions, conversion rate) plus four breakdowns: campaign, utm_source, source-label, conversionPath. Reads via `dbRead` (replica-routed). CLI mirror: - `toolctrl-cli marketing funnel --days 30` - `toolctrl-cli marketing attribution --dimension campaign --days 30 --limit 25` ## Configuration (Phase 2) | Env | Default | Purpose | |------------------------------|---------|------------------------------------------------------------| | `LEAD_CONVERSION_SECRET` | — | HMAC key for signed lead-tokens (min 32 bytes). | | `LEAD_CONVERSION_ENABLED` | `true` | Set `false` to disable the conversion flow without DB rollback. | | `MARKETING_WEBHOOKS_ENABLED` | `true` | Handler-level kill switch once the general marketing integration subscriber is bootstrapped. It does not start that subscriber by itself. | ## Webhook payload (custom adapter) ```json { "schema": "marketing.lead.v1", "eventType": "marketing.lead.convertedtotrial", "occurredAt": "2026-05-17T00:00:00.000Z", "lead": { "id": "lead_abc", "email": "patrick@example.com", "firstName": "Patrick", "lastName": "Doe", "company": "Acme", "source": "landing-pro", "utm": { "source": "google", "medium": "cpc", "campaign": "q2-launch" }, "referrer": "https://google.com", "landingPage": "/pricing" }, "conversion": { "tenantId": "tenant_xyz", "targetStatus": "trial", "conversionPath": "self-service", "planId": "plan_pro" } } ``` When `signingSecret` is set, the receiver verifies `x-marketing-signature: sha256=` against `hmac_sha256(body, signingSecret)`. ## Rollback (Phase 2) - `LEAD_CONVERSION_ENABLED=false` — register loader ignores the token query params; admin override button hidden. - `MARKETING_WEBHOOKS_ENABLED=false` — once the general marketing integration subscriber is bootstrapped, the worker handler short-circuits. Today it is only a handler-level kill switch for the shipped foundation. Outbox events accumulate; raise outbox retention if you plan a long pause. - Per-integration disable toggle in the admin UI — faulty integrations can be parked without a deploy. - `TenantAcquisition` + `MarketingIntegration` tables can be dropped (no FK in to them from Tenant), but you'd lose attribution history.