# Email-Sequence Engine (Marketing-Funnel Phase 3) **Status:** Phase 05 Task #80 — the sequence data model, pure engine foundation, admin UI/CLI, local E2E readiness seed fixtures, unsubscribe route, open/click tracking routes, worker bootstrap contracts, delayed-step enqueueing foundation, bundled template renderer, `@toolctrl/mail` Mailjet adapter, signed Mailjet bounce webhook handling, unit-tested stale-lead/trial-expiring cron runner functions, and live admin statistics views are in place. This is shipped foundation/admin-preview work, not an end-to-end production email-delivery claim for marketing sequences. The worker entrypoints are default-safe: `MARKETING_SEQUENCES_ENABLED` must be explicitly set to `true` before sequence subscribers do work; entrypoint tests prove the sequence workers and readiness probes stay disabled while the flag is unset. Keep it unset or `false` in production until production/demo seed rollout evidence, statistics aggregation cron, Playwright E2E coverage, delivery webhook production evidence, and production scheduled-job registration/deploy evidence are complete. ## What ships | Component | Package | Notes | | ------------------------- | ------------------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------- | | Data model | `prisma/schema/emailSequences.prisma` | `EmailSequence`, `EmailSequenceRun`, `EmailSequenceStepDelivery` | | Engine | `@toolctrl/email-sequences` | pure-logic — types, conditions, engine, step-runner | | Template definitions | `@toolctrl/email-templates` | welcome / trial-reminder / re-activation / generic-step renderer used by the worker runtime | | Trigger worker contract | `apps/worker/src/jobs/email-sequence-trigger.ts` | consumes matching outbox events, persists trigger context, and starts runs idempotently | | Scheduler worker contract | `apps/worker/src/jobs/email-sequence-scheduler.ts` | step-runner foundation; runtime enqueues idempotent delayed jobs and sends through `@toolctrl/mail` when Mailjet env is configured | | Unsubscribe route | `apps/web/app/routes/unsubscribe.$token.tsx` | RFC-8058 one-click (GET + POST) | | Tracking — open | `apps/web/app/routes/email/track-open.$deliveryId[.gif].tsx` | 1×1 transparent GIF | | Tracking — click | `apps/web/app/routes/email/track-click.$deliveryId.tsx` | 302-redirect + first-touch update | | Provider bounce webhook | `POST /v1/email-sequences/mailjet/webhooks` | Signed Mailjet ingress marks correlated sequence deliveries as bounced | | Local readiness seed | `scripts/seed-e2e-readiness.ts` | deterministic welcome / trial-expiring / stale-lead / customer-welcome definitions for local E2E fixtures | This page claims active ownership of the email-sequence subscriber on `outbox.marketing`. It does not classify Email Sequences as the `email-marketing` CRM-contact campaign plugin candidate from `docs/feature-decisions/email-marketing-decision-report.md` or `docs/feature-manifests/plugin-candidates.json`. Treat Email Sequences as foundation/admin-preview work until the production follow-ups below land. The generic marketing integration handler/adapters for Slack, Discord, Mailchimp, HubSpot, Brevo, and custom webhooks are documented in `docs/features/marketing-funnel.md`; as of this status reconciliation that handler is pure foundation and the general subscriber is not bootstrapped from `apps/worker/src/index.ts`. ## Architecture The diagram below shows the sequence flow. The delayed-job producer, bundled template renderer, and `@toolctrl/mail` Mailjet adapter exist, but missing Mailjet configuration still fails closed and the remaining production follow-ups below must be completed before this path should be treated as approved production delivery. ``` ┌──────────────────────────────┐ lead.captured / tenant.signup outbox event │ ▼ ┌──────────────────────────────┐ │ email-sequence-trigger │ apps/worker/src/jobs │ - lookup matching seqs │ │ - createRun (idempotent) │ │ - records pending step │ └──────────────┬───────────────┘ │ delayed enqueue ▼ ┌──────────────────────────────┐ │ email-sequence-scheduler │ │ - runStep │ │ ├── conditions │ │ ├── renderTemplate │ bundled sequence templates │ ├── sendMail │ @toolctrl/mail Mailjet adapter │ ├── markDelivery │ │ └── advanceSequenceRun │ └──────────────────────────────┘ ``` ## Trigger events (per spec) - `lead.captured` — `marketing.lead.captured` outbox prefix is normalised - `lead.qualified` - `lead.converted_to_trial` / `lead.converted_to_customer` - `lead.stale_60d` — emitted by the stale-lead cron runner once scheduled in the worker runtime - `tenant.signup` - `tenant.trial_expiring` - `tenant.cancelled` - `custom` — operator-defined ## Step-condition DSL ```jsonc { "path": "lead.status", "op": "eq | neq | in | not_in | exists | not_exists", "value": "qualified", // string | number | boolean | array (for in / not_in) } ``` Per-step `conditions` are ANDed; sequence-level `conditions` gate the whole sequence at trigger-time. ## Unsubscribe + List-Unsubscribe When production delivery is wired, every sequence email must inject: ``` List-Unsubscribe: >, @example.com> List-Unsubscribe-Post: List-Unsubscribe=One-Click ``` The token is `u1..` signed with `MARKETING_TOKEN_SECRET` (falls back to `LEAD_CONVERSION_SECRET`). Tokens never expire — secret rotation is the kill-switch. ## Open / click tracking - Open: HTML `` — sets `openedAt` on first fetch. - Click: Every anchor URL is rewritten to `https://app.example.com/email/track-click/?u=` — sets `clickedAt` on first hit, 302s to the original target. Only `http:` / `https:` schemes accepted (no `javascript:`, `data:` etc.). ### Caveats - Apple Mail Privacy Protection pre-fetches images → false-positive opens. - Image-blocking mail clients (Outlook default) suppress the pixel → false-negative opens. - Click-tracking introduces an extra hop — bots and security scanners may pre-resolve URLs, inflating clicks. The admin stats dashboard documents these as approximations. It currently computes live per-sequence statistics from delivery rows; a persisted aggregation cron remains a follow-up for larger datasets. ## Provider bounce webhooks `POST /v1/email-sequences/mailjet/webhooks` accepts signed Mailjet provider events using `MAILJET_WEBHOOK_SECRET`. Bounce and rejection events correlate against `EmailSequenceStepDelivery.providerMessageId`, set `status=bounced`, record `bouncedAt` / `errorMessage`, and fail the related `EmailSequenceRun`. Unknown provider message ids and non-bounce events return `202` without echoing provider payloads to avoid retry storms. ## Templates Minimal React-Email-style template definitions exist. The worker runtime turns these definitions into a `@toolctrl/mail` send payload, but the remaining production follow-ups below still need to land before marketing sequences should be cited as approved production mail end to end. | Template | Trigger example | Variables | | -------------------- | ---------------------------------- | -------------------------------------------------------------------------------------- | | `WelcomeEmail` | `lead.captured` day-0 | `recipientName`, `productName`, `ctaUrl`, `ctaLabel?`, `unsubscribeUrl`, `senderName?` | | `TrialReminderEmail` | `tenant.trial_expiring` | `recipientName?`, `productName`, `daysRemaining`, `upgradeUrl`, `unsubscribeUrl` | | `ReActivationEmail` | `lead.stale_60d` | `recipientName?`, `productName`, `returnUrl`, `highlight?`, `unsubscribeUrl` | | `GenericStepEmail` | onboarding-drip / operator-defined | `headline`, `bodyParagraphs[]`, `ctaUrl?`, `ctaLabel?`, `unsubscribeUrl` | The full React-Email migration tracked under #75 is no longer the blocker here; the worker runtime renders the bundled sequence templates and sends through `@toolctrl/mail` when `MAILJET_API_KEY` and `MAILJET_FROM_EMAIL` are configured together. If either key is absent, step delivery fails closed with `email_sequence_mail_sender_unavailable`. ## Admin UI `/admin/email-sequences` (permission `admin.marketing.leads.view`): - **List view** — `index.tsx`. Name, trigger event, enabled badge, step count, per-status run counts (active / completed / unsubscribed) aggregated from `EmailSequenceRun.groupBy` against `dbRead`. - **Detail / edit** — `$id.tsx`. Edit `name`, `description`, `triggerEvent`, `stepsJson` (validated by `parseSequenceSteps` before save), `conditionsJson`. Inline toggle for `isEnabled`. - **Create** — `new.tsx`. Same form pre-seeded with a single-step template; new sequences are created with `isEnabled=false` so the operator can review the JSON before turning the trigger on. - **Stats** — `$id/stats.tsx`. Per-step `scheduled` / `sent` / `opened` / `clicked` / `bounced` / `skipped` counters with computed open-rate and click-rate (denominator = sent count, "—" when sent=0). Sequence-level enrollment + completion + unsubscribe + failure totals. No drag-drop step builder — operators paste step + condition JSON directly into the form. ## CLI `toolctrl-cli sequences ` exposes operator commands: | Subcommand | Purpose | `--json` | | -------------------------------------- | -------------------------------------------------------------- | -------- | | `list` | Print every sequence (id, name, trigger, on/off, step count) | yes | | `enable ` | Flip `isEnabled` to true | yes | | `disable ` | Flip `isEnabled` to false | yes | | `start-run --leadId / --tenantId` | Manually enrol a target (skips outbox trigger) | yes | | `stats` | Per-sequence active / completed / unsubscribed / failed counts | yes | All commands respect `MARKETING_SEQUENCES_ENABLED=false` indirectly where the worker contracts are wired — when false, `worker-mail` does not subscribe to the sequence trigger/scheduler queues, so outbox or BullMQ work can accumulate and replay after re-enable. The CLI itself never refuses to inspect DB state. In production the recommended value remains `false` until the remaining production follow-ups land and the Mailjet env pair is configured; otherwise runs, delivery rows, and delayed jobs can be created without an operator-approved delivery path. ## Cron triggers Two unit-tested cron runner functions exist for producing outbox events that flow into the sequence trigger handler. Production daily scheduler registration is still follow-up unless the deployment runbook for that environment proves it. - `apps/worker/src/jobs/lead-stale-trigger.ts` — finds `Lead` rows with `firstSeenAt < now - 60d AND status IN (new, contacted)` and emits `lead.stale_60d`. - `apps/worker/src/jobs/trial-expiring-trigger.ts` — finds tenant subscriptions whose trial ends within 3 days and emits `tenant.trial_expiring` with `daysRemaining` in the payload. Both use date-keyed dispatch keys (`lead.stale_60d..` / `tenant.trial_expiring..`) so re-runs on the same day are idempotent. The next code blocker for production scheduler registration is the trial source contract: `runTrialExpiringTriggerCron` needs a canonical `trialEndsAt`, but the current subscription schema does not expose one directly. Treat deploy-specific cron evidence, alerting, and production schedule verification as remaining operator/runtime follow-up work unless the deployment runbook for that environment proves otherwise. ## Env | Var | Default | Effect | | ----------------------------- | ---------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | `MARKETING_TOKEN_SECRET` | (falls back to `LEAD_CONVERSION_SECRET`) | HMAC key for unsubscribe tokens | | `MARKETING_SEQUENCES_ENABLED` | unset / `false` in production examples | Opt-in switch — trigger, scheduler, and sequence cron producers only do work when this is exactly `true`. Keep this unset or `false` in production until the remaining production follow-ups are approved. | | `MAILJET_API_KEY` | unset | Required with `MAILJET_FROM_EMAIL` for sequence step delivery through `@toolctrl/mail`; missing configuration fails closed. | | `MAILJET_FROM_EMAIL` | unset | Required with `MAILJET_API_KEY` for sequence step delivery through `@toolctrl/mail`; missing configuration fails closed. | | `MAILJET_FROM_NAME` | unset | Optional display name passed to `@toolctrl/mail`. | | `MAILJET_WEBHOOK_SECRET` | unset | Required on the API runtime for signed sequence bounce webhook ingestion at `/v1/email-sequences/mailjet/webhooks`. | ## Troubleshooting - Unsubscribe link 404s → check `MARKETING_TOKEN_SECRET` matches between mail send time and verify time. - Sequence does not start → confirm outbox dispatcher is running + `EmailSequence.isEnabled = true` + `triggerEvent` matches the (normalised) outbox event-type. - Steps stuck pending → check the scheduler worker logs, the `email-sequence.step` queue depth, Redis connectivity, and idempotent job ids (`email-sequence-step::`). - Open-rate looks impossible (>100%) → Apple Mail Privacy Protection pre-fetch; combine with click-rate for a more reliable engagement signal. - Hard bounce → run transitions to `failed` and no further steps fire. The Lead row's `unsubscribedAt` is NOT touched (Phase 1 lead-model owns that semantic; bounce is per-run only). - Sequence isn't sending — use this quick diagnostic checklist: 1. `toolctrl-cli sequences list --json` → confirm `isEnabled=true` on the matching row. 2. `triggerEvent` matches the normalised outbox event-type. Outbox emits `marketing.lead.captured` but sequences store `lead.captured` — the trigger handler normalises both directions. Manually configured sequences with a `marketing.`-prefixed trigger will never match. 3. The permanent `worker-mail` service is running with `WORKER_QUEUE_GROUP=mail`; the bootstrap helper short-circuits on other groups. 4. `MAILJET_API_KEY` and `MAILJET_FROM_EMAIL` are both configured on `worker-mail`. If either is missing, step delivery fails closed with `email_sequence_mail_sender_unavailable`. 5. `outbox.marketing` BullMQ queue depth > 0 means the dispatcher has already handed sequence trigger work to Redis; check `worker-mail`, `MARKETING_SEQUENCES_ENABLED`, and sequence subscriber errors next. Pending `marketing.*` rows in the database outbox instead mean the dispatcher still needs to be running on the `general` group. - Trial-expiring cron emits nothing — verify the subscription store actually has rows with a `trialEndsAt` inside the horizon (default 3 days). Bare Stripe-managed trials may not have a cached `trialEndsAt` column; the cron skips them silently. - Stale-lead cron over-eager — bump `MARKETING_STALE_LEAD_DAYS` (or pass a non-default `thresholdDays` to the cron when wiring) and re-run. ## Status Detail spec: `docs/improvement-plan/05-business-features/marketing-funnel-phase-3.md` Open follow-ups: - Promote the four local readiness sequence definitions into deployable demo/production seed data, with environment-specific owner approval. - Add a statistics aggregation cron and have the dashboard read persisted aggregates instead of live groupings. - Add provider delivery/open/click webhook reconciliation beyond the in-repo open/click tracking routes if operators choose Mailjet-native engagement events. - Add Playwright E2E coverage for sequence create/edit, enrollment visibility, tracking, and unsubscribe flows. - Wire/verify stale-lead/trial-expiring scheduled-job registration after choosing the canonical trial-end data source, then add deployment evidence, alerting, and runbook verification.