Track HiveGrid Cloud planning reference

This commit is contained in:
david committed 2026-06-26 20:48:01 +02:00
1 parent b3d5ea6764
commit e341c5938a
5784 files changed
+1104929 -3

No files matched your search

@@ -0,0 +1,186 @@
# Email Marketing Decision Report
## Decision
Decision: `extract-plugin`.
Email Marketing and Newsletter are optional plugin surfaces. They are not boilerplate Core because campaign management, subscriber/audience selection, sender configuration, tracking, and marketing consent rules are product-specific. They are also not immediate-removal targets because the current implementation has real campaign, sender, outbound-email, route, schema, translation, and CRM-contact dependencies that must be unwound atomically.
The canonical strategy is to keep Email Marketing as a plugin candidate that currently depends on CRM contacts. A future extraction must either bundle Email Marketing with the CRM plugin, declare an explicit dependency on the CRM plugin, or introduce a plugin-owned audience/list abstraction before CRM contacts or CRM entities are removed from Core.
This decision is deliberately narrower than the Email Sequences and marketing-outbound foundations. Email Marketing means the CRM-contact campaign surface under `apps/web/app/modules/emailMarketing` and the `email-marketing` admin/tenant routes. Email Sequences are documented in `docs/features/email-sequences.md` as foundation/admin-preview work with production follow-ups. The generic marketing-outbound handler is documented in `docs/features/marketing-funnel.md` as an unbootstrapped general handler foundation.
This report is documentation-only. It records the decision and safe execution order for later extraction/removal work; it does not change runtime behavior.
## Scope
In scope for this decision:
- Classify Email Marketing and Newsletter as optional plugin surfaces with action `extract-plugin`.
- Keep Email Marketing scoped to CRM-contact campaign management, not the Email Sequences foundation/admin-preview surfaces.
- Preserve transactional mail as Core and keep marketing mail outside that Core boundary.
- Document current Email Marketing routes, modules, database/schema surfaces, command/i18n references, and manifest/docs dependencies.
- Record the current CRM contact/audience dependency and the required extraction choices before CRM removal.
- Define an atomic execution plan and a static verification plan for later tasks.
Out of scope for this decision:
- Removing `apps/web/app/modules/emailMarketing`, routes, Prisma models, translations, command entries, or manifests in this slice.
- Migrating CRM, replacing contacts with a new audience model, or changing campaign send behavior in this slice.
- Moving provider secrets or adding runtime plugin packaging mechanics in this slice.
- Changing transactional email behavior in `apps/web/app/modules/emails`, `apps/web/app/application/emails`, `apps/web/app/utils/services/emailService.ts`, or `packages/mail`.
- Reclassifying Email Sequences as an Email Marketing plugin candidate or claiming production delivery before seed data, stats aggregation, E2E, Mailjet delivery evidence/reconciliation, and scheduler/deploy evidence follow-ups land.
- Bootstrapping the generic `marketing-outbound` subscriber or treating `MARKETING_WEBHOOKS_ENABLED` as evidence that Slack, Discord, Mailchimp, HubSpot, Brevo, or custom webhook delivery is active.
## Current Code Surfaces
The current known Email Marketing surfaces are:
| Surface | Current role | Future handling |
| --- | --- | --- |
| `apps/web/app/modules/emailMarketing` | Email Marketing module root for campaign, sender, outbound-email, and send services. | Move into an Email Marketing plugin or remove after routes, schema, translations, and CRM dependency are handled. |
| `apps/web/app/modules/emailMarketing/services/EmailMarketingService.ts` | Main campaign service. Imports `CrmService` and `ContactDto`, loads contact views, validates marketing subscribers, performs contact variable replacement, sends tests and campaigns, and writes outbound status. | Plugin-owned service. Must not remain in Core while importing CRM unless Email Marketing and CRM are bundled or dependency-gated. |
| `apps/web/app/modules/emailMarketing/db/emailSender.ts` | Email sender persistence surface. Stores provider, stream, API key, from/reply-to fields, and optional tenant ownership through `EmailSender`. | Plugin-owned settings/storage surface; extraction must preserve secret ownership rules and migration. |
| `apps/web/app/modules/emailMarketing/db/campaigns.db.server.ts` | Campaign persistence and campaign-recipient composition. | Plugin-owned persistence surface tied to `Campaign` and `OutboundEmail` schema. |
| `apps/web/app/modules/emailMarketing/db/outboundEmails.db.server.ts` | Outbound email tracking persistence for previews, sends, delivery, opens, clicks, bounces, spam complaints, and unsubscribes. | Plugin-owned persistence and analytics surface. |
| `apps/web/app/routes/admin/email-marketing.tsx` | Admin Email Marketing layout route. Validates CRM entities through `CrmService.validate(null)` and exposes Overview, Campaigns, Activity, and Senders navigation. | Plugin admin route shell; CRM validation must be replaced by declared plugin dependency or audience validation. |
| `apps/web/app/routes/admin/email-marketing/*` | Admin nested Email Marketing routes for overview/campaign/activity/sender workflows. | Move/delete atomically with admin layout and module services. |
| `apps/web/app/routes/app.$tenant/email-marketing.tsx` | Tenant Email Marketing layout route. Requires `tenantEmailMarketing` feature flag, validates CRM through `CrmService.validate(tenantId)`, and exposes tenant navigation. | Plugin tenant route shell; must be disabled clearly when plugin or CRM/audience dependency is unavailable. |
| `apps/web/app/routes/app.$tenant/email-marketing/*` | Tenant nested Email Marketing routes for tenant campaign/activity/sender workflows. | Move/delete atomically with tenant layout and feature-gate behavior. |
| `prisma/schema/email.prisma` | Defines inbound email models plus Email Marketing models `EmailSender`, `Campaign`, `OutboundEmail`, `OutboundEmailOpen`, and `OutboundEmailClick`. `OutboundEmail.contactRowId` references CRM contact rows through generic `Row`. | Split or migrate plugin-owned schema. Preserve Core transactional mail boundary and avoid deleting inbound/system mail accidentally. |
| `apps/web/app/services/command-registry/commandRegistry.ts` | Shared command registry named by the slice contract as a possible Email Marketing/CRM command visibility surface. | Future verifier/extraction must confirm whether Email Marketing commands exist and move/remove them through plugin command registration if present. |
| `apps/web/public/locales/en/translations.json` | Contains Email Marketing strings such as `emailMarketing.title` and sender labels used by routes. | Move plugin strings with plugin UI or remove only when UI is removed. |
| `apps/web/public/locales/de/translations.json` | German Email Marketing translation counterpart. | Move plugin strings with plugin UI or remove only when UI is removed. |
| `docs/feature-manifests/plugin-candidates.json` | Lists `email-marketing` as a plugin candidate with module/routes/emails code areas and broader plugin extraction rules. | Keep consistent with this report and later split into a dedicated plugin manifest if extraction begins. |
| `docs/feature-manifests/transactional-mail.json` | Defines transactional mail as Core and explicitly says newsletter/marketing abstractions may move to plugins. | Must remain the boundary source for Core mail. This report must not reclassify marketing mail as transactional Core. |
| `docs/feature-decisions/crm-decision-report.md` | Canonical CRM decision. It identifies Email Marketing as the most important downstream CRM dependency. | Email Marketing extraction/removal order must follow this CRM dependency contract. |
Additional inventory evidence:
- `docs/feature-inventory.md` classifies Email Marketing/Newsletter as Optional/Plugin and Core mail as Transactional Email only.
- `docs/feature-manifests/plugin-candidates.json` includes `apps/web/app/modules/emailMarketing`, `apps/web/app/routes/admin/email-marketing*`, and `apps/web/app/routes/app.$tenant/email-marketing*` under the `email-marketing` component.
- `docs/feature-manifests/transactional-mail.json` keeps password reset, invitation, verification, and system email in Core while allowing Email Marketing and Newsletter abstractions to move to plugins.
- `docs/features/email-sequences.md` keeps Email Sequences separate as foundation/admin-preview work gated by `MARKETING_SEQUENCES_ENABLED`, with production delivery blocked on explicit follow-ups.
- `docs/features/marketing-funnel.md` keeps the general `marketing-outbound` handler separate as a shipped foundation that is not bootstrapped from `apps/worker/src/index.ts`.
## Couplings
### CRM contacts and audience selection
`apps/web/app/modules/emailMarketing/services/EmailMarketingService.ts` directly imports `CrmService` and `ContactDto` from `~/modules/crm/services/CrmService`. Campaign drafts call `EntityViewsApi.get()` for entity name `contact`, load contact row IDs, call `CrmService.getContactsInRowIds()`, and reject campaigns when contacts are missing or not marked as `marketingSubscriber`. Contact previews and campaign sends call `CrmService.getContact()` and use contact fields for variable replacement.
This makes CRM the current audience provider for Email Marketing. CRM cannot be removed from Core first unless Email Marketing is also extracted with CRM, is disabled behind a clear CRM-plugin dependency, or receives a plugin-owned audience/list abstraction that replaces CRM contacts and the `marketingSubscriber` contact field.
### Rows/entities runtime
Campaign recipients use `OutboundEmail.contactRowId` in `prisma/schema/email.prisma`, and that field relates to generic `Row`. This ties marketing recipients to the rows/entities runtime and, in practice, to CRM contact rows. Future extraction may keep rows/entities as Core infrastructure, but Email Marketing must own or declare the contact/audience semantics that sit on top of rows.
### Routes and feature gating
`apps/web/app/routes/admin/email-marketing.tsx` and `apps/web/app/routes/app.$tenant/email-marketing.tsx` are layout routes with nested route directories. Both validate CRM before rendering. The tenant route also checks `appConfiguration.app.features.tenantEmailMarketing` and redirects when tenant Email Marketing is disabled. Future plugin packaging must preserve clear disabled-state behavior and avoid leaving routable pages that fail because CRM entities are missing.
### Sender, provider, and outbound services
Email Marketing owns sender configuration, campaign composition, outbound email records, tracking records, and broadcast send calls. This is a marketing-send surface, not a Core transactional mail surface. Future extraction must move these pieces together so Core password reset/invitation/system email remains independent from marketing campaign providers and tracking.
### Commands, navigation, and translations
Routes use `emailMarketing.title` and `emailMarketing.senders.plural` translation keys and email/crm icon navigation. `apps/web/app/services/command-registry/commandRegistry.ts` is the shared command-registration surface that must be audited for Email Marketing or CRM-dependent entries during extraction. Translation keys in `apps/web/public/locales/en/translations.json` and `apps/web/public/locales/de/translations.json` should move with plugin UI or be removed only when all plugin UI is removed.
### Documentation and manifests
`docs/feature-inventory.md`, `docs/feature-manifests/plugin-candidates.json`, `docs/feature-manifests/transactional-mail.json`, and `docs/feature-decisions/crm-decision-report.md` together define the boundary: Email Marketing is a plugin candidate, CRM is a plugin candidate, transactional mail remains Core, and CRM contacts currently provide the Email Marketing audience. Future implementation and verification must keep those sources consistent.
`docs/feature-manifests/plugin-candidates.json` must not use the `email-sequences` admin-preview foundation or the unbootstrapped `marketing-outbound` handler as evidence that Email Marketing has already been extracted. Email Sequences and marketing-outbound may interact with marketing queues, but they are separate status surfaces: Email Sequences are foundation/admin-preview with production followups, while marketing-outbound is a general handler foundation until a worker bootstrap wires its subscriber.
## Sender Secrets and Provider Boundary
Email Marketing sender credentials are plugin-owned marketing credentials, not Core transactional-mail credentials. `prisma/schema/email.prisma` defines `EmailSender.apiKey`, `provider`, `stream`, from-address, reply-to, tenant, campaign, and outbound-email relations. `docs/feature-manifests/plugin-candidates.json` states that plugin provider secrets must be admin/platform-owned unless explicitly tenant-owned, while `docs/feature-manifests/transactional-mail.json` states transactional mail provider API keys are platform/admin-owned and never tenant-owned.
Future extraction must therefore make ownership explicit:
1. Marketing sender API keys must move with the Email Marketing plugin and be encrypted/stored according to the platform's secret-storage standard.
2. Tenant-scoped senders may exist because `EmailSender.tenantId` is optional, but tenant ownership does not imply arbitrary tenant access to raw provider secrets.
3. Transactional mail settings for password reset, invitation, verification, and system email remain Core and must not be reused implicitly for bulk marketing campaigns.
4. Marketing sends, campaign tracking, unsubscribe handling, opens, clicks, bounces, and spam complaints are plugin responsibilities.
5. A future verifier must fail if the report or manifests blur transactional mail and marketing mail boundaries.
## Schema and Data Impact
`prisma/schema/email.prisma` contains both inbound/general email models and marketing campaign models. The marketing-owned schema impact is:
- `EmailSender` stores sender/provider configuration, API key, optional tenant ownership, and relations to campaigns/outbound emails.
- `Campaign` stores campaign metadata, content, status, sender relation, optional tenant ownership, and recipient relation.
- `OutboundEmail` stores each recipient/send record, optional campaign relation, `contactRowId`, email address, sender, preview flag, errors, sent/delivery/bounce/spam/unsubscribe timestamps, and open/click relations.
- `OutboundEmailOpen` and `OutboundEmailClick` store tracking events.
- `OutboundEmail.contactRowId` links to `Row`, which currently represents CRM contacts selected through contact entity views.
Future extraction must choose one data strategy before removing CRM contact rows:
1. Bundle with CRM: keep `contactRowId` and contact variables because CRM remains installed with Email Marketing.
2. Depend on CRM plugin: declare the dependency and fail clearly when CRM is not enabled.
3. Own audience/list data: introduce plugin-owned audience, subscriber, list, consent, and merge-field tables or row templates, then migrate campaigns/outbound records away from direct CRM contact assumptions.
Data migration must preserve campaign history, outbound delivery/tracking history, unsubscribe state, and sender configuration. Existing rows should not be silently deleted when a plugin is disabled.
## CRM Audience Decision
The current audience decision is explicit: Email Marketing depends on CRM contacts today. `EmailMarketingService.ts` uses CRM contacts as campaign recipients, enforces `marketingSubscriber`, replaces variables from CRM contact/company fields, and rejects non-contact campaign recipients. The CRM decision report also says CRM extraction must audit Email Marketing first.
Future extraction must pick exactly one path before CRM removal:
1. **Bundle path:** extract Email Marketing and CRM together as a vertical plugin where CRM contact rows remain the canonical audience.
2. **Dependency path:** extract Email Marketing separately but declare CRM as a required plugin dependency; disabled or missing CRM must produce a clear unavailable-plugin status instead of route/runtime errors.
3. **Audience abstraction path:** create an Email Marketing-owned audience/list/subscriber abstraction with consent fields, import/sync hooks, and merge-field support; migrate campaign creation and send flows away from `CrmService` and `contactRowId` before CRM is removed.
Until one path is implemented, Email Marketing must not be treated as standalone Core and CRM must not be deleted ahead of Email Marketing.
## Execution Plan
Future Email Marketing extraction/removal work must run in this atomic sequence:
1. Decision/report: land this `extract-plugin` report and the static verifier before runtime changes.
2. Manifest/docs alignment: update `docs/feature-inventory.md`, `docs/feature-manifests/plugin-candidates.json`, and any dedicated Email Marketing manifest so they point to this canonical report and agree with `docs/feature-manifests/transactional-mail.json`.
3. CRM audience decision: choose bundle, dependency, or audience abstraction. Do not remove CRM contacts, `CrmService`, or CRM entity templates before this choice is implemented.
4. Route/module packaging: move or remove `apps/web/app/modules/emailMarketing`, `apps/web/app/routes/admin/email-marketing.tsx`, `apps/web/app/routes/admin/email-marketing/*`, `apps/web/app/routes/app.$tenant/email-marketing.tsx`, and `apps/web/app/routes/app.$tenant/email-marketing/*` in one implementation slice.
5. Schema migration: move or migrate plugin-owned `EmailSender`, `Campaign`, `OutboundEmail`, `OutboundEmailOpen`, and `OutboundEmailClick` ownership while preserving transactional/inbound mail models needed by Core.
6. Sender secret boundary: migrate marketing sender provider secrets to plugin-owned settings and ensure Core transactional mail credentials remain separate.
7. Command/navigation/i18n cleanup: remove or plugin-register Email Marketing command entries in `apps/web/app/services/command-registry/commandRegistry.ts`, route/sidebar visibility, icons, and translation keys in `apps/web/public/locales/en/translations.json` and `apps/web/public/locales/de/translations.json`.
8. Tests/static scans: add focused tests and static checks so Core no longer imports Email Marketing when the plugin is disabled, transactional mail remains available, CRM dependency behavior is explicit, and docs/manifests stay consistent.
## Verification Plan
T03 must add a local static verifier, expected at `scripts/verify-email-marketing-decision-report.mjs`, and a focused test file, expected at `scripts/verify-email-marketing-decision-report.test.mjs`. The verifier must fail on the negative cases named in this task plan:
- Missing required heading: Decision, Scope, Current Code Surfaces, Couplings, Sender Secrets and Provider Boundary, Schema and Data Impact, CRM Audience Decision, Execution Plan, Verification Plan, Risks.
- Missing exact decision token `Decision: `extract-plugin``.
- Placeholder drift such as unresolved TODO/TBD placeholders outside explicit verifier-requirement wording.
- Unresolved required path references for concrete current paths and route directories.
- Manifest drift between this report, `docs/feature-inventory.md`, `docs/feature-manifests/plugin-candidates.json`, `docs/feature-manifests/transactional-mail.json`, and `docs/feature-decisions/crm-decision-report.md`.
- Missing CRM coupling statements for `CrmService`, `ContactDto`, contact views, `marketingSubscriber`, and `OutboundEmail.contactRowId`.
- Boundary drift that mixes transactional mail with marketing mail or omits sender API-key ownership.
The verifier should specifically check that this report references the current required surfaces: `apps/web/app/modules/emailMarketing`, `apps/web/app/routes/admin/email-marketing.tsx`, `apps/web/app/routes/admin/email-marketing/*`, `apps/web/app/routes/app.$tenant/email-marketing.tsx`, `apps/web/app/routes/app.$tenant/email-marketing/*`, `apps/web/app/modules/emailMarketing/services/EmailMarketingService.ts`, `apps/web/app/modules/emailMarketing/db/emailSender.ts`, `apps/web/app/modules/emailMarketing/db/campaigns.db.server.ts`, `apps/web/app/modules/emailMarketing/db/outboundEmails.db.server.ts`, `prisma/schema/email.prisma`, `apps/web/app/services/command-registry/commandRegistry.ts`, `apps/web/public/locales/en/translations.json`, `apps/web/public/locales/de/translations.json`, `docs/feature-manifests/plugin-candidates.json`, `docs/feature-manifests/transactional-mail.json`, and `docs/feature-decisions/crm-decision-report.md`.
Task-level smoke verification for this report is:
- `test -s docs/feature-decisions/email-marketing-decision-report.md`
- `grep -q "Decision: \`extract-plugin\`" docs/feature-decisions/email-marketing-decision-report.md`
Manual verification for a future runtime extraction should cover:
- Password reset, invitation, verification, and system emails still work without Email Marketing installed.
- Email Marketing routes are absent or show a clear disabled-plugin state when the plugin is disabled.
- Campaign creation and sending either work with the declared CRM dependency or use the new audience/list abstraction.
- Sender credentials are not exposed to tenants beyond intended administration workflows.
- Existing campaigns, outbound history, tracking events, unsubscribes, and sender settings remain migratable.
## Risks
- Missing the CRM contact coupling would make later CRM extraction break campaign preview/send flows.
- Mixing transactional mail with marketing mail would risk breaking password reset, invitation, verification, or system email while removing campaign features.
- Omitting sender API-key ownership would make plugin extraction unsafe because marketing credentials may be tenant-scoped in data but still require platform-grade secret handling.
- Removing Email Marketing routes without schema/data migration can leave campaigns, outbound records, tracking data, and sender secrets orphaned.
- Removing CRM contacts before an audience decision can leave campaigns without recipients and variable replacement without data.
- Translation, command, sidebar, and route-helper leftovers can keep a removed plugin visible or navigable.
- Existing tenants may rely on campaign history, unsubscribes, and sender configuration; plugin extraction should include compatibility, export, or migration notes rather than silent data loss.