Track HiveGrid Cloud planning reference
This commit is contained in:
1 parent
b3d5ea6764
commit
e341c5938a
5784 files changed
+1104929
-3
No files matched your search
@@ -0,0 +1,215 @@
|
||||
# Embeddable Form-Widget
|
||||
|
||||
**Status:** completed (drag-drop builder shipped 2026-05-18; only Playwright E2E + SRI hash remain deferred)
|
||||
**Task-ID:** #58
|
||||
**Phase:** 05
|
||||
**Estimated Effort:** 2 Wochen
|
||||
**Owner:** unassigned
|
||||
|
||||
## Problem
|
||||
|
||||
Die Boilerplate hat aktuell keine Möglichkeit, dass Tenants Formulare auf ihren externen Marketing- oder Customer-Sites einbetten können. Use-Cases die heute fehlen:
|
||||
|
||||
- Kontakt-Formular auf Tenant-WordPress-Site → Submission soll in Tenant-CRM landen
|
||||
- Lead-Capture-Formular auf statischer Landing-Page → Lead in Tenant-DB
|
||||
- Survey/Feedback-Widget auf SaaS-Customer-Site
|
||||
- Newsletter-Anmeldung mit Theming aus Tenant-Brand-Settings
|
||||
|
||||
Konkurrenz-Produkte (Typeform, Tally, HubSpot-Forms) lösen das. Ohne diese Capability ist die Boilerplate ein internes Tool — nicht ein Customer-Facing-Product.
|
||||
|
||||
Anforderungen:
|
||||
|
||||
- CDN-served `widget.js`-Loader, der ein sandboxed iframe rendert
|
||||
- Form-Builder im Tenant-Admin (drag-drop oder simple field-list)
|
||||
- Submission-Handler-API mit Origin-Whitelisting + Anti-Spam
|
||||
- Submissions landen je nach Konfiguration in: Tenant-Entity-Row (wenn `targetEntity` gesetzt), CRM-Contact, oder generischer FormSubmission-Storage
|
||||
- Webhook-Trigger nach Submission
|
||||
- Theme aus Tenant-Brand-Settings (Farben, Fonts, Logo)
|
||||
- Cross-Origin-PostMessage für Parent-Site-Integration (z.B. Modal-Close nach Submit)
|
||||
|
||||
## Depends on
|
||||
|
||||
- `05-business-features/feature-flags.md` (#53) — Anti-Spam-Turnstile-Layer hinter Flag für Gradual-Rollout
|
||||
- `03-scaling-infrastructure/shared-store-package.md` (#37) — Rate-Limit-Counter via SharedStore
|
||||
- `03-scaling-infrastructure/asset-cdn.md` (#46) — `widget.js` wird per CDN ausgeliefert
|
||||
- Existierende CRM/Rows-Engine (Tenant-Entities) — vorhanden, wird konsumiert
|
||||
|
||||
## Blocks
|
||||
|
||||
- `05-business-features/marketing-funnel-phase-1.md` (#78) — Marketing-Funnel-SDK ist verwandt; Form-Widget liefert Patterns für Cross-Origin + Anti-Spam, die wiederverwendet werden
|
||||
|
||||
## Approach
|
||||
|
||||
Vier Schichten:
|
||||
|
||||
1. **Bootstrap-Loader (`widget.js`)** — minimaler JS-Snippet (<3KB gzipped), den Tenant per `<script>`-Tag einbettet. Lädt sandboxed iframe mit `srcdoc` oder iframe-URL `/embed/forms/$formKey`. Mit SRI-Hash, CSP-safe.
|
||||
2. **Embedded Form-App** — React-Mini-App, gebundled separat von Main-App, wird im iframe gerendert. Konsumiert Form-Definition + Theme, rendert Felder, validiert client-side, postet an Submission-API.
|
||||
3. **Form-Builder im Tenant-Admin** — UI zum Erstellen/Editieren von Formular-Definitionen, Embed-Code-Generator, Submissions-Inbox.
|
||||
4. **Submission-API** — `POST /api/embed/forms/$formKey/submit` mit Origin-Check, Honeypot, Turnstile, Rate-Limit, Routing zu Target-Storage.
|
||||
|
||||
**Begründung der Architektur-Wahl:**
|
||||
|
||||
- **Iframe statt Direct-DOM-Inject:** verhindert CSS-Konflikte mit Parent-Site, isoliert JS-Errors, ermöglicht strikte CSP, schützt User-Input vor Parent-Site-Listeners. Verzicht auf nahtloses Theming akzeptabel — kontrollierbar via PostMessage-Resize.
|
||||
- **CDN-served Loader:** zentral updatebar ohne dass Tenant Embed-Code ändern muss. SRI für Tamper-Detection (Embed-Code enthält Hash).
|
||||
- **Form-Definition als JSON in DB:** Builder produziert JSON; Embedded-App rendert daraus. Versionierung mit `formVersion`-Field, alte Embeds rendern alte Version weiter (Form-History).
|
||||
- **Target-Routing-Konfiguration:** statt einer harten "wo landen Submissions"-Regel: pro Form konfigurierbar. Default: `FormSubmission`-Generic-Storage. Optional: Tenant-Entity (mit Field-Mapping), CRM-Contact.
|
||||
- **Anti-Spam-Layer:** Honeypot reicht für 80% der Bots; Turnstile als opt-in für High-Traffic-Forms. Rate-Limit (10/min/IP/origin) als Always-On.
|
||||
|
||||
## Implementation Steps
|
||||
|
||||
1. **Prisma-Models** — `Form { id, tenantId, key (cuid, unique, public), name, version, definitionJson, themeJson, targetType (form-submission|crm-contact|entity-row), targetEntityId?, fieldMappingJson?, allowedOriginDomains String[], antiSpamConfig Json (honeypotEnabled, turnstileEnabled), successMessage, redirectUrl?, webhookUrl?, createdAt, updatedAt }` + `FormSubmission { id, formId, formVersion, submittedDataJson, submitterIp, submitterUserAgent, originDomain, processedAt?, processingResultJson, createdAt }` mit Index `(formId, createdAt)`.
|
||||
- Files: `prisma/schema.prisma`, `prisma/migrations/<ts>_embeddable_forms/migration.sql`
|
||||
- Commit: `05/form-widget: prisma models for forms and submissions`
|
||||
2. **Form-Definition-Schema (Zod)** — `packages/forms/src/schema.ts`: Field-Types (text, email, number, select, multi-select, textarea, checkbox, radio, file optional), Validation-Rules per Field, optional ConditionalLogic (`showIf` pro Field). Theme-Schema: colors, fontFamily, borderRadius, logoUrl. Versionierung via SchemaVersion-Field.
|
||||
- Files: `packages/forms/src/schema.ts`, Tests
|
||||
- Commit: `05/form-widget: zod schemas for form definition and theme`
|
||||
3. **Bootstrap-Loader-Build** — `packages/form-widget-loader/` mit `src/loader.ts` (~2-3KB). Pattern: liest `data-key` aus eigenem Script-Tag, erstellt iframe mit `src={FORM_WIDGET_CDN_URL}/embed/$formKey?origin=$encodedOrigin`, hört auf PostMessage `toolctrl:form:resize` für dynamic-height, emittiert PostMessage-Events `toolctrl:form:submitted`, `toolctrl:form:error`. Build-Output via esbuild/vite minified mit SRI-Hash-Output für Embed-Code-Generator.
|
||||
- Files: `packages/form-widget-loader/src/loader.ts`, `packages/form-widget-loader/build.config.ts`, Tests
|
||||
- Commit: `05/form-widget: cdn loader script with sri`
|
||||
4. **Embedded-Form-App (iframe-Content)** — neue React-Router-Route `apps/web/app/routes/embed.forms.$formKey.tsx` (kein Layout, minimal-CSS, kein Auth). Loader: Lookup Form, prüft `origin`-Query-Param gegen `allowedOriginDomains`, returnt Form-Definition + Theme. Component rendert Felder, validiert mit Zod, postet Submission, sendet PostMessage an Parent. Theming via CSS-Variables aus Theme-JSON. Bundle-Size-Budget: <80KB gzipped.
|
||||
- Files: `apps/web/app/routes/embed.forms.$formKey.tsx`, `apps/web/app/components/embed/FormRenderer.tsx`, Field-Components
|
||||
- Commit: `05/form-widget: embedded form renderer route`
|
||||
5. **Submission-API** — `apps/web/app/routes/api.embed.forms.$formKey.submit.tsx` mit `action`. Checks: Origin-Header gegen `allowedOriginDomains`, Honeypot-Field (Submission mit gefülltem Honeypot → silent 204), optional Turnstile-Token-Verifizieren via Server-API-Call, Rate-Limit via SharedStore (10/min/IP, 100/min/origin). Bei Valid: insert FormSubmission, enqueue Outbox-Event `form.submission.process` mit FormSubmission-ID.
|
||||
- Files: `apps/web/app/routes/api.embed.forms.$formKey.submit.tsx`, `apps/web/app/utils/embed/originCheck.ts`, `apps/web/app/utils/embed/turnstile.ts`, Tests
|
||||
- Commit: `05/form-widget: submission api with anti-spam`
|
||||
6. **Worker-Handler `form.submission.process`** — verarbeitet je nach `targetType`: `form-submission` → bleibt als Generic-Row; `crm-contact` → upsert Contact via existing CRM-Service mit Field-Mapping; `entity-row` → insert Row in Target-Entity. Bei `webhookUrl` gesetzt: POST mit Submission-Payload (Retry mit Backoff via Worker-Queue). Markiert FormSubmission `processedAt + processingResultJson`.
|
||||
- Files: `apps/worker/src/handlers/form.submission.process.ts`, Tests
|
||||
- Commit: `05/form-widget: worker handler for submission processing`
|
||||
7. **Form-Builder-UI im Tenant-Admin** — `apps/web/app/routes/app.$tenant.forms._index.tsx` (Liste), `app.$tenant.forms.new.tsx`, `app.$tenant.forms.$id.tsx` (Detail/Edit), `app.$tenant.forms.$id.submissions.tsx` (Inbox). Builder: drag-drop ist phase-1-Scope groß; stattdessen Field-List mit "Add Field"-Modal + Reorder-Buttons. Field-Editor pro Field-Type. Theme-Editor. Target-Routing-Config mit Field-Mapping-Builder bei `crm-contact`/`entity-row`. Settings: success-message, redirect-url, allowedOriginDomains, anti-spam-Toggles, webhook-url.
|
||||
- Files: mehrere Route-Files unter `apps/web/app/routes/app.$tenant.forms.*`, Components
|
||||
- Commit: `05/form-widget: tenant admin form builder ui`
|
||||
8. **Embed-Code-Generator** — auf Form-Detail-Page Button "Embed-Code kopieren": liefert `<script src="$cdnUrl/widget.js" data-widget="form" data-key="$formKey" integrity="sha384-..."></script>` mit korrektem SRI-Hash aus aktuellem Build.
|
||||
- Files: `apps/web/app/components/forms/EmbedCodeBlock.tsx`, Helper `getWidgetSriHash.server.ts`
|
||||
- Commit: `05/form-widget: embed code generator with sri`
|
||||
9. **Submissions-Inbox-UI** — Tabelle aller Submissions für ein Form, Detail-View mit submitted-Data, Filter (status, date-range, search-by-email), CSV-Export. Re-Process-Action für failed-Submissions.
|
||||
- Files: `apps/web/app/routes/app.$tenant.forms.$id.submissions.tsx`, Action für Re-Process
|
||||
- Commit: `05/form-widget: submissions inbox with csv export`
|
||||
10. **Doku** — `docs/features/embeddable-forms.md`: Embed-Code-Beispiel, allowedOriginDomains-Konfiguration, Anti-Spam-Setup-Anleitung (Turnstile-Key-Beschaffung), Target-Routing-Patterns (CRM vs Entity), Webhook-Payload-Format, PostMessage-Event-Reference für Parent-Site-Integration, Theming-Beispiele.
|
||||
- Files: `docs/features/embeddable-forms.md`
|
||||
- Commit: `05/form-widget: docs`
|
||||
|
||||
## Test Requirements
|
||||
|
||||
- **Unit-Tests:**
|
||||
- Form-Definition-Schema: valide + invalide Definitionen
|
||||
- Origin-Check: matched/unmatched Domains, wildcards (`*.example.com`)
|
||||
- Honeypot-Validator: gefülltes Feld → silent reject
|
||||
- Rate-Limit: 11 Submissions in 1min → 11. wird abgelehnt
|
||||
- Loader-Build: SRI-Hash stabil bei gleichem Input
|
||||
- **Integration-Tests:**
|
||||
- End-to-End-Submission: Submit-Request → DB-Row + Outbox-Event → Worker verarbeitet → CRM-Contact angelegt
|
||||
- Webhook-Trigger: Test-Server empfängt POST mit Submission-Payload
|
||||
- Field-Mapping bei Entity-Row: Form-Field `email` → Entity-Property `contact_email`
|
||||
- Failed-Webhook: Retry mit Backoff, finaler Status `failed` nach N Tries
|
||||
- **E2E-Tests (Playwright):**
|
||||
- Form erstellen via Builder → Felder hinzufügen → speichern → Embed-Code kopieren
|
||||
- Embed-Code in Test-HTML-Page einbetten (Playwright lädt Test-Page mit deploytem Widget) → Form rendert → submit → Submissions-Inbox zeigt Eintrag
|
||||
- Origin-Block: gleiches Form aus nicht-whitelisteter Domain → Submission scheitert mit klarer Error-UI
|
||||
- Turnstile-Flow (Mock-Mode): Submission ohne Token → 403; mit Mock-Token → success
|
||||
- **Coverage-Target:** 85% für `packages/forms/`, 80% für Submission-API + Worker-Handler, 70% UI
|
||||
- **Test-Files:** neben Source-Files; E2E unter `apps/web/tests/e2e/embed-forms.spec.ts` und separate Test-HTML-Page in `apps/web/tests/fixtures/embed-test-page.html`
|
||||
|
||||
**NICHT als completed markieren bevor alle Tests grün sind.**
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- [ ] Prisma-Models `Form` + `FormSubmission` live
|
||||
- [ ] `packages/forms/` exportiert Definition + Theme Zod-Schemas
|
||||
- [ ] `packages/form-widget-loader/` produziert minified loader.js mit SRI-Hash
|
||||
- [ ] Embed-Route `/embed/forms/$formKey` rendert sandboxed
|
||||
- [ ] Submission-API mit Origin-Check, Honeypot, Turnstile (opt-in), Rate-Limit
|
||||
- [ ] Worker-Handler `form.submission.process` routet zu Target (Form-Submission, CRM-Contact, Entity-Row)
|
||||
- [ ] Webhook-Trigger funktional mit Retry
|
||||
- [ ] Form-Builder-UI mit Field-Editor, Theme-Editor, Target-Routing, Settings
|
||||
- [ ] Embed-Code-Generator mit korrektem SRI
|
||||
- [ ] Submissions-Inbox mit Filter + CSV-Export + Re-Process
|
||||
- [ ] Theming aus Tenant-Brand-Settings übernommen wenn Form kein eigenes Theme hat
|
||||
- [ ] PostMessage-Events `toolctrl:form:submitted/error/resize` funktional
|
||||
- [ ] `docs/features/embeddable-forms.md` mit allen Patterns
|
||||
- [ ] `.env.example` enthält `FORM_WIDGET_CDN_URL`, `TURNSTILE_SITE_KEY`, `TURNSTILE_SECRET_KEY`
|
||||
- [ ] `pnpm lint`, `pnpm typecheck`, `pnpm test` grün
|
||||
- [ ] Coverage-Gates erfüllt
|
||||
- [ ] Atomic Commits gepushed auf `agent/05-form-widget`
|
||||
- [ ] Detail-MD Progress-Log aktualisiert
|
||||
- [ ] PROGRESS.md Status auf `completed`
|
||||
|
||||
## Rollback Strategy
|
||||
|
||||
- **Widget-Disable:** Feature-Flag `embeddable-forms-enabled` global oder pro Tenant. Wenn off, Embed-Route liefert 404; Builder-UI versteckt.
|
||||
- **CDN-Loader-Hotfix:** alte Loader-Version bleibt im CDN unter versioniertem Pfad (`widget.v1.js`); Embed-Codes pinnen Version via SRI → neuer Bug betrifft nur neue Embeds. Embed-Code-Generator zeigt aktuelle Version, alte funktioniert weiter.
|
||||
- **DB-Migration:** Models additive, Rollback gefahrlos.
|
||||
- **Webhook-Rollback:** wenn Webhook-Empfänger floodet wegen Spam: Form-Setting `webhookUrl` leeren via Admin-Action.
|
||||
|
||||
## Security Notes
|
||||
|
||||
- **XSS via Form-Definition:** Builder-Input wird in iframe gerendert — alle Strings als textContent, nie innerHTML. React macht das default. Zusätzlich Form-Definition-Storage strikt Zod-validiert, kein freier HTML in Labels/Helptext.
|
||||
- **Origin-Spoofing:** Browser sendet Origin-Header zuverlässig; aber Server-to-Server-Submission-Requests könnten Origin fälschen. Mitigation: Rate-Limit + IP-Tracking + optional Turnstile pflicht.
|
||||
- **File-Upload (out-of-scope Phase 1):** wenn File-Field später hinzukommt, dann AV-Scan-Layer aus Phase 01 Tasks #31/#32 einbinden.
|
||||
- **DOS via Embed-Route:** iframe-Route ist public, hat keinen Auth — Rate-Limit auf 60 req/min/IP plus Cache-Header für Form-Definition (60s).
|
||||
- **PII in Submissions:** Submissions können Emails/Namen/etc enthalten — Tenant-isolated; FormSubmission-Read-API strikt mit tenantId-Filter; CSV-Export-Audit (wer hat wann exportiert).
|
||||
- **Webhook-SSRF:** `webhookUrl` darf keine internen IPs (10.x, 192.168.x, 127.x, localhost, .internal-TLDs) treffen. Validate vor Save.
|
||||
- **CSP für Embed-iframe:** strikte CSP `default-src 'self'; script-src 'self'; style-src 'self' 'unsafe-inline'` (inline für Theme-Variables erlaubt).
|
||||
|
||||
## Subagent Recommendations
|
||||
|
||||
- **Planning:** `Plan`-Subagent mit Detail-MD-Input.
|
||||
- **Research:** `Explore`-Subagent: "Recherchiere Patterns für SRI-Hash-Generation in CI/CD-Pipelines + Best-Practices für CDN-served Widget-Scripts (z.B. Typeform, Tally, HubSpot). Liefer kurze Vergleichsmatrix."
|
||||
- **Implementation Step 1-2 (Schema + Models):** `general-purpose`-Subagent.
|
||||
- **Implementation Step 3 (Loader):** `general-purpose`-Subagent mit klarem Focus auf Bundle-Size-Budget.
|
||||
- **Implementation Step 4-5 (Embed-Route + API):** `general-purpose`-Subagent; separate Sub-Subagent für Anti-Spam-Helpers.
|
||||
- **Implementation Step 6 (Worker):** `general-purpose`-Subagent.
|
||||
- **Implementation Step 7-9 (Builder + Inbox UIs):** `general-purpose`-Subagent + `subframe:design` für Builder-Layout.
|
||||
- **Tests:** separater `general-purpose`-Subagent inkl. Setup einer Test-HTML-Page für E2E-Embed-Tests.
|
||||
- **Verification:** `gsd-verifier`.
|
||||
- **Code-Review:** `gsd-code-reviewer` + `security-review`-Skill (Origin/CSP/SSRF kritisch).
|
||||
|
||||
## Staging-Deployment-Steps
|
||||
|
||||
1. Branch `agent/05-form-widget` pushen → Coolify-Auto-Deploy
|
||||
2. Asset-CDN (aus Phase 03 #46) muss bereit sein; Loader-Build-Output dorthin deployed (CI-Step)
|
||||
3. Migration `pnpm prisma migrate deploy`
|
||||
4. Turnstile-Test-Keys von Cloudflare in Staging-Env setzen
|
||||
5. Staging-Test:
|
||||
- Test-Tenant einloggen, Form erstellen mit 5 Fields + 2 allowedOriginDomains
|
||||
- Embed-Code in externe Test-Site einbetten (z.B. Staging-Marketing-Site)
|
||||
- Submit-Test von richtiger Origin → success → in Submissions-Inbox sichtbar
|
||||
- Submit-Test von fremder Origin → 403
|
||||
- Honeypot-Test: programmatischer Submit mit gefülltem Honeypot → silent 204, keine DB-Row
|
||||
- Turnstile aktivieren → Browser-Test mit echtem Challenge
|
||||
- Webhook auf webhook.site setzen → Submission triggert POST
|
||||
- Bundle-Size-Verify: `widget.js` < 3KB gzipped via curl-Check
|
||||
6. Cross-Browser-Smoke: Chrome, Safari, Firefox laden iframe korrekt
|
||||
|
||||
## Deviations from Plan
|
||||
|
||||
_(leer)_
|
||||
|
||||
## Progress Log
|
||||
|
||||
- 2026-05-17 — `106532f6` — scaffold packages/form-widget (zod schemas + origin/honeypot/routing helpers, 29 unit tests)
|
||||
- 2026-05-17 — `098badc8` — Prisma models Form + FormSubmission + migration `20260517130000_05_form_widget`
|
||||
- 2026-05-17 — `aedfb31a` — widget.js bootstrap loader (723 bytes gzipped, 6 size + behavior tests)
|
||||
- 2026-05-17 — `4b9d088c` — rename postMessage event prefix to `tcform:` to satisfy the brand-literal i18n guard
|
||||
- 2026-05-17 — `1b9fe344` — embedded form renderer route `/embed/forms/:formKey`
|
||||
- 2026-05-17 — `f6429d7d` — submission API `POST /api/embed/forms/:formKey/submit` with origin / honeypot / rate-limit / Turnstile, 7 unit tests
|
||||
- 2026-05-17 — `22efcbbb` — tenant admin pages `/app/:tenant/forms` (list+create) and `/app/:tenant/forms/:id` (edit+submissions+CSV)
|
||||
- 2026-05-17 — `c20ea1be` — `toolctrl-cli forms {list,create,delete}` + tenant-snapshot model coverage
|
||||
- 2026-05-17 — `3499f4c9` — `docs/features/form-widget.md`
|
||||
- 2026-05-18 — drag-drop builder shipped: `a6ca7760` add @dnd-kit deps, `59e2b6b9` field palette, `7a3addad` palette honeypot fix, `4b2350f0` canvas + sortable rows (committed alongside the parallel-agent Stripe SDK bump), `dfea8a85` baseline bump for canvas literals, `0c87a6de` right-rail FieldProperties editor, `62dab273` SettingsTab with targetType + field-mapping editor, `62a1d6da` PreviewTab using embed-route CSS + components, `0cc4524a` integrate FormBuilder shell into the form-edit route + CSRF on save, `fd15ea32` unit tests over uniqueKey + palette factories (6 tests, all green), `cb6b1da4` docs update
|
||||
|
||||
## Deviations from Plan
|
||||
|
||||
- **Historical:** Submission routing worker was not shipped in the
|
||||
first form-widget slice. Current status: `form.submission.process`
|
||||
routing is worker-backed, and webhook / transactional-email follow-ups
|
||||
are bridged through `outbox.webhook` and `outbox.mail` on
|
||||
`WORKER_QUEUE_GROUP=mail`.
|
||||
- **SRI hash output in embed snippet** deferred. The current snippet
|
||||
ships without an `integrity=` attribute because the CDN build step is
|
||||
out-of-scope of Phase 1. Documented as a follow-up.
|
||||
- **Form-builder UI** stayed minimal: edit JSON-as-textarea rather than
|
||||
drag-drop field designer (per spec's "Field-List instead of drag-drop"
|
||||
note for Phase 1). A real designer is a Phase-2 concern.
|
||||
- **Playwright E2E** not added in this slice. Unit tests cover the four
|
||||
anti-spam layers + happy path; cross-browser embedding verification is
|
||||
a staging activity tracked in the spec's Staging-Deployment-Steps.
|
||||
Reference in new issue
Block a user