Track HiveGrid Cloud planning reference
This commit is contained in:
david committed
1 parent
b3d5ea6764
commit
e341c5938a
5784 files changed
+1104929
-3
No files matched your search
@@ -0,0 +1,263 @@
|
||||
# Embeddable Form Widget
|
||||
|
||||
Tenant-authored forms that other websites can embed via a single
|
||||
`<script>` tag. Submissions land in the tenant inbox and can optionally
|
||||
route to CRM contacts, entity rows, webhooks or notification emails.
|
||||
|
||||
Phase 05 task #58 — spec:
|
||||
`docs/improvement-plan/05-business-features/form-widget.md`.
|
||||
|
||||
## Architecture
|
||||
|
||||
Four layers:
|
||||
|
||||
1. **Bootstrap loader** — hand-minified ES5 script at
|
||||
`apps/web/public/embed/widget.js` (~723 bytes gzipped). Reads
|
||||
`data-form-key` from its own `<script>` tag, inserts a sandboxed
|
||||
iframe and bridges postMessage events to the parent page.
|
||||
2. **Embedded form app** — React route at `/embed/forms/:formKey`.
|
||||
No auth, no app chrome; renders the form definition + theme inside
|
||||
the iframe.
|
||||
3. **Submission API** — POST `/api/embed/forms/:formKey/submit` with
|
||||
origin allowlist, honeypot, sliding-window rate-limit and optional
|
||||
Cloudflare Turnstile verification (feature-flag gated).
|
||||
4. **Worker routing + follow-ups** — the submit route stores a
|
||||
`FormSubmission` row, then emits `form.submission.process` through
|
||||
the outbox dispatcher to the `outbox.form-submission-route` queue.
|
||||
The routing handler, bootstrap helper and follow-up planner are
|
||||
implemented, tested, and wired from the `worker-general` entrypoint.
|
||||
Webhook follow-ups emit `webhook.form-submission.deliver`, which is
|
||||
bridged by `worker-mail` from `outbox.webhook` onto the existing
|
||||
`toolctrl.email` `deliver-webhook` processor.
|
||||
5. **Tenant admin** — list + create at `/:tenant/forms`, edit and
|
||||
submissions inbox at `/:tenant/forms/:id`. The edit page hosts a
|
||||
visual drag-drop **form-builder** (see below) backed by
|
||||
`apps/web/app/modules/formWidget/builder/`.
|
||||
|
||||
## Drag-drop form-builder
|
||||
|
||||
The form-edit page renders three tabs:
|
||||
|
||||
| Tab | What you do |
|
||||
| -------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| Builder | Add fields from the left-rail palette (click or drag-to-canvas), reorder them via pointer-drag or keyboard arrow-keys on the focused drag-handle, edit per-field properties in the right-rail panel (label, key, required, placeholder, help-text, min/max length, select/radio options). |
|
||||
| Settings | `allowedOrigins`, `notifyEmails`, `webhookUrl`, `targetType` (`generic` / `crm-contact` / `entity-row`) and a field-mapping editor that appears when `targetType` is anything other than `generic`. |
|
||||
| Preview | Live render using the **same field components + inline CSS** the embedded `/embed/forms/:formKey` route uses, so what the operator sees is exactly what visitors will see. Submission is disabled in the preview. |
|
||||
|
||||
Save persists `definition` (FormDefinition), `theme` (FormTheme),
|
||||
`allowedOrigins`, `notifyEmails`, `webhookUrl`, `targetType` and
|
||||
`fieldMapping` on the parent `Form` row in a single transaction.
|
||||
|
||||
### Field palette
|
||||
|
||||
Built-in field types (each maps to a `FormFieldType`):
|
||||
|
||||
`text`, `email`, `textarea`, `select`, `checkbox`, `radio`, `number`,
|
||||
`honeypot`. New palette entries can be added in
|
||||
`apps/web/app/modules/formWidget/builder/palette.ts`; the `defaults`
|
||||
factory must produce a value that passes `FormFieldSchema.safeParse`.
|
||||
|
||||
### Keyboard accessibility
|
||||
|
||||
Reorder works without a pointer:
|
||||
|
||||
1. Tab to the drag-handle on a field. The focus-visible ring confirms
|
||||
focus.
|
||||
2. Press `Space` to "pick up" the field (announced by the screen-reader
|
||||
via dnd-kit's built-in live region).
|
||||
3. Use `Arrow Up` / `Arrow Down` to move the field.
|
||||
4. Press `Space` again to drop, or `Escape` to cancel.
|
||||
|
||||
Every interactive button on the canvas (drag-handle, select, delete)
|
||||
exposes a templated `aria-label` so screen-readers describe each row by
|
||||
its field-label.
|
||||
|
||||
### Adding a new field-type
|
||||
|
||||
1. Append a `PaletteItem` in `palette.ts` with a fresh `type` literal
|
||||
(extend `FieldTypeSchema` in `packages/form-widget/src/schema.ts`
|
||||
first; the inferred Zod type drives both the embed renderer and the
|
||||
builder).
|
||||
2. Render the new type in `FieldProperties.tsx` (right rail) and in
|
||||
the `<PreviewField/>` component inside `PreviewTab.tsx`.
|
||||
3. Also extend the canonical embed renderer at
|
||||
`apps/web/app/routes/embed.forms.$formKey.tsx` so the customer-facing
|
||||
render matches the preview.
|
||||
|
||||
### Out-of-scope (Phase-1 deferred)
|
||||
|
||||
- Conditional logic (`showIf`) on fields
|
||||
- Multi-step / paginated forms
|
||||
- File-upload fields (requires AV-scan integration from Phase 01 #31)
|
||||
- Drag-drop Playwright E2E coverage (covered locally by unit tests
|
||||
over `uniqueKey` + palette factories)
|
||||
- SRI hash on the embed snippet (waits on the CDN build step)
|
||||
|
||||
## Embedding
|
||||
|
||||
Copy the snippet from the form-detail page or write it manually:
|
||||
|
||||
```html
|
||||
<script src="https://forms.example.com/embed/widget.js" data-form-key="<the-form-public-key>" async></script>
|
||||
```
|
||||
|
||||
Optional attributes:
|
||||
|
||||
- `data-host="https://forms.example.com"` — explicit iframe host. Defaults
|
||||
to the origin of `widget.js` itself.
|
||||
|
||||
## Anti-spam layers
|
||||
|
||||
The submission endpoint applies four layers in order. Each layer
|
||||
either rejects the request or quietly drops it (honeypot).
|
||||
|
||||
| Layer | Mechanism | Configurable |
|
||||
| ---------------- | ---------------------------------------------------------------------------------------------------------------------------- | ---------------------- |
|
||||
| Origin allowlist | `Origin` header matched against `Form.allowedOrigins`. Empty allowlist = no submissions. Supports `*.example.com` wildcards. | per-form |
|
||||
| Honeypot | A hidden field with key `hp_extra_data`. When filled, the API returns 204 silently. | always on |
|
||||
| Rate-limit | Sliding-window via `@toolctrl/shared-store`: 10/min/IP, 100/min/origin. | static |
|
||||
| Turnstile | Cloudflare CAPTCHA, opt-in per form (`config.turnstileEnabled=true`) + feature flag `form-widget-turnstile`. | per-form + global flag |
|
||||
|
||||
## Submission targets
|
||||
|
||||
`Form.targetType` chooses how a submission is processed:
|
||||
|
||||
- `generic` (default) — the submission is stored immediately; the
|
||||
routing handler marks it `processed`.
|
||||
- `crm-contact` — the routing handler maps submitted fields through
|
||||
`Form.fieldMapping`, calls the injected CRM adapter, and marks the
|
||||
submission `routed-to-crm` with the created contact id.
|
||||
- `entity-row` — the routing handler maps submitted fields through
|
||||
`Form.fieldMapping`, calls the injected entity-row adapter for
|
||||
`targetEntityId`, and marks the submission `routed-to-entity` with
|
||||
the created row id.
|
||||
|
||||
Routing is asynchronous and idempotent. The handler short-circuits rows
|
||||
that are no longer `received`, rejects tenant mismatches between the
|
||||
outbox envelope, payload, and stored submission row, writes
|
||||
`route-failed` plus a redacted error on failure, and rethrows so BullMQ
|
||||
retries and error reporting still see the issue. `apps/worker/src/index.ts`
|
||||
wires `bootstrapFormSubmissionRoute(...)` on `WORKER_QUEUE_GROUP=general`
|
||||
with the Prisma store, CRM/entity adapters, Redis worker factory, and
|
||||
follow-up publisher.
|
||||
|
||||
Webhook targets are delivered through a second queue hop: routing emits
|
||||
`webhook.form-submission.deliver`; `WORKER_QUEUE_GROUP=mail` subscribes
|
||||
to `outbox.webhook`, validates the tenant-scoped payload, then enqueues
|
||||
the existing `toolctrl.email` `deliver-webhook` job with an idempotency key
|
||||
based on the outbox event id, dispatch key, or submission id. Notification
|
||||
email follow-ups use the same pattern: routing emits
|
||||
`mail.transactional.form-submission`; `WORKER_QUEUE_GROUP=mail`
|
||||
subscribes to `outbox.mail`, validates the tenant-scoped payload, renders
|
||||
a compact operator notification, then enqueues the existing
|
||||
`toolctrl.email` `send-transactional-email` job.
|
||||
|
||||
The PII-sensitive submitter IP is never stored raw; only the
|
||||
`sha256(ip).slice(0, 32)` lives in `FormSubmission.ipHash`.
|
||||
|
||||
## Theming
|
||||
|
||||
`Form.theme` accepts a partial of:
|
||||
|
||||
```json
|
||||
{
|
||||
"primaryColor": "#2563eb",
|
||||
"backgroundColor": "#ffffff",
|
||||
"textColor": "#111111",
|
||||
"fontFamily": "system-ui, sans-serif",
|
||||
"borderRadius": "6px"
|
||||
}
|
||||
```
|
||||
|
||||
The renderer exposes these as CSS variables `--tcform-primary`,
|
||||
`--tcform-bg`, `--tcform-text`, `--tcform-font`, `--tcform-radius`.
|
||||
Empty theme inherits browser defaults; future work will pull defaults
|
||||
from `TenantSettingsRow` brand tokens.
|
||||
|
||||
## postMessage protocol
|
||||
|
||||
The iframe emits the following messages; the bootstrap loader
|
||||
re-broadcasts them on the host document as `CustomEvent`s of the same name:
|
||||
|
||||
| Type | Payload |
|
||||
| ------------------ | --------------------------------------------------------------------------------------------------------------- |
|
||||
| `tcform:resize` | `{ height, formKey }` — fires whenever the iframe content reflows. The loader clamps height to `[100, 8000]px`. |
|
||||
| `tcform:submitted` | `{ formKey }` — fires once the submission API returns `ok=true`. |
|
||||
| `tcform:error` | `{ formKey, error }` — fires on validation / 4xx / 5xx. |
|
||||
|
||||
Listen on the parent page:
|
||||
|
||||
```js
|
||||
document.addEventListener("tcform:submitted", (event) => {
|
||||
console.log("submitted!", event.detail);
|
||||
});
|
||||
```
|
||||
|
||||
## Webhook payload
|
||||
|
||||
When `Form.webhookUrl` is set, the follow-up planner emits a
|
||||
`webhook.form-submission.deliver` outbox event after successful routing.
|
||||
The payload shape is:
|
||||
|
||||
```json
|
||||
{
|
||||
"submissionId": "ckxxxxxxxxxxxxxxxxxxxxxxx",
|
||||
"formId": "ckxxxxxxxxxxxxxxxxxxxxxxx",
|
||||
"tenantId": "...",
|
||||
"webhookUrl": "https://www.example.com/hooks/form",
|
||||
"targetType": "crm-contact",
|
||||
"targetId": "ckxxxxxxxxxxxxxxxxxxxxxxx",
|
||||
"values": { "email": "alice@example.com", "...": "..." }
|
||||
}
|
||||
```
|
||||
|
||||
When `notifyEmails` is non-empty, the same follow-up planner emits
|
||||
`mail.transactional.form-submission` with `recipients`, `targetType`,
|
||||
`targetId` and `values`. The worker host publishes the returned outbox
|
||||
events transactionally during routing, and `worker-mail` bridges them
|
||||
from `outbox.mail` into `toolctrl.email`. Operator-owned delivery
|
||||
concerns include provider credentials, queue health and any
|
||||
tenant-facing delivery-status UX.
|
||||
|
||||
URLs are validated against an internal-host blocklist (`localhost`,
|
||||
`127.0.0.1`, `10.x`, `192.168.x`, `169.254.x`, `*.internal`) before
|
||||
they're allowed to save (SSRF defense).
|
||||
|
||||
## CLI
|
||||
|
||||
```
|
||||
toolctrl-cli forms list --tenant <tenantId>
|
||||
toolctrl-cli forms create --tenant <tenantId> --name "Contact" --origin https://www.example.com
|
||||
toolctrl-cli forms delete --id <formId>
|
||||
```
|
||||
|
||||
## Environment variables
|
||||
|
||||
Add to `.env`:
|
||||
|
||||
| Name | Purpose |
|
||||
| ---------------------- | ----------------------------------------------------------------------------------- |
|
||||
| `FORM_WIDGET_CDN_URL` | Public CDN/host that serves `/embed/widget.js`. Optional — falls back to `APP_URL`. |
|
||||
| `TURNSTILE_SITE_KEY` | Cloudflare Turnstile site key (front-end). Required when Turnstile is enabled. |
|
||||
| `TURNSTILE_SECRET_KEY` | Cloudflare Turnstile server secret. Required when Turnstile is enabled. |
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
- **403 "Origin not allowed"** — add the embedding site's origin to
|
||||
`Form.allowedOrigins`. Browsers send only scheme+host+port; no path.
|
||||
- **204 with no DB row** — the honeypot triggered. Check that automation
|
||||
scripts skip the `hp_extra_data` field.
|
||||
- **429** — the per-IP or per-origin sliding window kicked in. Each
|
||||
window is 60 seconds.
|
||||
- **iframe height stuck** — the parent page is blocking the loader's
|
||||
message listener. Confirm `widget.js` is loaded with `async` and not
|
||||
filtered by a strict CSP.
|
||||
|
||||
## Security notes
|
||||
|
||||
- iframe `sandbox="allow-scripts allow-forms allow-same-origin"` — no
|
||||
storage/popups/top-navigation.
|
||||
- Form definitions are Zod-validated on read; React escapes every text
|
||||
node so XSS via crafted labels is not possible.
|
||||
- IPs are stored as truncated SHA-256 hashes.
|
||||
- Webhook URLs are validated against an internal-host blocklist.
|
||||
Reference in new issue
Block a user