Refresh HiveGrid Cloud reference snapshot
This commit is contained in:
1 parent
5e99e14772
commit
7f88026f79
5955 files changed
+1177325
-188362
No files matched your search
@@ -0,0 +1,148 @@
|
||||
# Team-Chat: v1-Produkt- und UX-Vertrag
|
||||
|
||||
**Stand:** 2026-07-15
|
||||
|
||||
**Ticket:** `AK-P15b` / HIVEGRID-45
|
||||
|
||||
**Architektur:** Bespoke WebSocket-Chat nach AD-9, kein Hocuspocus
|
||||
|
||||
Dieses Dokument konkretisiert die bereits beauftragten Chat-Funktionen als belastbaren Produktvertrag. Es
|
||||
trennt notwendige Ausführungsqualität von späteren Erweiterungen und verhindert, dass die Interim-UI einen
|
||||
technisch vollständigen, aber im Alltag hakeligen Chat verdeckt.
|
||||
|
||||
## 1. Produktprinzipien
|
||||
|
||||
1. **Sofort reagieren, serverseitig entscheiden.** Senden, Reactions und Pins erscheinen optimistisch. Die
|
||||
Datenbank bleibt autoritativ; Fehler bleiben am betroffenen Element sichtbar und können dort erneut versucht
|
||||
werden.
|
||||
2. **Kein Verlust und keine Duplikate.** Jeder Schreibvorgang besitzt vor dem ersten Versuch eine stabile
|
||||
Client-ID. Wiederholungen mit derselben ID liefern dasselbe Resultat.
|
||||
3. **Realtime ist ein Beschleuniger, nicht die Datenbank.** Redis verteilt Events; nach Reconnect werden Lücken
|
||||
per Cursor aus PostgreSQL geschlossen.
|
||||
4. **Ruhe beim Lesen.** Neue Nachrichten reißen Nutzer nicht ans Listenende. Wer hochgescrollt hat, sieht einen
|
||||
Hinweis mit Anzahl und Sprungaktion.
|
||||
5. **Mobile und Tastatur sind Primärpfade.** Alle Aktionen funktionieren per Touch, Tastatur und Screenreader.
|
||||
|
||||
## 2. Nachrichten- und Reconnect-Vertrag
|
||||
|
||||
### Optimistisches Senden
|
||||
|
||||
- UI-Zustände: `sending → sent` oder `sending → failed`. Einen erfundenen gerätebezogenen
|
||||
`delivered`-Zustand gibt es in v1 nicht.
|
||||
- `clientMessageId` wird vor dem ersten Versuch erzeugt und bei Retry wiederverwendet. Der Server hält
|
||||
`(tenantId, authorUserId, clientMessageId)` eindeutig und antwortet bei Wiederholung mit derselben Nachricht.
|
||||
- Die Local-Echo-Nachricht erscheint sofort mit dezentem Sendestatus. Server-Antwort und WebSocket-Echo werden
|
||||
gegen die Client-ID abgeglichen und dürfen weder doppelte Zeilen noch doppelte Reaktionen erzeugen.
|
||||
- Fehler entfernen den Text nicht. Direkt an der Nachricht stehen „Erneut versuchen" und „Verwerfen"; kein
|
||||
Toast-Spam. Automatischer Retry gilt nur für Timeout, `429` mit `Retry-After` und `5xx`, mit Backoff und Jitter.
|
||||
- Innerhalb eines Channels/Threads bleibt die Reihenfolge stabil; unabhängige Channels dürfen parallel senden.
|
||||
|
||||
### Verbindung und Gap-Fill
|
||||
|
||||
- Sichtbare Zustände: `connecting`, `online`, `reconnecting`, `offline`, `blocked`.
|
||||
- Nach Reconnect sendet der Client den letzten bestätigten Cursor und erhält verpasste persistente Events in
|
||||
stabiler Reihenfolge. Erst danach gilt die Verbindung als vollständig synchronisiert.
|
||||
- Presence, Typing und Transportstatus sind flüchtig und werden nicht als Chat-Historie gespeichert.
|
||||
- Queue-Grenzen und `WebSocket.bufferedAmount` werden überwacht, weil der klassische Browser-WebSocket keine
|
||||
Backpressure bereitstellt.
|
||||
- Vollständiges Offline-Senden über Reload oder Background Sync ist nicht Teil von v1. Entwürfe bleiben lokal
|
||||
erhalten; ein offline ausgelöster Send bleibt sichtbar fehlgeschlagen und kann nach Reconnect wiederholt
|
||||
werden.
|
||||
|
||||
## 3. Lesen, Scrollen und Entwürfe
|
||||
|
||||
- History wird cursorbasiert paginiert. Beim Nachladen älterer Nachrichten bleibt der visuelle Scroll-Anker
|
||||
stabil.
|
||||
- Automatisches Scrollen ans Ende erfolgt nur, wenn der Nutzer bereits am Ende ist oder selbst sendet. Sonst
|
||||
erscheint „N neue Nachrichten".
|
||||
- Read-Cursor sind monoton. Eine Nachricht wird nur als gelesen gewertet, wenn Channel und Dokument aktiv sind
|
||||
und die Nachricht sichtbar war; Updates werden gebündelt. Eigene Nachrichten erzeugen kein Unread.
|
||||
- Channel zeigen getrennt Gesamt-Unread und Mention-Marker. Mute unterdrückt Benachrichtigungen, aber niemals
|
||||
Unread; direkte User-Mentions und DMs umgehen Mute gemäß aktivem Chat-Vertrag (`AK-P15b`).
|
||||
- Der Composer speichert Textentwürfe debounced pro `Tenant + User + Channel` lokal und tenant-isoliert. Ein
|
||||
Entwurf wird erst nach erfolgreichem Server-Ack gelöscht. Datei-Binaries landen nicht im Local Storage.
|
||||
|
||||
## 4. Composer und Nachrichten-QoL
|
||||
|
||||
- `Enter` sendet, `Shift+Enter` erzeugt eine neue Zeile. Während IME-Composition darf `Enter` nicht senden.
|
||||
- Markdown bleibt auf den freigegebenen Umfang beschränkt und wird serverseitig sanitisiert.
|
||||
- Eigene Nachrichten können innerhalb von 24 Stunden editiert werden; Änderungen tragen einen
|
||||
„bearbeitet"-Marker. Löschen erzeugt einen Tombstone, damit Threads und Antworten verständlich bleiben.
|
||||
- Hover-Aktionen sind identisch per Fokus und Mobile-Long-Press erreichbar: Reaktion, Thread, Bearbeiten,
|
||||
Löschen, Pin und Als-Aufgabe-markieren gemäß Berechtigung.
|
||||
- Mentions verwenden eine zugängliche Combobox mit Pfeiltasten, `Enter` und `Escape`. Autocomplete und
|
||||
Benachrichtigung zeigen beziehungsweise adressieren nur Nutzer, die den Channel lesen dürfen.
|
||||
- Reactions werden optimistisch gesetzt und bei Fehler zurückgerollt. Gleiche Reaktion eines Users auf dieselbe
|
||||
Nachricht ist eindeutig.
|
||||
- Voice Notes sind einfache Audio-Anhänge mit Aufnahme-/Upload-Fortschritt, Dauer, Abspielen, Abbrechen und
|
||||
Retry. Es gibt kein Transkript und keine KI-Verarbeitung.
|
||||
- Uploads nutzen die bestehende Files-Pipeline inklusive AV-Prüfung. Bilder unterstützen Vorschau und
|
||||
verständlichen Alternativtext; Mobile bietet den direkten Kamera-Input.
|
||||
|
||||
## 5. Presence, Typing, Threads und Pins
|
||||
|
||||
- Typing startet erst nach echter Eingabe, wird gedrosselt, läuft nach kurzer TTL aus und endet bei Senden,
|
||||
Leeren, Channelwechsel oder Disconnect. Die UI zeigt höchstens zwei Namen plus „N weitere".
|
||||
- Presence wird aus aktiven Sessions abgeleitet und läuft über TTL aus; sie ist kein belastbarer
|
||||
Arbeitszeitnachweis.
|
||||
- Threads haben genau eine Ebene. Desktop nutzt ein Seitenpanel, Mobile eine eigene Detailansicht. Wer erstellt,
|
||||
antwortet oder erwähnt wird, folgt dem Thread automatisch.
|
||||
- Message-Pins sind in Group- und Projekt-Channels auf 50 begrenzt. Nach bestätigtem Produktentscheid gibt es
|
||||
in DMs keine Message-Pins. Der persönliche Channel-Pin bleibt ein getrenntes Modell.
|
||||
|
||||
## 6. Benachrichtigungen und bestätigte v1-Grenzen
|
||||
|
||||
- Mentions, DMs und Thread-Antworten laufen über `notify()` und öffnen per Deep Link die exakte Nachricht.
|
||||
- Der allgemeine Browser-Web-Push inklusive Permission-Prompt bleibt bei `VA-P27a`; Chat-v1 liefert In-App-
|
||||
Notifications und Deep Links.
|
||||
- Tenant-Admin-Audit liegt unter `/settings/chat/audit`. Eine kurzlebige Audit-Session verlangt einen Grund;
|
||||
jeder fremde Lesezugriff wird auditiert und DMs zeigen den Audit-Banner.
|
||||
- `FileEntityType` wird minimal um `chat_message` erweitert, damit Anhänge die bestehende `FileEntityLink`-
|
||||
Pipeline nutzen und kein paralleles Datei-System entsteht.
|
||||
|
||||
## 7. Accessibility und Mobile
|
||||
|
||||
- Die Nachrichtenliste ist ein benanntes `role="log"` ohne pauschales Live-Announcement. Separate,
|
||||
dosierte `aria-live="polite"`-Statusbereiche melden nur neue fremde Nachrichten; initial geladene Historie
|
||||
und eigene Local Echoes werden nicht doppelt angesagt.
|
||||
- Verbindungs- und Sendestatus verwenden einen separaten Statusbereich und nie nur Farbe.
|
||||
- Fokus bleibt nach Send im Composer; Drawer/Dialog geben Fokus kontrolliert zurück. Channel-, Message- und
|
||||
Picker-Listen sind vollständig per Tastatur bedienbar.
|
||||
- Mobile verwendet Master/Detail-Navigation, mindestens 44 Pixel große Touch-Ziele, Safe-Area am sticky
|
||||
Composer und behält Draft sowie Scrollposition beim Channelwechsel.
|
||||
|
||||
## 8. Wiederverwendung des HiveGrid-UI-Baukastens
|
||||
|
||||
| Fläche | Vorhandene Bausteine | Neuer chat-spezifischer Verbund |
|
||||
|---|---|---|
|
||||
| Shell | `SubframeAppShell`, `PageShell`, `SidebarInside`, `ResizablePanel` | `ChatWorkspace` |
|
||||
| Channel-Liste | `SearchInput`, `Avatar`, `BadgeFeedback`, `DropdownMenu`, `ContextMenu`, `Tooltip` | `ChatChannelRow` |
|
||||
| Verlauf | `Avatar`, `Button`, `IconButton`, `ContextMenu`, `ScrollArea`, `Skeleton` | `ChatMessageList`, `ChatMessageRow` |
|
||||
| Composer | `TextArea`, `IconButton`, `Progress`, `BannerOffline` | `ChatComposer`, `ChatAttachmentTray` |
|
||||
| Threads | `Drawer`, `Dialog`, `Accordion` | `ThreadDrawer` |
|
||||
| Mentions/Suche | `SearchInput`, `Command`, `Avatar` | `MentionAutocomplete`, `ChatSearchResults` |
|
||||
| States | `EmptyState`, `Alert`, `BannerOffline`, `Skeleton`, `LoadingIndicatorSkeleton` | nur Chat-Kompositionen |
|
||||
| Medien | `uploadFileInChunks`, `AudioOrVideoPlayer`, `Progress` | Voice-Note- und Attachment-Steuerung |
|
||||
|
||||
Die bestehenden visuellen Primitives werden direkt genutzt. Nicht wiederverwendet werden die rein visuellen,
|
||||
nicht interaktiven `FileUpload`-/`Combobox`-Attrappen und der für Chat zu schwere Legacy-Rich-Text-Editor.
|
||||
|
||||
## 9. Bewusst später
|
||||
|
||||
- persistente Offline-Outbox und Background Sync über Reload
|
||||
- Scheduled Messages, Erinnerungen und „Für später speichern"
|
||||
- gerätebezogener `delivered`-Status und Read-Receipt-Privacy-Schalter
|
||||
- Slack-Power-Shortcuts, frei konfigurierbares Sendeverhalten und Pop-out-Threads
|
||||
- server-/geräteübergreifend synchronisierte Drafts
|
||||
- GIF-/Sticker-Apps, Huddles, Audio-/Videoanrufe und KI-Transkription
|
||||
|
||||
## Research-Quellen
|
||||
|
||||
- [React `useOptimistic`](https://react.dev/reference/react/useOptimistic)
|
||||
- [Matrix Client-Server API: Transaction IDs, Local Echo, Receipts, Typing, Threads](https://spec.matrix.org/latest/client-server-api/)
|
||||
- [MDN WebSocket API und fehlende Backpressure](https://developer.mozilla.org/en-US/docs/Web/API/WebSocket)
|
||||
- [Slack: Send, read and manage drafts](https://slack.com/help/articles/201457107-Send-and-read-messages)
|
||||
- [Slack: Threads](https://slack.com/help/articles/115000769927-Use-threads-to-organize-discussions)
|
||||
- [Slack: Keyboard navigation](https://slack.com/help/articles/115003340723-Navigate-Slack-with-your-keyboard)
|
||||
- [W3C ARIA23: `role=log`](https://www.w3.org/WAI/WCAG21/Techniques/aria/ARIA23)
|
||||
- [WAI-ARIA Combobox Pattern](https://www.w3.org/WAI/ARIA/apg/patterns/combobox/)
|
||||
@@ -38,9 +38,13 @@ eventHash_n = HMAC-SHA-256(AUDIT_CHAIN_SECRET, eventHash_{n-1} || canonicalJSON(
|
||||
`canonicalJSON` is the JSON encoding produced by
|
||||
`canonicalizeAuditPayload` in
|
||||
`packages/db/src/audit-hash.ts`: object keys sorted recursively,
|
||||
`undefined` dropped, arrays preserve order. Without
|
||||
`AUDIT_CHAIN_SECRET` the helper falls back to plain SHA-256 — still
|
||||
tamper-evident for read-only attackers but trivially forgeable.
|
||||
`undefined` dropped, arrays preserve order. Production fails closed unless
|
||||
`AUDIT_CHAIN_SECRET` contains at least 32 UTF-8 bytes. Local development and
|
||||
tests may use the explicit SHA-256 fallback, but it is never an accepted
|
||||
production configuration. New `v2:` hashes bind the persisted event ID and
|
||||
creation timestamp; the verifier retains dual compatibility with unversioned
|
||||
legacy hashes. Concurrent writers serialize through a per-chain PostgreSQL
|
||||
advisory transaction lock and an HMAC-authenticated O(1) chain-head row.
|
||||
|
||||
`verifyAuditChain(events, { secret })` walks the chain oldest → newest
|
||||
and returns the first break with one of:
|
||||
@@ -116,15 +120,14 @@ join with `|`; JSON metadata is stringified.
|
||||
`apps/worker/src/jobs/audit-retention.ts` exposes `runAuditRetentionJob`
|
||||
which the worker host wires to a daily cron.
|
||||
|
||||
Plan-based defaults (`resolveAuditRetentionPolicy`):
|
||||
The retention window is fixed at ten calendar years for every tenant chain and
|
||||
the platform chain. Tenant discovery uses one aggregate query. Deletes run in
|
||||
bounded, keyset-ordered batches; before each batch is deleted, the same
|
||||
transaction writes an append-only HMAC-authenticated checkpoint that binds the
|
||||
last removed event hash and the previous checkpoint MAC. Verifiers and exports
|
||||
authenticate this checkpoint before accepting the retained suffix.
|
||||
|
||||
| Regulatory scope on tenant events | Retention |
|
||||
| --------------------------------- | --------- |
|
||||
| `soc2` or `hipaa` | 7 years |
|
||||
| `gdpr` | 6 years |
|
||||
| none | 1 year |
|
||||
|
||||
Every non-empty sweep emits a meta-audit (`audit.retention.swept`) so
|
||||
Every non-empty sweep emits a meta-audit (`audit.retention.pruned`) so
|
||||
the cleanup itself is traceable.
|
||||
|
||||
## Operator Procedures
|
||||
@@ -134,8 +137,8 @@ the cleanup itself is traceable.
|
||||
1. Generate a secret: `openssl rand -base64 32`.
|
||||
2. Set `AUDIT_CHAIN_SECRET=<value>` in Coolify (platform-wide, never
|
||||
per-tenant).
|
||||
3. Apply the Prisma migration
|
||||
`20260517180000_10_audit_event_compliance`.
|
||||
3. Apply the migration that creates `AuditChainHead` and
|
||||
`AuditChainCheckpoint` before deploying the new writers.
|
||||
|
||||
### Self-service export (tenant admin)
|
||||
|
||||
|
||||
@@ -0,0 +1,143 @@
|
||||
# Customer-Portal: aktueller Produkt- und UX-Stand
|
||||
|
||||
<!-- markdownlint-disable MD013 -->
|
||||
|
||||
- **Stand:** 2026-07-22
|
||||
- **Technische Tickets:** `VA-P26a`, `VA-P26b`, `VA-P26c` jeweils 100 %
|
||||
- **Design-Nachzug:** `VD-P26` bleibt offen
|
||||
- **Staging:** `https://staging-portal.hivegrid.de`
|
||||
- **Tenant-Administration:** `https://staging-app.hivegrid.de/<tenant>/settings/customer-portal`
|
||||
|
||||
Dieses Dokument beschreibt das aktuell ausgelieferte Customer-Portal aus Sicht von Workspace-Admins
|
||||
und externen PortalUsern. Der Betriebs-, Secret- und Rollout-Vertrag steht separat in
|
||||
[`docs/deploy/customer-portal.md`](../deploy/customer-portal.md).
|
||||
|
||||
## 1. Produktgrenze
|
||||
|
||||
Das Customer-Portal ist eine eigene, extern sichtbare Oberfläche. PortalUser sind strikt von internen
|
||||
HiveGrid-Nutzern getrennt und erhalten nur Zugriff auf ausdrücklich zugewiesene Projekte, sichtbare
|
||||
Projektphasen und freigegebene Dokumente. Fehlende Berechtigungen, fremde Projekte und unbekannte Hosts
|
||||
liefern 404, damit keine Existenz fremder Daten offengelegt wird.
|
||||
|
||||
Core und Portal laufen als getrennte Runtime-Ressourcen aus demselben unveränderlichen Web-Image. Das
|
||||
Portal besitzt eigene Sessions, eine minimal berechtigte Datenbankrolle und keine internen Admin-, OAuth-,
|
||||
Billing- oder Ingress-Zugangsdaten.
|
||||
|
||||
## 2. Endkunden-Navigation und Hauptreisen
|
||||
|
||||
Die Portaloberfläche verwendet dieselbe HiveGrid-Shell wie die SaaS: gleich große Wortmarke, einklappbare
|
||||
Sidebar, Sidebar-Inset, Breadcrumbs, Seitenkopf, Avatar und mobile Navigation. Die Hauptnavigation ist
|
||||
vollständig klickbar und bewusst auf wenige Ziele reduziert:
|
||||
|
||||
- **Übersicht:** eigene Projekte, aktueller Status und sinnvolle nächste Aktionen.
|
||||
- **Projekte:** Liste der freigegebenen Projekte mit Fortschritt und letztem Update.
|
||||
- **Projekt:** kompakte Detailansicht mit den Bereichen Übersicht, Dateien und Anliegen.
|
||||
- **Upload:** projektbezogener Multi-Datei-Upload mit den konfigurierten Anzahl- und Größenlimits.
|
||||
- **Einstellungen:** Benachrichtigungen, Profildaten, DSGVO-Export und Kontolöschung; der Einstieg liegt
|
||||
getrennt von den Arbeitszielen unten in der Sidebar.
|
||||
|
||||
Projektstatus wird als verständliche Abfolge sichtbarer Phasen dargestellt. Freigegebene Dateien können
|
||||
heruntergeladen werden. PortalUser können Dateien hochladen und Anliegen beziehungsweise Reklamationen
|
||||
melden, sofern ihr Projektzugang dies erlaubt. Ein eingefrorener Workspace bleibt lesbar, verhindert aber
|
||||
Schreibaktionen mit einem Klartext-Hinweis.
|
||||
|
||||
## 3. Zugang und Einladungen
|
||||
|
||||
Ein berechtigter Workspace-Admin kann PortalUser aus drei fachlichen Einstiegen einladen:
|
||||
|
||||
1. aus der Customer-Portal-Verwaltung,
|
||||
2. aus einem Kontakt,
|
||||
3. aus einem Projekt; dabei wird der Projektzugang atomar vergeben.
|
||||
|
||||
Die Einladung verwendet einen einmaligen, gehashten Setup-Token mit sieben Tagen Laufzeit. Nach
|
||||
Datenschutzannahme und optionaler Passwortvergabe entsteht eine eigene Portal-Session. Magic-Link ist der
|
||||
primäre Login-Pfad, Passwort der sekundäre. Replays, fremde Tenant-/Projektzuordnungen und doppelte
|
||||
Einladungszustände werden fail-closed abgewiesen.
|
||||
|
||||
## 4. Tenant-Administration
|
||||
|
||||
Die zentrale Verwaltung liegt unter `/<tenant>/settings/customer-portal`. Sichtbarkeit und Aktionen werden
|
||||
über `portal.read`, `portal.manage_users`, `portal.manage_access`, `portal.activity.*`,
|
||||
`portal.delete_user` und `portal.admin` begrenzt.
|
||||
|
||||
Je nach Berechtigung kann der Workspace-Admin:
|
||||
|
||||
- Portal veröffentlichen oder einfrieren und den Lifecycle konfigurieren,
|
||||
- erlaubte Projekttypen, sichtbare Phasen und kundenseitige Phasenbezeichnungen festlegen,
|
||||
- Dokument-Auto-Freigaben und Upload-Limits verwalten,
|
||||
- Statusmails sowie Impressum-, Datenschutz- und AGB-Links konfigurieren,
|
||||
- Kunden und Subunternehmer einladen, erneut einladen oder deaktivieren,
|
||||
- projektbezogene Zugriffe, Upload-Rechte, Laufzeiten und sichtbare Inhalte festlegen,
|
||||
- Portalaktivitäten in der Service-Inbox bearbeiten,
|
||||
- Portal-Paket und optional eine Custom-Domain verwalten.
|
||||
|
||||
## 5. Tenant-Logo und Fallback
|
||||
|
||||
Ein Workspace-Admin mit `portal.admin` kann in den Portal-Einstellungen ein eigenes Portal-Logo
|
||||
hochladen, direkt als Vorschau prüfen und wieder entfernen.
|
||||
|
||||
- akzeptierte Formate: PNG, JPEG und WebP,
|
||||
- maximale Größe: 5 MB,
|
||||
- Prüfung: restriktive Bildsignatur und verpflichtender ClamAV-Scan vor Speicherung,
|
||||
- Speicherung: privates, tenant-isoliertes Objekt mit deterministischem Schlüssel,
|
||||
- Auslieferung: kurzlebige, serverseitig signierte URL; kein öffentlicher Bucket,
|
||||
- Cache-/Leak-Schutz: `private, no-store`, `no-referrer` und `nosniff`,
|
||||
- Fallback: fehlt das Tenant-Logo oder kann es nicht geladen werden, erscheint automatisch die normale
|
||||
HiveGrid-Wortmarke; in der eingeklappten Sidebar erscheint das HiveGrid-Signet.
|
||||
|
||||
Dasselbe Branding gilt auf dem Standardhost und auf einer aktivierten Custom-Domain. Es gibt aktuell kein
|
||||
frei konfigurierbares Theme, keine eigenen Fonts und kein vollständiges White-Labeling.
|
||||
|
||||
## 6. Hosts und Custom-Domain
|
||||
|
||||
Der Standardhost ist immer der Fallback:
|
||||
|
||||
- Staging: `https://staging-portal.hivegrid.de/<tenant>/...`
|
||||
- Produktion: `https://portal.hivegrid.de/<tenant>/...`
|
||||
|
||||
Auf einer verifizierten Custom-Domain entfallen Tenant-Slug und HiveGrid-Standardhost aus der sichtbaren URL;
|
||||
die fachlichen Seiten und das Tenant-Branding bleiben identisch. Workspace-Admins erhalten TXT-/CNAME-
|
||||
Anleitung, Status und Fehlerhilfe. Der Domain-Controller provisioniert ausschließlich exakte, verifizierte
|
||||
Hosts inklusive TLS. Entfernung, Entitlement-Verlust und Kill-Switch deaktivieren den Host; ein 30-tägiger
|
||||
Reclaim-Cooldown verhindert eine schnelle Fremdübernahme.
|
||||
|
||||
## 7. Bewusste UX-Entscheidungen des Rescue-Stands
|
||||
|
||||
Der UI-Nachzug aus PR `#212` reduziert visuelle Konkurrenz und richtet das Portal an der bestehenden
|
||||
HiveGrid-SaaS aus:
|
||||
|
||||
- keine doppelte Kopf- und Hauptnavigation,
|
||||
- Einstellungen getrennt von den täglichen Arbeitszielen,
|
||||
- weniger Dashboard-Karten und Wiederholungen,
|
||||
- Projektfortschritt und Tabs ohne Überlagerungen oder übergroße Leerflächen,
|
||||
- kompakter Datei-Upload mit klarer Auswahl, Aktion und Upload-Historie,
|
||||
- konsistente Abstände, Typografie, Status-Badges, Empty-/Loading-/Fehlerzustände und mobile Breakpoints.
|
||||
|
||||
Die lokale Demo-Route dient ausschließlich der visuellen Entwicklung und ist im Production-Build nicht
|
||||
registriert. Der aktuelle Screen bleibt formal die verbesserte AD-28-Interim-UI. `VD-P26` bleibt deshalb als
|
||||
finaler Subframe-/Design-Polish offen; der technische Funktionsumfang von `VA-P26a/b/c` bleibt abgeschlossen.
|
||||
|
||||
## 8. Release-Nachweis
|
||||
|
||||
- UI-/Branding-PR: [`#212`](https://git.tool-ctrl.de/toolctrl-labs/HiveGrid-Cloud/pulls/212)
|
||||
- Main-Revision: `090e2558a6965e1c5804eb24d3eab5da8c8611a8`
|
||||
- Staging-Deploy: [`#11018`](https://git.tool-ctrl.de/toolctrl-labs/HiveGrid-Cloud/actions/runs/11018)
|
||||
- CI: 13/13 PR-Jobs grün; Typecheck, Lint, Core-Tests, Visual-, A11y-, i18n-, Hot-Swap-, Bundle- und
|
||||
Docker-Gates grün.
|
||||
- Core und Portal verwenden denselben Web-Digest
|
||||
`sha256:064bd20e3e1694a1162198f772406b1aea10e3f7f7b73f877b6e4f0246c041e1`.
|
||||
- `staging-app.hivegrid.de` und `staging-portal.hivegrid.de` lieferten nach dem Rollout auf `/healthz`
|
||||
und `/readyz` HTTP 200 mit der exakten Main-Revision. Der Domain-Controller akzeptierte den Release;
|
||||
der Recovery-Pfad führte keinen Rollback aus.
|
||||
- Produktion, Plane-Tickets, DNS-Zustände, Schlüssel und Passwörter wurden durch diesen UI-Release nicht
|
||||
verändert.
|
||||
|
||||
## 9. Maßgebliche Codepfade
|
||||
|
||||
- Portal-Shell und Screens: `apps/web/app/components/screens/customer-portal/`
|
||||
- Portal-Routen: `apps/web/app/routes/customer-portal/`
|
||||
- Tenant-Verwaltung: `apps/web/app/routes/app.$tenant/settings/customer-portal.tsx`
|
||||
- Branding-Service: `apps/web/app/services/customer-portal/portal-branding.server.ts`
|
||||
- Portal-Domain-Controller: `apps/portal-domain-controller/`
|
||||
- View-Models: `packages/view-models/src/customer-portal/`
|
||||
- Visual-Baselines: `packages/ui-mock/visual-baseline/va-p26*-customer-portal-*`
|
||||
@@ -1,11 +1,13 @@
|
||||
# DSGVO Bulk-Ops (Self-Service Export, Deletion, Pseudonymisation)
|
||||
|
||||
**Phase:** 10 — Compliance Polish
|
||||
**Status:** active
|
||||
**Status:** legacy / dormant
|
||||
|
||||
This document describes the runtime architecture of the DSGVO bulk-ops
|
||||
pipeline that delivers the user-facing rights under GDPR Art. 15, 17
|
||||
and 20.
|
||||
This document describes a legacy bulk-ops implementation that is not wired into
|
||||
the current worker runtime. The active user-facing export paths are the P21
|
||||
privacy export and customer-portal export services: they store bounded objects
|
||||
with SSE-C and deliver them only through authenticated `200` app routes. Do not
|
||||
use this document as deployment evidence for those paths.
|
||||
|
||||
## Schema
|
||||
|
||||
@@ -62,8 +64,8 @@ interface identifier (/48) for `AuditEvent.actorIp` rewrites.
|
||||
|
||||
## Worker Pipeline
|
||||
|
||||
Three worker jobs (`apps/worker/src/jobs/dsgvo-*.ts`) execute the
|
||||
heavy lifting:
|
||||
Three dormant worker jobs (`apps/worker/src/jobs/dsgvo-*.ts`) describe the
|
||||
legacy implementation:
|
||||
|
||||
### 1. `dsgvo-export.ts`
|
||||
|
||||
@@ -86,9 +88,10 @@ Bundle layout:
|
||||
}
|
||||
```
|
||||
|
||||
After completion a presigned URL is minted (7-day validity), emailed
|
||||
to the user and audited as `EXPORT_PERSONAL_DATA` with
|
||||
`regulatoryScope=["gdpr"]`, `dataSensitivity="pii"`.
|
||||
The legacy code mints a presigned URL with seven-day validity and emails it to
|
||||
the user. It is not runtime-wired and must not be enabled as a fallback for the
|
||||
active SSE-C P21/portal export contract. Any future reactivation requires a
|
||||
separate security review and migration to the current authenticated proxy.
|
||||
|
||||
### 2. `dsgvo-deletion.ts`
|
||||
|
||||
|
||||
@@ -18,12 +18,8 @@
|
||||
| Local readiness seed | `scripts/seed-e2e-readiness.ts` | deterministic welcome / trial-expiring / stale-lead / customer-welcome definitions for local E2E fixtures |
|
||||
|
||||
This page claims active ownership of the email-sequence subscriber on
|
||||
`outbox.marketing`.
|
||||
It does not classify Email Sequences as the `email-marketing` CRM-contact
|
||||
campaign plugin candidate from
|
||||
`docs/feature-decisions/email-marketing-decision-report.md` or
|
||||
`docs/feature-manifests/plugin-candidates.json`. Treat Email Sequences as
|
||||
foundation/admin-preview work until the production follow-ups below land.
|
||||
`outbox.marketing`. It does not implement the tenant Email Marketing scope from `AK-P14*`.
|
||||
Treat Email Sequences as foundation/admin-preview work until the production follow-ups below land.
|
||||
The generic marketing integration handler/adapters for Slack, Discord,
|
||||
Mailchimp, HubSpot, Brevo, and custom webhooks are documented in
|
||||
`docs/features/marketing-funnel.md`; as of this status reconciliation that
|
||||
|
||||
@@ -227,7 +227,7 @@ they're allowed to save (SSRF defense).
|
||||
```
|
||||
hivegrid-cli forms list --tenant <tenantId>
|
||||
hivegrid-cli forms create --tenant <tenantId> --name "Contact" --origin https://www.example.com
|
||||
hivegrid-cli forms delete --id <formId>
|
||||
hivegrid-cli forms delete --tenant <tenantId> --id <formId>
|
||||
```
|
||||
|
||||
## Environment variables
|
||||
|
||||
@@ -1,404 +1,108 @@
|
||||
# Marketing-Funnel Overview
|
||||
# Plattform-Lead-Eingang und historischer Marketing-Funnel
|
||||
|
||||
This page is the canonical overview for the shipped lead-capture and
|
||||
self-service conversion foundation. Email sequences are covered separately in
|
||||
`docs/features/email-sequences.md`.
|
||||
Seit R-LEADS (`HIVEGRID-73`, AD-31) ist der native Plattform-Lead-Eingang die
|
||||
einzige aktive Zielarchitektur für HiveGrid-eigene Vertriebsanfragen. Diese Datei
|
||||
dokumentiert den aktuellen Vertrag und die während des Cutovers bewusst noch
|
||||
vorhandenen Legacy-Kompatibilitätspfade.
|
||||
|
||||
## Aktive Architektur
|
||||
|
||||
| Ebene | Aktiver Vertrag |
|
||||
| ---------------- | ------------------------------------------------------------------------------------------------------------------------ |
|
||||
| Datenmodell | `AdminLead`, `AdminLeadActivity`, `AdminLeadSource` und gehashte Source-Credentials in `prisma/schema/adminLeads.prisma` |
|
||||
| Eingang | `POST /v1/platform/leads` auf dem API-Host |
|
||||
| Producer | ausschließlich serverseitig; ein Source-Credential pro freigeschalteter Website/Partnerquelle |
|
||||
| Admin-Inbox | `/leads`, `/leads/new`, `/leads/:id` und `/leads/sources` auf dem Admin-Host |
|
||||
| Benachrichtigung | tenantlose, PII-minimierte M011-Outbox mit Worker-Retry und aktueller Empfänger-Revalidierung |
|
||||
|
||||
Das Source-Credential bestimmt die Quelle. Ein Request darf die Source nicht
|
||||
selbst autoritativ wählen. `Idempotency-Key` ist Pflicht und muss von einem
|
||||
Producer für dieselbe Submission bis zur bestätigten Antwort stabil
|
||||
wiederverwendet werden. Kontakt-, Nachrichten- und Attributionsfelder werden
|
||||
schema- und größenvalidiert; Logs, Outbox und Queue enthalten keine Lead-PII.
|
||||
|
||||
Der API-Eingang kann im Notfall mit `PLATFORM_LEADS_INGRESS_ENABLED=false`
|
||||
neutral als 404 abgeschaltet werden. Die Admin-Oberfläche ist davon unabhängig
|
||||
standardmäßig aktiv und wird über AdminUser, Permissions und Own-/All-RLS geschützt.
|
||||
|
||||
Details zu Source-Bootstrap, Backfill, Canary, Rollback und Decommission stehen in
|
||||
`docs/ops/platform-leads-cutover.md`. Der vollständige Bauvertrag steht in
|
||||
`.planning/ticket-docs/R-LEADS.md`.
|
||||
|
||||
## Aktiver API-Vertrag
|
||||
|
||||
`POST /v1/platform/leads` erwartet:
|
||||
|
||||
- `Authorization: Bearer <source credential>`
|
||||
- `Idempotency-Key: <stabile submission id>`
|
||||
- JSON mit mindestens einem nutzbaren Kontakt- oder Inhaltsfeld
|
||||
- optionale, begrenzte UTM-, Referrer-, Landing-Page- und Metadatenfelder
|
||||
|
||||
Relevante Antworten sind `201` für Create und identischen Replay, `400` für
|
||||
Validierung, `401` für ein ungültiges oder revoktes Credential, `409` für einen
|
||||
Idempotency-Hash-Konflikt, `413` für einen zu großen Body und `429` für das
|
||||
Source-/Ingress-Limit. OpenAPI ist die kanonische Feldreferenz.
|
||||
|
||||
Browser und statische Marketing-Sites erhalten niemals das Source-Credential.
|
||||
Die HiveGrid-Website besitzt deshalb einen serverseitigen Proxy/Action-Handler.
|
||||
Ein fehlgeschlagener API-Request darf dort
|
||||
nicht als erfolgreicher Lead angezeigt oder in eine Container-SQLite-Datei
|
||||
zurückgefallen werden.
|
||||
|
||||
## Retired: Browser-Capture und Embed-SDK
|
||||
|
||||
Die folgenden Komponenten sind kein aktiver Lead-Eingang mehr:
|
||||
|
||||
- `POST /api/leads/capture` liefert während des Drain-Fensters bewusst `410 Gone`
|
||||
und verweist auf `/v1/platform/leads`; `OPTIONS` bleibt nur für eine saubere
|
||||
CORS-Fehlermeldung verfügbar.
|
||||
- `apps/web/public/embed/marketing.js` ist entfernt. Es gibt kein öffentliches
|
||||
`window.hivegridMarketing`-SDK und keinen `data-hivegrid-lead`-Hook mehr.
|
||||
- `/marketing-leads`, `/incoming-leads-board` und `/marketing-funnels` leiten auf
|
||||
die native `/leads`-Inbox um.
|
||||
- Die Legacy-Tabellen `Lead`/`LeadEvent` sind bis zum Ende des Rollback-/Drain-
|
||||
Fensters read-only. Verbleibende CLI-Reads dienen ausschließlich Export und
|
||||
Rollback; sie sind kein zweites aktives Produkt-Frontend und werden nach dem
|
||||
bestätigten Sieben-Tage-Drain entfernt.
|
||||
|
||||
`MARKETING_ALLOWED_ORIGINS` beeinflusst nur noch die CORS-Header der
|
||||
`410 Gone`-Antwort. `MARKETING_RATE_LIMIT_PER_MIN` ist für den retired Endpoint
|
||||
wirkungslos und bleibt vorübergehend nur als kompatibler, deprecated Env-Key im
|
||||
Konfigurationskatalog.
|
||||
|
||||
## Historische Conversion-Kompatibilität
|
||||
|
||||
Die signierte Self-Service-Conversion bestehender Legacy-Leads bleibt während
|
||||
des Rollback-Fensters bewusst separat erhalten. Sie darf keine neuen
|
||||
`Lead`/`LeadEvent`-Captures erzeugen. Die SECURITY-DEFINER-Funktion
|
||||
`hivegrid_convert_self_service_lead` ist nur für den signierten Registrierungsweg
|
||||
freigegeben; direkte Runtime-Schreibrechte auf Legacy-Tabellen bleiben entzogen.
|
||||
|
||||
`LEAD_CONVERSION_SECRET` und `LEAD_CONVERSION_ENABLED` gehören zu diesem
|
||||
Kompatibilitätspfad. Eine Überführung dieser Conversion in die spätere
|
||||
`AdminDeal`-/M040b-Welt ist nicht Teil von R-LEADS.
|
||||
|
||||
## Outbound- und E-Mail-Marketing-Abgrenzung
|
||||
|
||||
Die vorhandenen Marketing-Webhook-Adapter und die E-Mail-Sequenz-Engine sind
|
||||
eigenständige Foundations. Sie machen den retired Browser-Capture nicht wieder
|
||||
aktiv und dürfen nicht automatisch von einem AdminLead-Import ausgelöst werden.
|
||||
Die jeweiligen Runtime-Schalter und Betriebsgrenzen stehen in
|
||||
`docs/features/email-sequences.md` und den zugehörigen Worker-Dokumenten.
|
||||
|
||||
This page also keeps the marketing-outbound boundary explicit: the
|
||||
`packages/marketing-outbound` adapters and
|
||||
`apps/worker/src/jobs/marketing-outbound.ts` handler are a shipped general
|
||||
handler foundation, but the general subscriber is not bootstrapped from
|
||||
`apps/worker/src/index.ts`. That foundation is separate from the
|
||||
`email-marketing` CRM-contact campaign plugin candidate in
|
||||
`docs/feature-decisions/email-marketing-decision-report.md`.
|
||||
|
||||
The marketing-funnel ships a four-layer pipeline that lets external landing
|
||||
pages, ads, and blog CTAs feed structured leads into the operator-side admin
|
||||
inbox without an external CRM.
|
||||
|
||||
| Layer | Component |
|
||||
|------------------|------------------------------------------------------------|
|
||||
| 1. Data model | `Lead` + `LeadEvent` Prisma tables (`prisma/schema/marketingLeads.prisma`) |
|
||||
| 2. Ingestion API | `POST /api/leads/capture` (`apps/web/app/routes/api.leads.capture.tsx`) |
|
||||
| 3. SDK | `apps/web/public/embed/marketing.js` (origin-served mini-snippet; CDN-ready while A7/B2 remains backlog) |
|
||||
| 4. Admin UI | `/admin/marketing-leads` (list) + `/admin/marketing-leads/:id` (detail) |
|
||||
|
||||
## Status machine
|
||||
|
||||
Leads move through seven states:
|
||||
|
||||
```
|
||||
new → contacted → qualified → trial → customer
|
||||
↘ churned
|
||||
↘ disqualified
|
||||
```
|
||||
|
||||
Transitions are recorded as `LeadEvent` rows of type `status-changed` for the
|
||||
activity-timeline in the detail-view. Operators move leads forward (or to a
|
||||
terminal state) from the detail page. The `score`, `notes`, and `convertedAt`
|
||||
columns are operator-managed.
|
||||
|
||||
## Attribution
|
||||
|
||||
Every Lead carries first-touch attribution columns populated by the SDK on
|
||||
the initial visit and persisted in localStorage so subsequent page visits
|
||||
attribute to the same source:
|
||||
|
||||
| Column | Source |
|
||||
|---------------|-----------------------------------------------------|
|
||||
| `utmSource` | `?utm_source=…` |
|
||||
| `utmMedium` | `?utm_medium=…` |
|
||||
| `utmCampaign` | `?utm_campaign=…` |
|
||||
| `utmTerm` | `?utm_term=…` |
|
||||
| `utmContent` | `?utm_content=…` |
|
||||
| `gclid` | `?gclid=…` (Google Ads click identifier) |
|
||||
| `fbclid` | `?fbclid=…` (Meta Ads click identifier) |
|
||||
| `referrer` | `document.referrer` |
|
||||
| `landingPage` | `location.href` at the first visit |
|
||||
| `firstSeenAt` | `new Date()` on initial insert (never overwritten) |
|
||||
| `lastTouchAt` | `new Date()` on every re-capture |
|
||||
|
||||
`source` is operator-supplied (via the form's `data-source` attribute or the
|
||||
SDK call's `source` field) and identifies the surface — `"landing-pro"`,
|
||||
`"blog-cta"`, `"webinar-form"` — distinct from the paid-channel `utmSource`.
|
||||
|
||||
## Anti-spam (defence in depth)
|
||||
|
||||
The capture endpoint stacks four layers, each weakening different bot
|
||||
classes; none of them is sufficient alone.
|
||||
|
||||
1. **Origin allowlist** — `MARKETING_ALLOWED_ORIGINS` (CSV with `*.host`
|
||||
wildcards). 403 on miss. Defeats browser-based form-spam from
|
||||
non-whitelisted pages; can be spoofed by server-to-server bots.
|
||||
2. **Honeypot** — the SDK injects a hidden `website_url` field. When the
|
||||
server sees it filled, it responds with a silent 204 and inserts nothing.
|
||||
80%+ of crude form-fillers fall here.
|
||||
3. **Sliding-window rate-limit** — 5/min per IP (configurable via
|
||||
`MARKETING_RATE_LIMIT_PER_MIN`) and 1/min per email. The per-email limit
|
||||
doubles as a re-capture dedupe so the LeadEvent timeline doesn't flood.
|
||||
4. **Cloudflare Turnstile** (optional) — gated behind the
|
||||
`marketing-funnel-turnstile` feature flag. When enabled, callers must
|
||||
include a valid `turnstileToken`; verification hits the Cloudflare API.
|
||||
|
||||
Successful responses are intentionally identical for new and duplicate
|
||||
emails (empty body, 200 OK) so bots cannot enumerate the customer list.
|
||||
|
||||
## API contract
|
||||
|
||||
### `POST /api/leads/capture`
|
||||
|
||||
Public, CORS-enabled. Accepts `application/json` or
|
||||
`application/x-www-form-urlencoded`. Body:
|
||||
|
||||
```json
|
||||
{
|
||||
"email": "alice@example.com",
|
||||
"firstName": "Alice",
|
||||
"lastName": "Liddell",
|
||||
"phone": "+44 …",
|
||||
"company": "Acme",
|
||||
"source": "landing-pro",
|
||||
"attribution": {
|
||||
"utmSource": "google",
|
||||
"utmCampaign": "spring-launch",
|
||||
"gclid": "EAIaIQ…",
|
||||
"referrer": "https://google.com/",
|
||||
"landingPage": "https://example.com/landing"
|
||||
},
|
||||
"turnstileToken": "0.OOOO…",
|
||||
"website_url": ""
|
||||
}
|
||||
```
|
||||
|
||||
Responses:
|
||||
|
||||
| Status | Meaning |
|
||||
|--------|--------------------------------------------------|
|
||||
| 200 | Captured (or silently de-duplicated). Empty body.|
|
||||
| 204 | Honeypot triggered or per-email rate-limit hit. |
|
||||
| 400 | Invalid payload shape. |
|
||||
| 403 | Origin not allowlisted or Turnstile failed. |
|
||||
| 413 | Body exceeds 32 KiB. |
|
||||
| 429 | Per-IP rate-limit exceeded. |
|
||||
|
||||
### `OPTIONS /api/leads/capture`
|
||||
|
||||
CORS preflight. Echoes the matched origin in `Access-Control-Allow-Origin`.
|
||||
|
||||
## SDK (`marketing.js`)
|
||||
|
||||
Drop the script tag onto every marketing page:
|
||||
|
||||
```html
|
||||
<script src="https://cdn.example.com/embed/marketing.js"
|
||||
data-api-url="https://app.example.com" defer></script>
|
||||
```
|
||||
|
||||
The script self-bootstraps on `DOMContentLoaded`:
|
||||
|
||||
- Reads `?utm_source/medium/campaign/term/content`, `?gclid`, `?fbclid` from
|
||||
the URL and merges them with the stored first-touch attribution.
|
||||
- Persists merged attribution to `localStorage["__tcm_attr"]` so multi-page
|
||||
flows hit the same lead source.
|
||||
- Auto-binds any `<form data-hivegrid-lead>` on the page — submitting
|
||||
POSTs to `/api/leads/capture` instead of the form's action.
|
||||
|
||||
Imperative API exposed as `window.hivegridMarketing`:
|
||||
|
||||
```js
|
||||
hivegridMarketing.captureLead({
|
||||
email: "alice@example.com",
|
||||
firstName: "Alice",
|
||||
source: "blog-cta",
|
||||
});
|
||||
|
||||
hivegridMarketing.bindFormById("lead-form");
|
||||
hivegridMarketing.bindFormsByAttr();
|
||||
hivegridMarketing.getAttribution(); // → merged first-touch object
|
||||
hivegridMarketing.resetAttribution(); // wipes localStorage cache
|
||||
```
|
||||
|
||||
Custom events on bound forms:
|
||||
|
||||
- `tcm:captured` — `{ ok: boolean, status: number }`
|
||||
- `tcm:error` — `{ error: string }`
|
||||
|
||||
The SDK is hand-minified to stay under the 3 KB gzipped budget (currently
|
||||
~2 KB) and ships from `apps/web/public/embed/marketing.js`.
|
||||
|
||||
## Admin UI
|
||||
|
||||
- **List** at `/admin/marketing-leads` — filter by status (seven badge
|
||||
shortcuts), source, or search by email/name/company. Paginated 25 per
|
||||
page. Permission: `admin.marketing.leads.view`.
|
||||
- **Detail** at `/admin/marketing-leads/:id` — profile + attribution
|
||||
panels, status transition buttons (every state reachable via one click),
|
||||
inline notes editor with history, convert-to-tenant action, and the
|
||||
LeadEvent activity timeline. Permission: `admin.marketing.leads.manage`.
|
||||
|
||||
## CLI
|
||||
|
||||
```
|
||||
hivegrid-cli leads list [--status new] [--limit 50]
|
||||
hivegrid-cli leads export [--out leads.csv] [--status qualified]
|
||||
hivegrid-cli leads delete --id <leadId> # DSGVO right-to-be-forgotten
|
||||
```
|
||||
|
||||
## Configuration
|
||||
|
||||
| Env | Default | Purpose |
|
||||
|--------------------------------|---------|-------------------------------------------------------------|
|
||||
| `MARKETING_ALLOWED_ORIGINS` | (empty) | CSV of allowed origins for capture endpoint. Fail-closed. |
|
||||
| `MARKETING_RATE_LIMIT_PER_MIN` | 5 | Per-IP sliding-window submission limit. |
|
||||
| `TURNSTILE_SITE_KEY` | — | Cloudflare Turnstile site key (shared with form-widget). |
|
||||
| `TURNSTILE_SECRET_KEY` | — | Cloudflare Turnstile secret (shared with form-widget). |
|
||||
|
||||
The `marketing-funnel-turnstile` feature flag must be enabled for Turnstile
|
||||
to actually gate submissions; the secret alone is not enough.
|
||||
tenant Email Marketing scope in `.planning/ticket-docs/AK-P14a.md` and proves neither an active
|
||||
subscriber nor production delivery.
|
||||
|
||||
## Rollback
|
||||
|
||||
- `MARKETING_ALLOWED_ORIGINS=` empties the allowlist → all browser captures
|
||||
are rejected with 403 while leaving the admin UI usable for triage.
|
||||
- Drop the `Lead` + `LeadEvent` tables to wipe captured data (no other
|
||||
tables reference them today). Export with `hivegrid-cli leads export`
|
||||
first.
|
||||
|
||||
---
|
||||
|
||||
# Phase 2 — Self-Service Conversion + Outbound Webhooks + Attribution
|
||||
|
||||
Phase 2 (task #79) extends Phase 1's inbox with three independently-shippable
|
||||
layers:
|
||||
|
||||
1. **Self-service conversion deep-links** — a signed lead-token in the URL
|
||||
pre-fills the registration form. The normal registration service creates
|
||||
the User + Tenant, then records the Lead conversion against that Tenant:
|
||||
`Lead.convertedToTenantId`, a `TenantAcquisition` snapshot, and the
|
||||
marketing outbox event are written in one transaction.
|
||||
2. **Outbound webhook foundation** — six pluggable adapters (Slack, Discord,
|
||||
Mailchimp, HubSpot, Brevo, custom HTTPS) plus the
|
||||
`apps/worker/src/jobs/marketing-outbound.ts` handler contract. General
|
||||
marketing integration delivery is still a production follow-up because
|
||||
that handler is not currently bootstrapped from `apps/worker/src/index.ts`.
|
||||
3. **Attribution reports** — `/admin/marketing-funnels` aggregates leads
|
||||
+ acquisitions by source, campaign, utm_source, conversionPath. Mirror
|
||||
via `hivegrid-cli marketing funnel|attribution`.
|
||||
|
||||
## Signed lead-token
|
||||
|
||||
`signLeadToken({ leadId, planId, expiresInDays = 7 })` produces a token of
|
||||
the shape `v1.<base64url(payload)>.<base64url(hmac-sha256)>`. The HMAC
|
||||
key is `LEAD_CONVERSION_SECRET` (min 32 bytes). `verifyLeadToken(token)`
|
||||
returns the payload when the signature matches AND the token has not
|
||||
expired — otherwise `null`. Both functions fail-closed when the secret
|
||||
env is missing.
|
||||
|
||||
Email pre-filled from a Lead is rendered read-only on the register form
|
||||
to prevent lead-hijacking ("I use your token but with my email"). The signed
|
||||
token is also submitted as a hidden field and verified again in the register
|
||||
action. If email verification is enabled, the verified conversion context is
|
||||
stored on the pending `Registration` row and replayed from `/verify/:token`
|
||||
after the visitor sets a password. To change the email the visitor must
|
||||
abandon the link and start a fresh signup.
|
||||
|
||||
## Atomic conversion service
|
||||
|
||||
`convertLeadToTenant({ leadId, tenant: { name, slug, inboundAddress }, ... })`
|
||||
runs inside a single Prisma `$transaction`:
|
||||
|
||||
1. `tenant.create` (slug + inbound address pre-resolved by caller).
|
||||
2. `lead.update` → `status` advances to `trial`/`customer`,
|
||||
`convertedToTenantId` + `convertedAt` populated.
|
||||
3. `leadEvent.create` of type `converted` with full conversion payload.
|
||||
4. `tenantAcquisition.create` snapshot copying UTM/referrer/landingPage
|
||||
from the Lead.
|
||||
5. `publishOutboxEvent` with type `marketing.lead.convertedtotrial` or
|
||||
`marketing.lead.convertedtocustomer` and dispatchKey
|
||||
`lead-convert:<leadId>` (idempotent on retry).
|
||||
|
||||
Already-converted leads (`convertedToTenantId !== null`) short-circuit
|
||||
and return the existing tenant.
|
||||
|
||||
For the self-service register flow, the already-created Tenant is linked via
|
||||
`recordExistingTenantLeadConversion({ leadId, tenantId, ... })`. It reuses the
|
||||
same conversion event/outbox payload and upserts the `TenantAcquisition` row
|
||||
instead of creating a second Tenant.
|
||||
|
||||
## Admin manual override
|
||||
|
||||
`/admin/marketing-leads/<id>` exposes a "Create Tenant" form. Operator
|
||||
can override the tenant name + supply an optional planId. The action
|
||||
calls `convertLeadToTenant({ conversionPath: "admin-override", actorId })`.
|
||||
A native confirm dialog guards accidental clicks.
|
||||
|
||||
## Outbound webhook adapters
|
||||
|
||||
`packages/marketing-outbound` ships six adapters under
|
||||
`src/adapters/*.ts`:
|
||||
|
||||
| Adapter | Config keys (encrypted at rest) | Notes |
|
||||
|---------|----------------------------------------------------------------|-------|
|
||||
| slack | `webhookUrl`, `channel?` | Block-Kit message. |
|
||||
| discord | `webhookUrl`, `username?` | Embed with color-coded event. |
|
||||
| mailchimp | `apiKey`, `listId`, `status?`, `tags?` | PUT to lists/{id}/members/{md5(email)}. |
|
||||
| hubspot | `accessToken` | POST /crm/v3/objects/contacts; 409 = idempotent. |
|
||||
| brevo | `apiKey`, `listIds?` | POST /v3/contacts with `updateEnabled: true`. |
|
||||
| webhook | `url`, `signingSecret?`, `headers?` | POSTs canonical envelope; optional HMAC-SHA256 sig. |
|
||||
|
||||
All adapters share a single `adapterFetch` helper that enforces:
|
||||
|
||||
- HTTPS only (rejects http: / localhost / RFC1918 / 169.254.* /
|
||||
`metadata.google.internal` / `.internal` / `.local` / `.lan`).
|
||||
- 10 s default timeout (AbortController).
|
||||
- Returns structured `AdapterResult { ok, status?, errorTag? }` so the
|
||||
worker can categorise failures consistently.
|
||||
|
||||
The `ctx.maskPii.email` toggle masks the local part of the email
|
||||
(`patrick@example.com → ***@example.com`) before any payload leaves the
|
||||
host. Use it for data-egress-restricted tenants.
|
||||
|
||||
## Apps/worker handler status
|
||||
|
||||
`apps/worker/src/jobs/marketing-outbound.ts` defines a pure handler contract
|
||||
for marketing integration delivery, but it does not own or subscribe to
|
||||
`outbox.marketing`. That queue is currently used by the email-sequence trigger
|
||||
subscriber when `MARKETING_SEQUENCES_ENABLED=true`; this does not imply
|
||||
Slack/Discord/Mailchimp/HubSpot/Brevo/custom webhook delivery is active.
|
||||
Treat the adapters, handler, admin UI, and env flag as a shipped foundation
|
||||
until a bootstrap module wires a distinct general marketing integration
|
||||
subscriber into the worker runtime with explicit ownership tests.
|
||||
|
||||
The handler accepts only these event types today:
|
||||
`marketing.lead.captured`, `marketing.lead.qualified`,
|
||||
`marketing.lead.convertedtotrial`, and
|
||||
`marketing.lead.convertedtocustomer`. New `marketing.lead.*` events require
|
||||
an update to `MARKETING_LEAD_EVENT_TYPES` plus tests. For each accepted event:
|
||||
|
||||
1. Parse + validate envelope (`parseMarketingOutboundJob`).
|
||||
2. Bail out when `MARKETING_WEBHOOKS_ENABLED=false`.
|
||||
3. Fetch every enabled `MarketingIntegration` whose `triggerEvents`
|
||||
includes the eventType.
|
||||
4. For each integration: decrypt config, resolve adapter via
|
||||
`resolveAdapter(type)`, call `adapter.send`. One integration's
|
||||
failure does not affect siblings.
|
||||
5. Persist diagnostics (`lastDeliveredAt`, `lastError`) on the
|
||||
integration row so the admin UI can surface them.
|
||||
|
||||
## Admin integrations UI
|
||||
|
||||
`/admin/marketing-integrations` (Phase 05 #79). CRUD list with:
|
||||
|
||||
- Add form (type picker + name + trigger-events multi-select + config
|
||||
JSON textarea).
|
||||
- Per-integration enable/disable toggle.
|
||||
- Delete with native confirm dialog.
|
||||
- Last-delivered / last-error display.
|
||||
- Config display masked (`***` for `webhookUrl`, `apiKey`,
|
||||
`accessToken`, `signingSecret`).
|
||||
|
||||
Secrets are encrypted with the existing AES-256-GCM CryptoApi helper
|
||||
(`apps/web/app/utils/api/.server/CryptoApi.ts`) keyed off `CRYPTO_SECRET`.
|
||||
|
||||
## Attribution reports
|
||||
|
||||
`/admin/marketing-funnels` (Phase 05 #79). Top-line cards (leads
|
||||
captured, conversions, conversion rate) plus four breakdowns: campaign,
|
||||
utm_source, source-label, conversionPath. Reads via `dbRead`
|
||||
(replica-routed). CLI mirror:
|
||||
|
||||
- `hivegrid-cli marketing funnel --days 30`
|
||||
- `hivegrid-cli marketing attribution --dimension campaign --days 30 --limit 25`
|
||||
|
||||
## Configuration (Phase 2)
|
||||
|
||||
| Env | Default | Purpose |
|
||||
|------------------------------|---------|------------------------------------------------------------|
|
||||
| `LEAD_CONVERSION_SECRET` | — | HMAC key for signed lead-tokens (min 32 bytes). |
|
||||
| `LEAD_CONVERSION_ENABLED` | `true` | Set `false` to disable the conversion flow without DB rollback. |
|
||||
| `MARKETING_WEBHOOKS_ENABLED` | `true` | Handler-level kill switch once the general marketing integration subscriber is bootstrapped. It does not start that subscriber by itself. |
|
||||
|
||||
## Webhook payload (custom adapter)
|
||||
|
||||
```json
|
||||
{
|
||||
"schema": "marketing.lead.v1",
|
||||
"eventType": "marketing.lead.convertedtotrial",
|
||||
"occurredAt": "2026-05-17T00:00:00.000Z",
|
||||
"lead": {
|
||||
"id": "lead_abc",
|
||||
"email": "patrick@example.com",
|
||||
"firstName": "Patrick",
|
||||
"lastName": "Doe",
|
||||
"company": "Acme",
|
||||
"source": "landing-pro",
|
||||
"utm": { "source": "google", "medium": "cpc", "campaign": "q2-launch" },
|
||||
"referrer": "https://google.com",
|
||||
"landingPage": "/pricing"
|
||||
},
|
||||
"conversion": {
|
||||
"tenantId": "tenant_xyz",
|
||||
"targetStatus": "trial",
|
||||
"conversionPath": "self-service",
|
||||
"planId": "plan_pro"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
When `signingSecret` is set, the receiver verifies
|
||||
`x-marketing-signature: sha256=<hex>` against
|
||||
`hmac_sha256(body, signingSecret)`.
|
||||
|
||||
## Rollback (Phase 2)
|
||||
|
||||
- `LEAD_CONVERSION_ENABLED=false` — register loader ignores the token
|
||||
query params; admin override button hidden.
|
||||
- `MARKETING_WEBHOOKS_ENABLED=false` — once the general marketing
|
||||
integration subscriber is bootstrapped, the worker handler short-circuits.
|
||||
Today it is only a handler-level kill switch for the shipped foundation.
|
||||
Outbox events accumulate; raise outbox retention if you plan a long
|
||||
pause.
|
||||
- Per-integration disable toggle in the admin UI — faulty integrations
|
||||
can be parked without a deploy.
|
||||
- `TenantAcquisition` + `MarketingIntegration` tables can be dropped
|
||||
(no FK in to them from Tenant), but you'd lose attribution history.
|
||||
Während des dokumentierten Drain-Fensters können Producer auf den gesicherten
|
||||
externen Legacy-Dienst zurückgestellt werden. Der HiveGrid-Cloud-Browserendpoint
|
||||
bleibt auch im Rollback `410`; eine Reaktivierung wäre eine neue Architektur- und
|
||||
Security-Entscheidung. Native Importe sind idempotent und löschen keine
|
||||
Quellbestände. Das letzte verschlüsselte, restore-geprüfte Backup wird bis zu
|
||||
einem separaten Retention-Entscheid behalten.
|
||||
@@ -135,7 +135,6 @@ For documentation-only changes:
|
||||
|
||||
```sh
|
||||
pnpm run verify:docs-links
|
||||
pnpm run verify:production-docs
|
||||
```
|
||||
|
||||
For runtime/plugin changes:
|
||||
|
||||
@@ -139,6 +139,8 @@ Rollback: dropping `CollabDoc` is safe — no other code path reads from it.
|
||||
|
||||
All entries are also documented in `packages/config/src/catalog.ts` so the env-drift verifier accepts them.
|
||||
|
||||
`VITE_HOCUSPOCUS_URL` ist öffentliche Konfiguration und kein Secret. Die gleichnamige Gitea-Repository-Variable wird ausschließlich an das Docker-Target `web-build` übergeben und sowohl in das Browser-Bundle als auch in dessen authentifizierte CSP eingebettet. Da die Production-Promotion exakt denselben bereits gestagten Image-Digest verwendet, muss der konfigurierte öffentliche WebSocket-Endpunkt für beide Deployments gültig sein; bei einem geteilten Image ist dafür ein umgebungsneutraler Gateway-Hostname zu verwenden.
|
||||
|
||||
## Out of scope / deferred
|
||||
|
||||
- **Snapshot + cleanup cron** — a planned follow-up will land `apps/worker/src/cron/collab-cleanup.ts` plus the `CollabDocArchive` table referenced in the spec.
|
||||
|
||||
@@ -29,21 +29,37 @@ in phase 04 (#48). It is the user-facing complement to
|
||||
- **Source of truth** is Postgres. Meilisearch is a projected read-side
|
||||
populated asynchronously through the outbox pattern (#42).
|
||||
- **Tenant isolation** is enforced inside `@hivegrid/search`. Every read
|
||||
and write goes through `searchInIndex / bulkIndex / deleteFromIndex`
|
||||
with an explicit `tenantId`. The wrapper injects
|
||||
and write goes through the tenant-scoped package helpers with an explicit
|
||||
`tenantId`. The wrapper injects
|
||||
`tenantId = "<id>"` into the Meilisearch filter expression and
|
||||
escapes the value to defeat filter injection. Raw Meilisearch SDK
|
||||
access is not re-exported.
|
||||
- **Estate namespace:** schemas, outbox payloads and worker pagers use logical
|
||||
names such as `projects`. The `SearchClient` adapter is the only boundary
|
||||
that maps them to physical Meilisearch UIDs (`hivegrid-projects`). Callers
|
||||
never add the prefix themselves, and the adapter rejects already-prefixed
|
||||
input so no unprefixed or double-prefixed namespace can be touched.
|
||||
- **Graceful degradation:** every hot-path is wrapped in
|
||||
`searchWithFallback`, which routes to the legacy SQL `LIKE`
|
||||
implementation when Meilisearch raises a connectivity error. The
|
||||
search stays degraded-but-up instead of returning 500s.
|
||||
|
||||
## User-Facing Contract
|
||||
|
||||
Since the 2026-07-12 scope cut, global search is exposed only as a Spotlight
|
||||
overlay. On desktop it opens through Cmd+K/Ctrl+K or the header search trigger;
|
||||
on mobile the header trigger opens a viewport-filling overlay. There is no
|
||||
user-facing `/search` page, shareable search URL, or full-page filter/pagination
|
||||
surface in v1. `/search/api` remains the protected, tenant-scoped internal
|
||||
resource used by the overlay. `R-P11b` continues to consume the same index and
|
||||
search contracts.
|
||||
|
||||
## Index Schema Reference
|
||||
|
||||
The current schemas live in `packages/search/src/schemas/`. Each is a
|
||||
whitelist — only fields listed in `allowedDocumentFields` reach
|
||||
Meilisearch.
|
||||
Meilisearch. Names in this table are logical contract names; the physical UID
|
||||
always has the estate prefix `hivegrid-`.
|
||||
|
||||
| Index | Searchable | Filterable | Sortable |
|
||||
| -------------- | --------------------------------------------------- | ------------------------------------------------------ | --------------------------------------- |
|
||||
@@ -73,13 +89,13 @@ one otherwise).
|
||||
|
||||
## Hot-Path Migrations
|
||||
|
||||
| Path | Migrated | Notes |
|
||||
| ------------------------------------------------ | --------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| KB public search (`KnowledgeBaseService.search`) | yes (#48) | wraps `findKnowledgeBaseArticles` via `searchKnowledgeBaseArticles` helper with `searchWithFallback`. Tenant id resolved from KB record. |
|
||||
| CRM contact helper | shipped | `apps/web/app/modules/crm/services/crmContactSearch.server.ts` wraps `crm-contacts` with `searchWithFallback` and a tenant-scoped SQL fallback. |
|
||||
| Rows helper | shipped | `apps/web/app/modules/rows/services/rowSearch.server.ts` wraps the `rows` index with an `entityId` filter and tenant-scoped SQL fallback. |
|
||||
| CRM/Rows producer and callsite integration | yes | `RowsApi.getAll` calls `searchCrmContacts` for tenant-scoped CRM contact-list free-text filters and `searchRows` for other tenant-scoped row filters. `RowsApi.create / createSimple / update / del` emit `search.index.*` events through `rowSearchOutbox.server.ts`. Contact rows fan out to both `crm-contacts` and `rows`. |
|
||||
| HiveGrid global-search producers | yes | `apps/web/app/services/search/searchIndexProjection.server.ts` publishes `search.index.*` events and mirrors `SearchIndexEntry` for Kontakt/Firma/Lead/Projekt/File/Task/User lifecycle mutations. Cmd+K and `/search` read through tenant- and permission-aware SQL+Meili search. |
|
||||
| Path | Migrated | Notes |
|
||||
| ------------------------------------------------ | --------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| KB public search (`KnowledgeBaseService.search`) | yes (#48) | wraps `findKnowledgeBaseArticles` via `searchKnowledgeBaseArticles` helper with `searchWithFallback`. Tenant id resolved from KB record. |
|
||||
| CRM contact helper | shipped | `apps/web/app/modules/crm/services/crmContactSearch.server.ts` wraps `crm-contacts` with `searchWithFallback` and a tenant-scoped SQL fallback. |
|
||||
| Rows helper | shipped | `apps/web/app/modules/rows/services/rowSearch.server.ts` wraps the `rows` index with an `entityId` filter and tenant-scoped SQL fallback. |
|
||||
| CRM/Rows producer and callsite integration | yes | `RowsApi.getAll` calls `searchCrmContacts` for tenant-scoped CRM contact-list free-text filters and `searchRows` for other tenant-scoped row filters. `RowsApi.create / createSimple / update / del` emit `search.index.*` events through `rowSearchOutbox.server.ts`. Contact rows fan out to both `crm-contacts` and `rows`. |
|
||||
| HiveGrid global-search producers | yes | `apps/web/app/services/search/searchIndexProjection.server.ts` publishes `search.index.*` events and mirrors `SearchIndexEntry` for Kontakt/Firma/Lead/Projekt/File/Task/User lifecycle mutations. The Spotlight overlay reads through `/search/api` using tenant- and permission-aware SQL+Meili search. |
|
||||
| Bulk reindex pagers | partial | `apps/worker/src/index.ts` registers `createSearchIndexerBulkPagers({ prisma })`; `BULK_REINDEX_SUPPORTED_INDEX_NAMES` exposes `kb-articles`, `crm-contacts`, `users`, and `projects`. File/task live producers are wired, but their disaster-recovery bulk pagers still need explicit coverage before `--index=all` can rebuild every projected index. |
|
||||
|
||||
Open hot-path gaps still serve from the legacy SQL path. Until a specific
|
||||
@@ -98,10 +114,10 @@ post-write rather than inside the row mutation transaction.
|
||||
```bash
|
||||
# Reindex every supported bulk-reindex index for every tenant via outbox events.
|
||||
# Today this means KB articles, CRM contacts, users, and projects; unsupported names are rejected.
|
||||
pnpm cli search reindex --tenant=all --index=all
|
||||
pnpm exec tsx packages/cli/bin/hivegrid-cli.mjs search reindex --tenant=all --index=all
|
||||
|
||||
# Reindex one tenant's KB articles.
|
||||
pnpm cli search reindex --tenant=tnt_abc123 --index=kb-articles
|
||||
pnpm exec tsx packages/cli/bin/hivegrid-cli.mjs search reindex --tenant=tnt_abc123 --index=kb-articles
|
||||
```
|
||||
|
||||
The CLI never writes to Meilisearch directly. It produces
|
||||
@@ -142,7 +158,7 @@ The Meilisearch volume is treated as cache. To rebuild from scratch:
|
||||
|
||||
1. `docker volume rm meilisearch-data` (after stopping the container).
|
||||
2. Restart the container.
|
||||
3. `pnpm cli search reindex --tenant=all --index=all`.
|
||||
3. `pnpm exec tsx packages/cli/bin/hivegrid-cli.mjs search reindex --tenant=all --index=all`.
|
||||
|
||||
This rebuild currently covers the supported bulk-reindex set only:
|
||||
`kb-articles`, `crm-contacts`, `users`, and `projects`. Entities require a
|
||||
|
||||
Reference in new issue
Block a user