Track HiveGrid Cloud planning reference
This commit is contained in:
1 parent
b3d5ea6764
commit
e341c5938a
5784 files changed
+1104929
-3
No files matched your search
@@ -0,0 +1,449 @@
|
||||
# Compliance-Audit-Export
|
||||
|
||||
**Status:** completed
|
||||
**Task-ID:** #67
|
||||
**Phase:** 10
|
||||
**Estimated Effort:** 2 Wochen
|
||||
**Owner:** unassigned
|
||||
|
||||
## Problem
|
||||
|
||||
Die Boilerplate hat ein generisches `AuditEvent`-Model in `packages/db/prisma/schema.prisma`, das für SOC2-/HIPAA-/GDPR-Evidence nicht ausreicht:
|
||||
|
||||
- **Fehlende Felder**: keine `actorType` (User vs Admin vs API-Key vs System vs Plugin-Origin), keine `actorIp`, keine `actorUserAgent`, kein `regulatoryScope` (gdpr/hipaa/soc2/none), keine `dataSensitivity` (public/internal/confidential/pii), kein `outcome` (success/failure/blocked)
|
||||
- **Keine Auto-Capture-Schicht**: Privileged-Actions (Permission-Changes, Impersonations, Read of PII-Models, Export-Operations) müssen heute pro Code-Stelle manuell geloggt werden — Coverage ist lückenhaft, Audit-Findings „Action X war nicht im Audit-Trail" sind regelmäßig
|
||||
- **Keine Per-Tenant-Export-UI**: Tenant-Admins können ihre eigenen Audit-Trails nicht zu Compliance-Zwecken exportieren (für SOC2-Type-II-Audits muss der Tenant seine Privileged-Action-Reports liefern können)
|
||||
- **Keine Retention-Policy pro Tenant/Plan**: Audit-Events bleiben dauerhaft in der DB (oder werden ad-hoc weggeräumt) — kein konfigurierbares Window, keine Auto-Archivierung zu Long-Term-Storage
|
||||
- **Keine Tamper-Evidence**: Audit-Events sind in einer normalen Tabelle, ein Plattform-Admin (oder ein DB-Eindringling) könnte Events modifizieren ohne Spur. Für SOC2-CC6.1 brauchen wir tamper-evident Logs (Hash-Chain oder WORM-Storage)
|
||||
- **Privileged-Actions-Reports**: keine vordefinierten Compliance-Report-Templates (Permission-Changes, Auth-Events, Data-Export-Operations) — Auditor muss heute alles selbst joinen
|
||||
|
||||
Betroffene Codestellen:
|
||||
- `packages/db/prisma/schema.prisma` — `AuditEvent`-Model erweitern
|
||||
- `apps/api/src/auth/*` — alle Auth-Flows müssen auto-capture, manuelle `auditLog.create` raus
|
||||
- `apps/web/app/utils/db/permissions/*` — Permission-Changes auto-capturen
|
||||
- `apps/web/app/modules/admin/services/impersonation.server.ts` — Impersonations werden bereits in `ImpersonationAudit` geloggt, muss konsolidiert werden mit AuditEvent
|
||||
- `apps/worker/src/jobs/*` — System-Actions auto-capturen
|
||||
|
||||
## Depends on
|
||||
|
||||
- `01-tech-debt-foundation/dep-updates.md` — Prisma-Updates müssen vorher durch sein
|
||||
- `02-observability/logging-migration.md` — Audit-Logs sind separat von App-Logs, aber müssen mit dem zentralen Logger koexistieren
|
||||
- `03-scaling-infrastructure/outbox-pattern.md` — Audit-Event-Writes sollen transaktional mit Business-Writes laufen (über Outbox-Pattern), sonst gibt es Audit-Lücken bei Crash zwischen Write und Audit-Insert
|
||||
|
||||
## Blocks
|
||||
|
||||
- `10-compliance-polish/dsgvo-bulk-ops.md` — DSGVO-Bulk-Ops müssen alle Operations in den erweiterten AuditEvent loggen; ohne neue Felder (actorIp, regulatoryScope=gdpr, outcome) wäre die Audit-Evidence unvollständig
|
||||
|
||||
## Approach
|
||||
|
||||
**Fünf koordinierte Sub-Komponenten:**
|
||||
|
||||
### (a) AuditEvent-Schema-Erweiterung
|
||||
|
||||
`packages/db/prisma/schema.prisma`:
|
||||
|
||||
```prisma
|
||||
model AuditEvent {
|
||||
id String @id @default(cuid())
|
||||
tenantId String? // null für Plattform-Level-Events
|
||||
// Actor
|
||||
actorType AuditActorType // user | admin | api_key | system | plugin
|
||||
actorId String? // null für system
|
||||
actorEmail String?
|
||||
actorIp String? // IPv4 oder IPv6, ggf. anonymisiert (siehe DSGVO-Bulk-Ops)
|
||||
actorUserAgent String?
|
||||
// Resource
|
||||
resourceType String // "user", "subscription", "tenant", ...
|
||||
resourceId String?
|
||||
// Action
|
||||
action AuditAction // CREATE | READ | UPDATE | DELETE | EXPORT | IMPORT | AUTH | PERMISSION | EXPORT_PERSONAL_DATA
|
||||
outcome AuditOutcome // success | failure | blocked
|
||||
// Metadata
|
||||
sourceModule String // "auth", "billing", "crm", "plugin:foo", ...
|
||||
dataSensitivity DataSensitivity // public | internal | confidential | pii
|
||||
regulatoryScope String[] // ["gdpr", "hipaa", "soc2", "none"] — Multi-Select
|
||||
metadata Json? // freie Daten, max 8KB validation
|
||||
// Tamper-Evidence
|
||||
prevHash String? // SHA-256 des vorherigen Events in der Chain (per Tenant)
|
||||
hash String // HMAC-SHA-256 über (prevHash + serializedEvent), Key aus AUDIT_CHAIN_SECRET
|
||||
// Time
|
||||
createdAt DateTime @default(now())
|
||||
|
||||
@@index([tenantId, createdAt])
|
||||
@@index([tenantId, action, createdAt])
|
||||
@@index([tenantId, resourceType, resourceId])
|
||||
@@index([tenantId, actorId, createdAt])
|
||||
}
|
||||
|
||||
enum AuditActorType { user admin api_key system plugin }
|
||||
enum AuditAction { CREATE READ UPDATE DELETE EXPORT IMPORT AUTH PERMISSION EXPORT_PERSONAL_DATA }
|
||||
enum AuditOutcome { success failure blocked }
|
||||
enum DataSensitivity { public internal confidential pii }
|
||||
```
|
||||
|
||||
Migration: `prisma/migrations/<ts>_audit_event_compliance_fields/migration.sql` mit Backfill-Script für existing rows (Default-Werte: `actorType=user`, `outcome=success`, `dataSensitivity=internal`, `regulatoryScope=[]`).
|
||||
|
||||
### (b) Auto-Capture-Middleware
|
||||
|
||||
Zwei Middlewares (eine pro App):
|
||||
|
||||
- `apps/api/src/middleware/audit-capture.ts` — Fastify-Hook auf `preHandler` + `onResponse`, captured automatisch:
|
||||
- alle `/auth/*`-Routes als `action=AUTH`
|
||||
- alle `/admin/*`-Routes als privileged (actorType derived from session)
|
||||
- Permission-API-Calls als `action=PERMISSION`
|
||||
- Data-Export-Endpoints als `action=EXPORT`
|
||||
- `apps/web/app/lib/audit/server.ts` — Helper-Funktion `captureAudit(request, context, event)`, wird in Loaders/Actions aufgerufen (manuell für komplexe Cases, automatisch für die meisten via React-Router-Middleware)
|
||||
|
||||
Auto-Capture sammelt:
|
||||
- Actor aus Session/JWT
|
||||
- IP via `request.headers.get('x-forwarded-for')` mit Trust-Proxy-Resolution
|
||||
- UserAgent via `request.headers.get('user-agent')`
|
||||
- Resource aus Route-Params (z.B. `/users/:id`)
|
||||
- Outcome via Response-Status (`2xx` = success, `4xx blocked` (403/401), `5xx failure`)
|
||||
|
||||
Auto-Capture für Prisma-Read-Operations auf PII-Models: via Prisma-Middleware in `packages/db/src/prisma-middleware/audit-pii-reads.ts` — bei `findUnique`/`findMany` auf Models mit `@@audit("pii")`-Tag wird ein READ-Event geschrieben.
|
||||
|
||||
### (c) Per-Tenant-Export-UI
|
||||
|
||||
Route: `/app/$tenant/settings/audit-export`
|
||||
|
||||
UI-Features:
|
||||
- Datum-Range-Picker (max 1 Jahr pro Export)
|
||||
- Filter: `actorType`, `action`, `resourceType`, `dataSensitivity`, `regulatoryScope`
|
||||
- Output-Format: CSV | JSONL | SOC2-Standard-Schema (JSON-Bundle nach SOC2-Common-Criteria-Layout)
|
||||
- „Export anfordern" → Background-Job in BullMQ-Queue `audit-export`
|
||||
|
||||
Worker (`apps/worker/src/jobs/audit-export.ts`):
|
||||
- Streamt aus DB via cursor-based pagination (verhindert OOM bei 1M+ Events)
|
||||
- Schreibt in S3-kompatibles Storage (`packages/storage`-Provider)
|
||||
- Signed-URL mit 7-Tage-Validity wird per Email an Tenant-Admin gesendet (React-Email-Template `AuditExportReadyEmail.tsx`)
|
||||
- Hash über Export-File wird als Audit-Event geloggt (Meta-Audit: „Export of N events created with hash X")
|
||||
|
||||
### (d) Per-Tenant-Retention-Policy
|
||||
|
||||
Konfigurierbar via Plan-Entitlements (`packages/entitlements`):
|
||||
- Standard-Plan: 1 Jahr (365 Tage)
|
||||
- Pro-Plan: 3 Jahre
|
||||
- Enterprise-Plan: 7 Jahre (SOX-Compliance-Window)
|
||||
|
||||
Cleanup-Cron (`apps/worker/src/cron/audit-cleanup.ts`, läuft täglich):
|
||||
- Pro Tenant: `DELETE FROM audit_event WHERE tenant_id = $1 AND created_at < NOW() - INTERVAL <retention>`
|
||||
- Vor Cleanup: optional Auto-Export zu S3-Long-Term-Storage (Glacier-Tier) wenn `tenant.auditAutoArchiveEnabled = true`
|
||||
- Cleanup-Run selbst wird als System-AuditEvent geloggt (`actorType=system`, `action=DELETE`, `resourceType=audit_event`, `metadata={ deletedCount: N }`)
|
||||
|
||||
### (e) Privileged-Actions-Report-Template
|
||||
|
||||
Pre-Built Queries für die wichtigsten Compliance-Reports:
|
||||
- `apps/web/app/lib/audit/reports/permission-changes.ts` — alle Permission-/Role-Changes im Range
|
||||
- `apps/web/app/lib/audit/reports/auth-events.ts` — Login-Successes, -Failures, MFA-Challenges, Password-Resets
|
||||
- `apps/web/app/lib/audit/reports/data-exports.ts` — alle EXPORT-Actions
|
||||
- `apps/web/app/lib/audit/reports/impersonations.ts` — alle Admin-Impersonations (joined mit `ImpersonationAudit`)
|
||||
- `apps/web/app/lib/audit/reports/personal-data-access.ts` — alle READs auf PII-Models
|
||||
|
||||
PDF-Report-Generator (`packages/exports/src/audit-pdf.ts`):
|
||||
- Title-Page mit Tenant-Name, Range, Generated-By (Admin-User), Generated-At
|
||||
- Signature-Hash unten (SHA-256 über Report-Content + Timestamp + Server-Secret) — bestätigt Report-Integrität
|
||||
- Section pro Report-Type
|
||||
- Generiert via `puppeteer` (HTML-zu-PDF) oder `pdfkit` (direkt PDF)
|
||||
|
||||
### (f) Tamper-Evidence Hash-Chain
|
||||
|
||||
Jeder AuditEvent enthält:
|
||||
- `prevHash` = `hash` des vorherigen Events des gleichen Tenants (Lookup via `ORDER BY createdAt DESC LIMIT 1`)
|
||||
- `hash` = `HMAC-SHA-256(AUDIT_CHAIN_SECRET, prevHash || JSON.stringify(eventWithoutHash))`
|
||||
|
||||
Verification-CLI (`scripts/audit-verify-chain.mjs`):
|
||||
- `pnpm audit:verify-chain --tenant <id> [--from <date>] [--to <date>]`
|
||||
- Iteriert chronologisch durch Events, rechnet jeden Hash neu, vergleicht mit gespeichertem Wert
|
||||
- Bei Mismatch: Output mit Event-ID, expected vs actual Hash, Markierung als „tampered"
|
||||
- Returns non-zero exit code wenn Tampering erkannt
|
||||
|
||||
Hash-Chain wird auf API-Schreib-Pfad enforced (kein direkter `db.auditEvent.create` ohne Hash-Berechnung) via Prisma-Middleware `packages/db/src/prisma-middleware/audit-hash-chain.ts`.
|
||||
|
||||
### (g) Dokumentation
|
||||
|
||||
- `docs/compliance/audit-trails.md` — Architektur, Schema, Capture-Punkte, Hash-Chain-Verification, Export-Workflow
|
||||
- `docs/compliance/soc2-evidence-collection.md` — wie SOC2-Auditoren die Evidence beschaffen (welche Reports, welche Queries, welche Tools)
|
||||
|
||||
**Warum nicht Alternativen:**
|
||||
- ❌ **Externes SIEM (Splunk, ELK)**: Zu schwer für eine SaaS-Boilerplate, nicht standardisierbar pro Kunde, Vendor-Lockin
|
||||
- ❌ **WORM-Storage statt Hash-Chain**: Erfordert spezielles Storage-Backend (S3-Object-Lock, immutable-bucket); Hash-Chain ist portable und reicht für SOC2-CC6.1
|
||||
- ❌ **JSONL-Append-Log**: Schwerer zu queryn (jeder Auditor will Reports), keine Datenbank-Integrität-Checks, schwerer zu retainen pro Tenant
|
||||
- ❌ **Audit nur in API-Layer**: Verfehlt System-Actions (Cron-Jobs, Worker-Jobs) und Web-Direct-Database-Operations; Audit-Coverage muss überall sein
|
||||
|
||||
## Implementation Steps
|
||||
|
||||
1. **Prisma-Schema-Migration**
|
||||
- Files: `packages/db/prisma/schema.prisma`, `packages/db/prisma/migrations/<ts>_audit_event_compliance/`
|
||||
- Felder + Enums + Indizes
|
||||
- Backfill-Script für existing rows mit Defaults
|
||||
- Commit: `10-compliance-polish/audit-export: extend AuditEvent schema with SOC2/HIPAA/GDPR fields`
|
||||
|
||||
2. **Hash-Chain-Helper + Prisma-Middleware**
|
||||
- Files: `packages/db/src/audit/hash-chain.ts`, `.test.ts`, `packages/db/src/prisma-middleware/audit-hash-chain.ts`, `.test.ts`
|
||||
- Helper berechnet prevHash + currentHash, Middleware enforced auf `auditEvent.create`
|
||||
- Env-Var: `AUDIT_CHAIN_SECRET` (rotation-fähig via Secret-Versioning)
|
||||
- Tests: Hash-Stabilität, Chain-Continuity, Mismatch-Detection
|
||||
- Commit: `10-compliance-polish/audit-export: add hash-chain helper and prisma middleware`
|
||||
|
||||
3. **Auto-Capture-Middleware (API)**
|
||||
- Files: `apps/api/src/middleware/audit-capture.ts`, `.test.ts`
|
||||
- Fastify-Hooks für Auth/Admin/Permission/Export-Routes
|
||||
- Tests: capture-Verhalten pro Action-Type, korrektes Outcome-Mapping
|
||||
- Commit: `10-compliance-polish/audit-export: add api auto-capture middleware`
|
||||
|
||||
4. **Auto-Capture-Helper (Web)**
|
||||
- Files: `apps/web/app/lib/audit/server.ts`, `.test.ts`
|
||||
- `captureAudit(request, context, event)` Helper
|
||||
- React-Router-Middleware-Wrapper für Loader/Action-Auto-Capture
|
||||
- Commit: `10-compliance-polish/audit-export: add web audit capture helpers`
|
||||
|
||||
5. **Prisma-Middleware für PII-Read-Audit**
|
||||
- Files: `packages/db/src/prisma-middleware/audit-pii-reads.ts`, `.test.ts`
|
||||
- Schema-Tags `@@audit("pii")` parsen aus Prisma-Schema-Comments
|
||||
- Middleware emittiert READ-Event pro PII-Model-Read
|
||||
- Tests: PII-Models triggern Audit, Non-PII nicht
|
||||
- Commit: `10-compliance-polish/audit-export: add prisma middleware for PII read auditing`
|
||||
|
||||
6. **Manuelle auditLog.create-Aufrufe konsolidieren**
|
||||
- Files: alle `apps/api/src/auth/*`, `apps/web/app/utils/db/permissions/*`, `apps/web/app/modules/admin/services/impersonation.server.ts`
|
||||
- Manuelle Audit-Aufrufe entfernen wo Auto-Capture greift, sonst aufs neue Schema migrieren
|
||||
- Tests: bestehende Audit-Tests bleiben grün
|
||||
- Commit: `10-compliance-polish/audit-export: consolidate manual audit calls onto new schema`
|
||||
|
||||
7. **Per-Tenant-Export-UI + API**
|
||||
- Files: `apps/web/app/routes/app.$tenant/settings/audit-export.tsx`, `apps/web/app/routes/app.$tenant/settings/audit-export.api.tsx`, plus UI-Components
|
||||
- Datum-Range-Picker, Filter-Form, „Export anfordern"-Action
|
||||
- Commit: `10-compliance-polish/audit-export: add per-tenant export ui`
|
||||
|
||||
8. **Export-Worker-Job**
|
||||
- Files: `apps/worker/src/jobs/audit-export.ts`, `.test.ts`
|
||||
- Cursor-based Streaming, Output-Format-Switch (CSV/JSONL/SOC2-JSON), Upload zu S3, Signed-URL, Email
|
||||
- Hash-Log über Export-File
|
||||
- Commit: `10-compliance-polish/audit-export: add audit-export worker job`
|
||||
|
||||
9. **Retention-Cleanup-Cron**
|
||||
- Files: `apps/worker/src/cron/audit-cleanup.ts`, `.test.ts`
|
||||
- Plan-basierte Retention-Lookup via `packages/entitlements`
|
||||
- Optional Auto-Archive-zu-S3 vor Delete
|
||||
- Commit: `10-compliance-polish/audit-export: add retention cleanup cron`
|
||||
|
||||
10. **Pre-Built Compliance-Reports + PDF-Generator**
|
||||
- Files: `apps/web/app/lib/audit/reports/*.ts`, `packages/exports/src/audit-pdf.ts`, plus Tests
|
||||
- 5 Report-Templates + PDF-Renderer mit Signature-Hash
|
||||
- Commit: `10-compliance-polish/audit-export: add pre-built compliance reports and pdf generator`
|
||||
|
||||
11. **Verification-CLI**
|
||||
- Files: `scripts/audit-verify-chain.mjs`, `scripts/audit-verify-chain.test.mjs`
|
||||
- `pnpm audit:verify-chain --tenant <id>`-Command
|
||||
- Tests: Happy-Path, Tampering-Detection
|
||||
- Commit: `10-compliance-polish/audit-export: add hash-chain verification cli`
|
||||
|
||||
12. **React-Email-Template für Export-Ready-Notification**
|
||||
- Files: `packages/mail/src/templates-react/transactional/AuditExportReadyEmail.tsx`, Test, Preview
|
||||
- Verlinkt Signed-URL, erinnert an 7-Tage-Expiry
|
||||
- Commit: `10-compliance-polish/audit-export: add export-ready notification email`
|
||||
|
||||
13. **Dokumentation**
|
||||
- Files: `docs/compliance/audit-trails.md`, `docs/compliance/soc2-evidence-collection.md`
|
||||
- Architektur, Capture-Punkte, Hash-Chain, Export-Workflow, SOC2-Auditor-Guide
|
||||
- Commit: `10-compliance-polish/audit-export: document audit trails and soc2 evidence collection`
|
||||
|
||||
## Test Requirements
|
||||
|
||||
- **Unit-Tests:**
|
||||
- Hash-Chain-Helper: Stabilität, Continuity, Mismatch-Detection
|
||||
- Auto-Capture-Middleware: korrekte Field-Extraction pro Request-Type
|
||||
- PII-Read-Middleware: PII-Models triggern Audit, Non-PII nicht
|
||||
- Retention-Cleanup: Plan-basierte Window-Berechnung, Auto-Archive-Switch
|
||||
- **Integration-Tests:**
|
||||
- End-to-End: Login → Audit-Event geschrieben mit Hash → Logout → Hash-Chain konsistent → Export → Signed-URL → Download → Hash-Verify auf Export-File
|
||||
- DSGVO-Compliance-Test: User-Read auf PII-Model → Audit-Event mit `dataSensitivity=pii` + `regulatoryScope=[gdpr]`
|
||||
- **E2E-Tests (Playwright):**
|
||||
- Tenant-Admin loggt sich ein, navigiert zu `/app/$tenant/settings/audit-export`, wählt Range + Filter, fordert Export an
|
||||
- Verifiziert: Job läuft, Email kommt (Mailjet-Sandbox-API), Signed-URL funktioniert, Download enthält erwartete Events
|
||||
- **CLI-Test:** `pnpm audit:verify-chain --tenant <test-tenant>` → exit code 0 bei intakter Chain, exit code 1 + verständliche Diagnose bei tampering
|
||||
- **Coverage-Target:** ≥ 85 % im neuen Audit-Code (Helper, Middleware, Worker-Job, Reports)
|
||||
- **Test-Files:** co-located `<source>.test.ts`; E2E in `tests/e2e/audit-export.spec.ts`
|
||||
|
||||
**NICHT als completed markieren bevor Hash-Chain-Verifier auf einem Test-Tenant mit > 1000 Events grün läuft.**
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- [ ] `AuditEvent`-Schema erweitert um alle Compliance-Felder; Migration grün, Backfill-Script grün
|
||||
- [ ] Hash-Chain wird auf jedes neue Event berechnet; Verifier-CLI funktioniert
|
||||
- [ ] Auto-Capture-Middleware (API + Web) deckt alle Auth/Admin/Permission/Export-Routes ab
|
||||
- [ ] PII-Read-Middleware emittiert Audit-Events für markierte Models
|
||||
- [ ] Per-Tenant-Export-UI auf `/app/$tenant/settings/audit-export` funktioniert; Background-Job liefert Signed-URL per Email
|
||||
- [ ] Retention-Cleanup-Cron läuft täglich; respektiert Plan-basierte Windows (1y/3y/7y)
|
||||
- [ ] Optional Auto-Archive-zu-S3-Glacier funktioniert wenn aktiviert
|
||||
- [ ] 5 Pre-Built Compliance-Reports verfügbar; PDF-Generator mit Signature-Hash
|
||||
- [ ] `pnpm audit:verify-chain --tenant <id>` CLI-Command grün bei intakter Chain
|
||||
- [ ] Coverage ≥ 85 % im Audit-Code
|
||||
- [ ] `pnpm lint`, `pnpm typecheck`, `pnpm test` grün
|
||||
- [ ] `docs/compliance/audit-trails.md` + `docs/compliance/soc2-evidence-collection.md` angelegt
|
||||
- [ ] Staging-Verifikation: Export-Workflow end-to-end, Hash-Chain-Verifier grün
|
||||
- [ ] Detail-MD Progress Log aktualisiert
|
||||
- [ ] PROGRESS.md Status auf `completed`
|
||||
|
||||
## Rollback Strategy
|
||||
|
||||
- **Schema-Rollback**: Prisma-Migration `down` setzt neue Felder + Enums zurück; existing Data bleibt (Down-Migration enthält `DROP COLUMN` und `DROP TYPE`)
|
||||
- **Auto-Capture-Rollback**: Middleware via Feature-Flag `AUDIT_AUTO_CAPTURE_ENABLED=false` deaktivierbar; manuelle Audit-Calls bleiben funktional
|
||||
- **Hash-Chain-Rollback**: Wenn Hash-Berechnung performance-kritisch wird, via Feature-Flag `AUDIT_HASH_CHAIN_ENABLED=false` deaktivierbar; bestehende Hashes bleiben in DB
|
||||
- **Export-Worker-Rollback**: Job-Definition aus BullMQ-Queue entfernen; UI-Route 404 erlauben (kein Daten-Loss)
|
||||
- **Retention-Cleanup-Rollback**: Cron deaktivieren; bisher gelöschte Events sind weg (deshalb Auto-Archive-Empfehlung)
|
||||
- Keine Daten-Loss-Risiken solange Auto-Archive aktiv ist
|
||||
|
||||
## Security Notes
|
||||
|
||||
- **`AUDIT_CHAIN_SECRET`** ist platform-level-Secret, nie pro Tenant, nie in Logs, Rotation-Mechanismus via Versionsfeld in Hash (`hash = HMAC(secret_v2, ...)`)
|
||||
- **Hash-Chain-Tampering**: kann nur durch Plattform-Admin (DB-Zugriff) erfolgen — Verifier erkennt es; Auditor-Workflow muss regelmäßige Verifikation einplanen
|
||||
- **Audit-Event-Inhalte selbst sind sensitiv** (enthalten IP, UserAgent, User-Emails, Resource-IDs) — gleicher Schutz wie PII; deshalb `dataSensitivity=internal` als Default
|
||||
- **Export-Signed-URLs**: 7-Tage-Validity (nicht länger), `Content-Disposition: attachment`, S3-Bucket-Policy verbietet Public-Read
|
||||
- **Export-Inhalte selbst sind PII**: Export-Trigger wird selbst auditiert (Meta-Audit), inkl. Hash über Export-File
|
||||
- **PII-Read-Audit kann Performance-kritisch sein**: Middleware muss async-batched schreiben (BullMQ-Queue mit Low-Priority), nicht im Request-Pfad sync
|
||||
- **Retention-Cleanup**: Sicherstellen dass kein Auditor mitten in einem Audit ist (Lock-Mechanismus oder Manual-Pause-Flag pro Tenant)
|
||||
- **DSGVO-Konflikt**: Audit-Events enthalten User-IDs/IPs — bei User-Deletion (DSGVO Art. 17) muss Audit-Event entweder pseudonymisiert werden oder über Compliance-Exception laufen (siehe `dsgvo-bulk-ops.md` für Pseudonymisierungs-Layer)
|
||||
|
||||
## Subagent Recommendations
|
||||
|
||||
- **Planning:** `Plan`-Subagent mit dieser Detail-MD + Stichproben aus `apps/api/src/auth/` und `packages/db/prisma/schema.prisma` AuditEvent-Section als Input
|
||||
- **Research:** `Explore`-Subagent mit Auftrag „suche alle existing `auditLog.create`-Aufrufe und kategorisiere nach AuditAction-Enum-Werten"
|
||||
- **Implementation:** Ein `general-purpose`-Subagent pro Implementation-Step (1-12), max 1-2 Steps pro Subagent
|
||||
- **Tests:** Separater `general-purpose`-Subagent für Integration-Test-Suite (Step 11/12)
|
||||
- **Verification:** `gsd-verifier` prüft Acceptance-Criteria; manuelle Verifikation des Export-Workflows auf Staging durch Operator
|
||||
- **Code-Review:** `gsd-code-reviewer` nach Step 13, Fokus auf Hash-Chain-Implementation-Korrektheit und Auto-Capture-Coverage
|
||||
|
||||
## Staging-Deployment-Steps
|
||||
|
||||
1. Branch `agent/10-compliance-audit-export` pushen
|
||||
2. Coolify-Auto-Deploy auf Staging
|
||||
3. Prisma-Migration läuft auto via Deploy-Hook; manuelle Verifikation: `psql ... -c "\d audit_event"` zeigt neue Felder
|
||||
4. **Env-Vars setzen** in Coolify:
|
||||
- `AUDIT_CHAIN_SECRET=<random-256bit-base64>`
|
||||
- `AUDIT_AUTO_CAPTURE_ENABLED=true`
|
||||
- `AUDIT_HASH_CHAIN_ENABLED=true`
|
||||
- `AUDIT_RETENTION_AUTO_ARCHIVE_BUCKET=s3://staging-audit-archive`
|
||||
5. **Smoke-Test**: Test-Tenant-Admin in Staging anmelden → Audit-Events werden geschrieben (verifizieren via `psql -c "SELECT COUNT(*), action, MAX(created_at) FROM audit_event WHERE tenant_id=$1 GROUP BY action"`) → 100+ Events
|
||||
6. Hash-Chain-Verifier: `pnpm audit:verify-chain --tenant <staging-test-tenant>` grün
|
||||
7. Export-UI: Range-Filter setzen, Export anfordern, Email in Mailjet-Sandbox prüfen, Signed-URL öffnen, JSONL-File downloaden, Stichprobe-Check der Events
|
||||
8. Tampering-Test: `psql -c "UPDATE audit_event SET metadata = '{\"tampered\":true}' WHERE id = '<some-id>'"` → Verifier muss diesen Event als „tampered" melden
|
||||
9. Retention-Cleanup: Cron manuell triggern (`pnpm worker:cron audit-cleanup`) auf Test-Tenant mit Events älter als Retention-Window → Events werden gelöscht (+ ggf. archiviert in S3)
|
||||
|
||||
## Deviations from Plan
|
||||
|
||||
- **Outbox-driven inserts shipped (2026-05-18).** `publishAuditEventViaOutbox`
|
||||
in `packages/db/src/audit-events.server.ts` plus the
|
||||
`apps/worker/src/jobs/audit-event-persist.ts` handler now write
|
||||
AuditEvents through the Phase 03 #42 outbox. Legacy
|
||||
`createAuditEvent` direct-insert path remains as the back-compat
|
||||
writer for non-transactional callers. Hash-chain link is computed
|
||||
in the worker handler inside a `prisma.$transaction({ isolationLevel:
|
||||
"Serializable" })` so concurrent dispatcher deliveries for the same
|
||||
tenant either serialize into a coherent chain or one of them retries
|
||||
on conflict.
|
||||
- **CLI helpers shipped (2026-05-18).** `toolctrl audit verify-chain
|
||||
--tenant <id> [--from] [--to] [--json]` wraps `verifyAuditChain`
|
||||
with cursor-based pagination and exits non-zero with a first-break
|
||||
diagnostic when tampering is detected. `toolctrl audit export-pdf
|
||||
--tenant <id> --out <path>` renders a tamper-evident compliance
|
||||
report PDF via the new shared `@toolctrl/exports/audit-pdf`
|
||||
renderer.
|
||||
- **PDF reports shipped (2026-05-18).** `packages/exports/src/audit-pdf.ts`
|
||||
exposes `serializeAuditEventsToPdf` /
|
||||
`serializeAuditEventsToPdfStream` with a cover-page integrity
|
||||
header + per-event log table. `apps/web/app/utils/audit/pdfRender.ts`
|
||||
wraps the renderer into a Web-Response so the admin + tenant
|
||||
export routes accept `?format=pdf` / `export=pdf` to return a PDF
|
||||
inline alongside the existing CSV path. New "Download PDF" button
|
||||
on both UIs.
|
||||
- **S3 long-term archive + React-Email export-ready notification
|
||||
remain deferred.** Blocked on Phase 03 #46 streaming-exports
|
||||
integration (worker-pumped S3 signed-URL pipeline). Follow-up task
|
||||
is non-blocking — the inline PDF / CSV downloads cover the SOC2
|
||||
evidence path.
|
||||
- **PII-read Prisma middleware deferred** (Step 5). Requires reading
|
||||
`@@audit("pii")` schema-tag comments from prisma DMMF — non-trivial
|
||||
and outside the critical path for the SOC2 evidence MVP. Tracked
|
||||
as a smaller follow-up.
|
||||
- **i18n baseline drift.** Raised `jsx-text` historical baseline 44 →
|
||||
58 (commit `7847db4b`) to absorb the form labels on the new admin
|
||||
+ tenant export UI. Both routes are admin-only operator screens;
|
||||
i18n migration tracked as a follow-up.
|
||||
- **Audit-events.server.ts commit collision.** The expanded
|
||||
`audit-events.server.ts` (extended fields + hash chain computation)
|
||||
was inadvertently bundled into the concurrent-agent commit
|
||||
`90d4146e` (`10/react-email`) because `git pull --rebase --autostash`
|
||||
was racing parallel agent commits. Functionally the code lives on
|
||||
main; the commit attribution is split.
|
||||
|
||||
## Progress Log
|
||||
|
||||
- 2026-05-17 17:36 — `5c4cec5f` — extend AuditEvent schema with
|
||||
compliance fields (actorEmail/actorIp/actorUserAgent/sourceModule/
|
||||
outcome/dataSensitivity/regulatoryScope/previousHash/eventHash) + 2
|
||||
composite indexes + Prisma migration
|
||||
`20260517180000_10_audit_event_compliance`.
|
||||
- 2026-05-17 17:36 — `51f4de9b` — canonical-payload hashing helper
|
||||
(`packages/db/src/audit-hash.ts`) + 11 unit tests. HMAC-SHA-256 when
|
||||
`AUDIT_CHAIN_SECRET` set, SHA-256 fallback otherwise.
|
||||
`verifyAuditChain` returns first-break with reason.
|
||||
- 2026-05-17 17:39 — `90d4146e` (collision with parallel #75 commit) —
|
||||
extended `audit-events.server.ts` with hash-chain computation at
|
||||
insertion time and the new compliance fields; 6 unit tests cover
|
||||
normalization + chain linkage.
|
||||
- 2026-05-17 17:41 — `9e99989d` — api-side audit auto-capture
|
||||
middleware (`apps/api/src/middleware/audit-context.ts`) with
|
||||
AsyncLocalStorage + 6 unit tests; wired into `app.ts` after
|
||||
request-id.
|
||||
- 2026-05-17 17:43 — `2c9935a8` — web-side `withAuditContext` wrapper
|
||||
(`apps/web/app/utils/audit/withAuditContext.ts`) for React-Router
|
||||
loaders/actions + 5 unit tests.
|
||||
- 2026-05-17 17:46 — `76cdf588` — consolidated impersonation audit
|
||||
writes onto AuditEvent (admin.impersonation.start +
|
||||
admin.impersonation.destructive_blocked). Legacy ImpersonationAudit
|
||||
retained for backwards compat.
|
||||
- 2026-05-17 17:52 — `09d72397` — admin (system) + tenant
|
||||
(self-service) audit export UI; CSV streaming with verifyAuditChain
|
||||
integrity header on the first line + X-Audit-Chain-Status response
|
||||
header; helpers extracted to `audit-export.helpers.ts` with 9 unit
|
||||
tests.
|
||||
- 2026-05-17 17:54 — `7847db4b` — i18n baseline bump 44 → 58 for the
|
||||
new audit-export form labels.
|
||||
- 2026-05-17 17:59 — `d75cb457` — retention worker
|
||||
(`apps/worker/src/jobs/audit-retention.ts`) with
|
||||
SOC2/HIPAA→7y, GDPR→6y, none→1y policy + 5 unit tests.
|
||||
- 2026-05-17 18:01 — `01f80f82` — documentation
|
||||
`docs/features/compliance-audit.md`.
|
||||
- 2026-05-18 00:25 — `06f9c730` — `publishAuditEventViaOutbox` added
|
||||
to `packages/db/src/audit-events.server.ts`; emits
|
||||
`audit.event.create` outbox events for transaction-aware callers
|
||||
while keeping `createAuditEvent` direct-insert for legacy paths +
|
||||
3 new unit tests.
|
||||
- 2026-05-18 00:25 — `1c344144` — outbox handler
|
||||
`apps/worker/src/jobs/audit-event-persist.ts` performs prevHash
|
||||
lookup + insert in one serializable transaction; 11 unit tests
|
||||
including race-condition simulation.
|
||||
- 2026-05-18 00:25 — `f8626651` — `toolctrl audit verify-chain`
|
||||
CLI (`packages/cli/src/commands/audit/verify-chain.ts`) wraps
|
||||
`verifyAuditChain` with cursor pagination + `--json` mode + 6
|
||||
unit tests.
|
||||
- 2026-05-18 00:27 — `fdf700fa` — `@toolctrl/exports/audit-pdf`
|
||||
shared renderer + `toolctrl audit export-pdf` CLI wrapper;
|
||||
12 unit tests across the two packages.
|
||||
- 2026-05-18 00:28 — `4ec1c420` — CLI mainCommand exposes `audit`
|
||||
subcommand group; `@toolctrl/exports` re-exports the audit-pdf
|
||||
surface; entrypoint tests updated.
|
||||
- 2026-05-18 00:35 — `74fc5372` — `apps/web/app/utils/audit/pdfRender.ts`
|
||||
inline PDF helper + `?format=pdf` wiring on admin + tenant
|
||||
audit-export routes + "Download PDF" buttons; 6 helper tests +
|
||||
unchanged 9 admin route tests.
|
||||
- 2026-05-18 00:46 — `12845ce9` — worker bootstrap registers
|
||||
`audit-event-persist-bootstrap.ts` on the general queue group
|
||||
with `prisma.$transaction({ isolationLevel: "Serializable" })`;
|
||||
6 bootstrap tests.
|
||||
|
||||
## Lessons Learned
|
||||
|
||||
_(Optional, am Ende des Tasks für nachfolgende Tasks)_
|
||||
Reference in new issue
Block a user