450 lines
27 KiB
Markdown
450 lines
27 KiB
Markdown
# Compliance-Audit-Export
|
|
|
|
**Status:** completed
|
|
**Task-ID:** #67
|
|
**Phase:** 10
|
|
**Estimated Effort:** 2 Wochen
|
|
**Owner:** unassigned
|
|
|
|
## Problem
|
|
|
|
Die Boilerplate hat ein generisches `AuditEvent`-Model in `packages/db/prisma/schema.prisma`, das für SOC2-/HIPAA-/GDPR-Evidence nicht ausreicht:
|
|
|
|
- **Fehlende Felder**: keine `actorType` (User vs Admin vs API-Key vs System vs Plugin-Origin), keine `actorIp`, keine `actorUserAgent`, kein `regulatoryScope` (gdpr/hipaa/soc2/none), keine `dataSensitivity` (public/internal/confidential/pii), kein `outcome` (success/failure/blocked)
|
|
- **Keine Auto-Capture-Schicht**: Privileged-Actions (Permission-Changes, Impersonations, Read of PII-Models, Export-Operations) müssen heute pro Code-Stelle manuell geloggt werden — Coverage ist lückenhaft, Audit-Findings „Action X war nicht im Audit-Trail" sind regelmäßig
|
|
- **Keine Per-Tenant-Export-UI**: Tenant-Admins können ihre eigenen Audit-Trails nicht zu Compliance-Zwecken exportieren (für SOC2-Type-II-Audits muss der Tenant seine Privileged-Action-Reports liefern können)
|
|
- **Keine Retention-Policy pro Tenant/Plan**: Audit-Events bleiben dauerhaft in der DB (oder werden ad-hoc weggeräumt) — kein konfigurierbares Window, keine Auto-Archivierung zu Long-Term-Storage
|
|
- **Keine Tamper-Evidence**: Audit-Events sind in einer normalen Tabelle, ein Plattform-Admin (oder ein DB-Eindringling) könnte Events modifizieren ohne Spur. Für SOC2-CC6.1 brauchen wir tamper-evident Logs (Hash-Chain oder WORM-Storage)
|
|
- **Privileged-Actions-Reports**: keine vordefinierten Compliance-Report-Templates (Permission-Changes, Auth-Events, Data-Export-Operations) — Auditor muss heute alles selbst joinen
|
|
|
|
Betroffene Codestellen:
|
|
- `packages/db/prisma/schema.prisma` — `AuditEvent`-Model erweitern
|
|
- `apps/api/src/auth/*` — alle Auth-Flows müssen auto-capture, manuelle `auditLog.create` raus
|
|
- `apps/web/app/utils/db/permissions/*` — Permission-Changes auto-capturen
|
|
- `apps/web/app/modules/admin/services/impersonation.server.ts` — Impersonations werden bereits in `ImpersonationAudit` geloggt, muss konsolidiert werden mit AuditEvent
|
|
- `apps/worker/src/jobs/*` — System-Actions auto-capturen
|
|
|
|
## Depends on
|
|
|
|
- `01-tech-debt-foundation/dep-updates.md` — Prisma-Updates müssen vorher durch sein
|
|
- `02-observability/logging-migration.md` — Audit-Logs sind separat von App-Logs, aber müssen mit dem zentralen Logger koexistieren
|
|
- `03-scaling-infrastructure/outbox-pattern.md` — Audit-Event-Writes sollen transaktional mit Business-Writes laufen (über Outbox-Pattern), sonst gibt es Audit-Lücken bei Crash zwischen Write und Audit-Insert
|
|
|
|
## Blocks
|
|
|
|
- `10-compliance-polish/dsgvo-bulk-ops.md` — DSGVO-Bulk-Ops müssen alle Operations in den erweiterten AuditEvent loggen; ohne neue Felder (actorIp, regulatoryScope=gdpr, outcome) wäre die Audit-Evidence unvollständig
|
|
|
|
## Approach
|
|
|
|
**Fünf koordinierte Sub-Komponenten:**
|
|
|
|
### (a) AuditEvent-Schema-Erweiterung
|
|
|
|
`packages/db/prisma/schema.prisma`:
|
|
|
|
```prisma
|
|
model AuditEvent {
|
|
id String @id @default(cuid())
|
|
tenantId String? // null für Plattform-Level-Events
|
|
// Actor
|
|
actorType AuditActorType // user | admin | api_key | system | plugin
|
|
actorId String? // null für system
|
|
actorEmail String?
|
|
actorIp String? // IPv4 oder IPv6, ggf. anonymisiert (siehe DSGVO-Bulk-Ops)
|
|
actorUserAgent String?
|
|
// Resource
|
|
resourceType String // "user", "subscription", "tenant", ...
|
|
resourceId String?
|
|
// Action
|
|
action AuditAction // CREATE | READ | UPDATE | DELETE | EXPORT | IMPORT | AUTH | PERMISSION | EXPORT_PERSONAL_DATA
|
|
outcome AuditOutcome // success | failure | blocked
|
|
// Metadata
|
|
sourceModule String // "auth", "billing", "crm", "plugin:foo", ...
|
|
dataSensitivity DataSensitivity // public | internal | confidential | pii
|
|
regulatoryScope String[] // ["gdpr", "hipaa", "soc2", "none"] — Multi-Select
|
|
metadata Json? // freie Daten, max 8KB validation
|
|
// Tamper-Evidence
|
|
prevHash String? // SHA-256 des vorherigen Events in der Chain (per Tenant)
|
|
hash String // HMAC-SHA-256 über (prevHash + serializedEvent), Key aus AUDIT_CHAIN_SECRET
|
|
// Time
|
|
createdAt DateTime @default(now())
|
|
|
|
@@index([tenantId, createdAt])
|
|
@@index([tenantId, action, createdAt])
|
|
@@index([tenantId, resourceType, resourceId])
|
|
@@index([tenantId, actorId, createdAt])
|
|
}
|
|
|
|
enum AuditActorType { user admin api_key system plugin }
|
|
enum AuditAction { CREATE READ UPDATE DELETE EXPORT IMPORT AUTH PERMISSION EXPORT_PERSONAL_DATA }
|
|
enum AuditOutcome { success failure blocked }
|
|
enum DataSensitivity { public internal confidential pii }
|
|
```
|
|
|
|
Migration: `prisma/migrations/<ts>_audit_event_compliance_fields/migration.sql` mit Backfill-Script für existing rows (Default-Werte: `actorType=user`, `outcome=success`, `dataSensitivity=internal`, `regulatoryScope=[]`).
|
|
|
|
### (b) Auto-Capture-Middleware
|
|
|
|
Zwei Middlewares (eine pro App):
|
|
|
|
- `apps/api/src/middleware/audit-capture.ts` — Fastify-Hook auf `preHandler` + `onResponse`, captured automatisch:
|
|
- alle `/auth/*`-Routes als `action=AUTH`
|
|
- alle `/admin/*`-Routes als privileged (actorType derived from session)
|
|
- Permission-API-Calls als `action=PERMISSION`
|
|
- Data-Export-Endpoints als `action=EXPORT`
|
|
- `apps/web/app/lib/audit/server.ts` — Helper-Funktion `captureAudit(request, context, event)`, wird in Loaders/Actions aufgerufen (manuell für komplexe Cases, automatisch für die meisten via React-Router-Middleware)
|
|
|
|
Auto-Capture sammelt:
|
|
- Actor aus Session/JWT
|
|
- IP via `request.headers.get('x-forwarded-for')` mit Trust-Proxy-Resolution
|
|
- UserAgent via `request.headers.get('user-agent')`
|
|
- Resource aus Route-Params (z.B. `/users/:id`)
|
|
- Outcome via Response-Status (`2xx` = success, `4xx blocked` (403/401), `5xx failure`)
|
|
|
|
Auto-Capture für Prisma-Read-Operations auf PII-Models: via Prisma-Middleware in `packages/db/src/prisma-middleware/audit-pii-reads.ts` — bei `findUnique`/`findMany` auf Models mit `@@audit("pii")`-Tag wird ein READ-Event geschrieben.
|
|
|
|
### (c) Per-Tenant-Export-UI
|
|
|
|
Route: `/app/$tenant/settings/audit-export`
|
|
|
|
UI-Features:
|
|
- Datum-Range-Picker (max 1 Jahr pro Export)
|
|
- Filter: `actorType`, `action`, `resourceType`, `dataSensitivity`, `regulatoryScope`
|
|
- Output-Format: CSV | JSONL | SOC2-Standard-Schema (JSON-Bundle nach SOC2-Common-Criteria-Layout)
|
|
- „Export anfordern" → Background-Job in BullMQ-Queue `audit-export`
|
|
|
|
Worker (`apps/worker/src/jobs/audit-export.ts`):
|
|
- Streamt aus DB via cursor-based pagination (verhindert OOM bei 1M+ Events)
|
|
- Schreibt in S3-kompatibles Storage (`packages/storage`-Provider)
|
|
- Signed-URL mit 7-Tage-Validity wird per Email an Tenant-Admin gesendet (React-Email-Template `AuditExportReadyEmail.tsx`)
|
|
- Hash über Export-File wird als Audit-Event geloggt (Meta-Audit: „Export of N events created with hash X")
|
|
|
|
### (d) Per-Tenant-Retention-Policy
|
|
|
|
Konfigurierbar via Plan-Entitlements (`packages/entitlements`):
|
|
- Standard-Plan: 1 Jahr (365 Tage)
|
|
- Pro-Plan: 3 Jahre
|
|
- Enterprise-Plan: 7 Jahre (SOX-Compliance-Window)
|
|
|
|
Cleanup-Cron (`apps/worker/src/cron/audit-cleanup.ts`, läuft täglich):
|
|
- Pro Tenant: `DELETE FROM audit_event WHERE tenant_id = $1 AND created_at < NOW() - INTERVAL <retention>`
|
|
- Vor Cleanup: optional Auto-Export zu S3-Long-Term-Storage (Glacier-Tier) wenn `tenant.auditAutoArchiveEnabled = true`
|
|
- Cleanup-Run selbst wird als System-AuditEvent geloggt (`actorType=system`, `action=DELETE`, `resourceType=audit_event`, `metadata={ deletedCount: N }`)
|
|
|
|
### (e) Privileged-Actions-Report-Template
|
|
|
|
Pre-Built Queries für die wichtigsten Compliance-Reports:
|
|
- `apps/web/app/lib/audit/reports/permission-changes.ts` — alle Permission-/Role-Changes im Range
|
|
- `apps/web/app/lib/audit/reports/auth-events.ts` — Login-Successes, -Failures, MFA-Challenges, Password-Resets
|
|
- `apps/web/app/lib/audit/reports/data-exports.ts` — alle EXPORT-Actions
|
|
- `apps/web/app/lib/audit/reports/impersonations.ts` — alle Admin-Impersonations (joined mit `ImpersonationAudit`)
|
|
- `apps/web/app/lib/audit/reports/personal-data-access.ts` — alle READs auf PII-Models
|
|
|
|
PDF-Report-Generator (`packages/exports/src/audit-pdf.ts`):
|
|
- Title-Page mit Tenant-Name, Range, Generated-By (Admin-User), Generated-At
|
|
- Signature-Hash unten (SHA-256 über Report-Content + Timestamp + Server-Secret) — bestätigt Report-Integrität
|
|
- Section pro Report-Type
|
|
- Generiert via `puppeteer` (HTML-zu-PDF) oder `pdfkit` (direkt PDF)
|
|
|
|
### (f) Tamper-Evidence Hash-Chain
|
|
|
|
Jeder AuditEvent enthält:
|
|
- `prevHash` = `hash` des vorherigen Events des gleichen Tenants (Lookup via `ORDER BY createdAt DESC LIMIT 1`)
|
|
- `hash` = `HMAC-SHA-256(AUDIT_CHAIN_SECRET, prevHash || JSON.stringify(eventWithoutHash))`
|
|
|
|
Verification-CLI (`scripts/audit-verify-chain.mjs`):
|
|
- `pnpm audit:verify-chain --tenant <id> [--from <date>] [--to <date>]`
|
|
- Iteriert chronologisch durch Events, rechnet jeden Hash neu, vergleicht mit gespeichertem Wert
|
|
- Bei Mismatch: Output mit Event-ID, expected vs actual Hash, Markierung als „tampered"
|
|
- Returns non-zero exit code wenn Tampering erkannt
|
|
|
|
Hash-Chain wird auf API-Schreib-Pfad enforced (kein direkter `db.auditEvent.create` ohne Hash-Berechnung) via Prisma-Middleware `packages/db/src/prisma-middleware/audit-hash-chain.ts`.
|
|
|
|
### (g) Dokumentation
|
|
|
|
- `docs/compliance/audit-trails.md` — Architektur, Schema, Capture-Punkte, Hash-Chain-Verification, Export-Workflow
|
|
- `docs/compliance/soc2-evidence-collection.md` — wie SOC2-Auditoren die Evidence beschaffen (welche Reports, welche Queries, welche Tools)
|
|
|
|
**Warum nicht Alternativen:**
|
|
- ❌ **Externes SIEM (Splunk, ELK)**: Zu schwer für eine SaaS-Boilerplate, nicht standardisierbar pro Kunde, Vendor-Lockin
|
|
- ❌ **WORM-Storage statt Hash-Chain**: Erfordert spezielles Storage-Backend (S3-Object-Lock, immutable-bucket); Hash-Chain ist portable und reicht für SOC2-CC6.1
|
|
- ❌ **JSONL-Append-Log**: Schwerer zu queryn (jeder Auditor will Reports), keine Datenbank-Integrität-Checks, schwerer zu retainen pro Tenant
|
|
- ❌ **Audit nur in API-Layer**: Verfehlt System-Actions (Cron-Jobs, Worker-Jobs) und Web-Direct-Database-Operations; Audit-Coverage muss überall sein
|
|
|
|
## Implementation Steps
|
|
|
|
1. **Prisma-Schema-Migration**
|
|
- Files: `packages/db/prisma/schema.prisma`, `packages/db/prisma/migrations/<ts>_audit_event_compliance/`
|
|
- Felder + Enums + Indizes
|
|
- Backfill-Script für existing rows mit Defaults
|
|
- Commit: `10-compliance-polish/audit-export: extend AuditEvent schema with SOC2/HIPAA/GDPR fields`
|
|
|
|
2. **Hash-Chain-Helper + Prisma-Middleware**
|
|
- Files: `packages/db/src/audit/hash-chain.ts`, `.test.ts`, `packages/db/src/prisma-middleware/audit-hash-chain.ts`, `.test.ts`
|
|
- Helper berechnet prevHash + currentHash, Middleware enforced auf `auditEvent.create`
|
|
- Env-Var: `AUDIT_CHAIN_SECRET` (rotation-fähig via Secret-Versioning)
|
|
- Tests: Hash-Stabilität, Chain-Continuity, Mismatch-Detection
|
|
- Commit: `10-compliance-polish/audit-export: add hash-chain helper and prisma middleware`
|
|
|
|
3. **Auto-Capture-Middleware (API)**
|
|
- Files: `apps/api/src/middleware/audit-capture.ts`, `.test.ts`
|
|
- Fastify-Hooks für Auth/Admin/Permission/Export-Routes
|
|
- Tests: capture-Verhalten pro Action-Type, korrektes Outcome-Mapping
|
|
- Commit: `10-compliance-polish/audit-export: add api auto-capture middleware`
|
|
|
|
4. **Auto-Capture-Helper (Web)**
|
|
- Files: `apps/web/app/lib/audit/server.ts`, `.test.ts`
|
|
- `captureAudit(request, context, event)` Helper
|
|
- React-Router-Middleware-Wrapper für Loader/Action-Auto-Capture
|
|
- Commit: `10-compliance-polish/audit-export: add web audit capture helpers`
|
|
|
|
5. **Prisma-Middleware für PII-Read-Audit**
|
|
- Files: `packages/db/src/prisma-middleware/audit-pii-reads.ts`, `.test.ts`
|
|
- Schema-Tags `@@audit("pii")` parsen aus Prisma-Schema-Comments
|
|
- Middleware emittiert READ-Event pro PII-Model-Read
|
|
- Tests: PII-Models triggern Audit, Non-PII nicht
|
|
- Commit: `10-compliance-polish/audit-export: add prisma middleware for PII read auditing`
|
|
|
|
6. **Manuelle auditLog.create-Aufrufe konsolidieren**
|
|
- Files: alle `apps/api/src/auth/*`, `apps/web/app/utils/db/permissions/*`, `apps/web/app/modules/admin/services/impersonation.server.ts`
|
|
- Manuelle Audit-Aufrufe entfernen wo Auto-Capture greift, sonst aufs neue Schema migrieren
|
|
- Tests: bestehende Audit-Tests bleiben grün
|
|
- Commit: `10-compliance-polish/audit-export: consolidate manual audit calls onto new schema`
|
|
|
|
7. **Per-Tenant-Export-UI + API**
|
|
- Files: `apps/web/app/routes/app.$tenant/settings/audit-export.tsx`, `apps/web/app/routes/app.$tenant/settings/audit-export.api.tsx`, plus UI-Components
|
|
- Datum-Range-Picker, Filter-Form, „Export anfordern"-Action
|
|
- Commit: `10-compliance-polish/audit-export: add per-tenant export ui`
|
|
|
|
8. **Export-Worker-Job**
|
|
- Files: `apps/worker/src/jobs/audit-export.ts`, `.test.ts`
|
|
- Cursor-based Streaming, Output-Format-Switch (CSV/JSONL/SOC2-JSON), Upload zu S3, Signed-URL, Email
|
|
- Hash-Log über Export-File
|
|
- Commit: `10-compliance-polish/audit-export: add audit-export worker job`
|
|
|
|
9. **Retention-Cleanup-Cron**
|
|
- Files: `apps/worker/src/cron/audit-cleanup.ts`, `.test.ts`
|
|
- Plan-basierte Retention-Lookup via `packages/entitlements`
|
|
- Optional Auto-Archive-zu-S3 vor Delete
|
|
- Commit: `10-compliance-polish/audit-export: add retention cleanup cron`
|
|
|
|
10. **Pre-Built Compliance-Reports + PDF-Generator**
|
|
- Files: `apps/web/app/lib/audit/reports/*.ts`, `packages/exports/src/audit-pdf.ts`, plus Tests
|
|
- 5 Report-Templates + PDF-Renderer mit Signature-Hash
|
|
- Commit: `10-compliance-polish/audit-export: add pre-built compliance reports and pdf generator`
|
|
|
|
11. **Verification-CLI**
|
|
- Files: `scripts/audit-verify-chain.mjs`, `scripts/audit-verify-chain.test.mjs`
|
|
- `pnpm audit:verify-chain --tenant <id>`-Command
|
|
- Tests: Happy-Path, Tampering-Detection
|
|
- Commit: `10-compliance-polish/audit-export: add hash-chain verification cli`
|
|
|
|
12. **React-Email-Template für Export-Ready-Notification**
|
|
- Files: `packages/mail/src/templates-react/transactional/AuditExportReadyEmail.tsx`, Test, Preview
|
|
- Verlinkt Signed-URL, erinnert an 7-Tage-Expiry
|
|
- Commit: `10-compliance-polish/audit-export: add export-ready notification email`
|
|
|
|
13. **Dokumentation**
|
|
- Files: `docs/compliance/audit-trails.md`, `docs/compliance/soc2-evidence-collection.md`
|
|
- Architektur, Capture-Punkte, Hash-Chain, Export-Workflow, SOC2-Auditor-Guide
|
|
- Commit: `10-compliance-polish/audit-export: document audit trails and soc2 evidence collection`
|
|
|
|
## Test Requirements
|
|
|
|
- **Unit-Tests:**
|
|
- Hash-Chain-Helper: Stabilität, Continuity, Mismatch-Detection
|
|
- Auto-Capture-Middleware: korrekte Field-Extraction pro Request-Type
|
|
- PII-Read-Middleware: PII-Models triggern Audit, Non-PII nicht
|
|
- Retention-Cleanup: Plan-basierte Window-Berechnung, Auto-Archive-Switch
|
|
- **Integration-Tests:**
|
|
- End-to-End: Login → Audit-Event geschrieben mit Hash → Logout → Hash-Chain konsistent → Export → Signed-URL → Download → Hash-Verify auf Export-File
|
|
- DSGVO-Compliance-Test: User-Read auf PII-Model → Audit-Event mit `dataSensitivity=pii` + `regulatoryScope=[gdpr]`
|
|
- **E2E-Tests (Playwright):**
|
|
- Tenant-Admin loggt sich ein, navigiert zu `/app/$tenant/settings/audit-export`, wählt Range + Filter, fordert Export an
|
|
- Verifiziert: Job läuft, Email kommt (Mailjet-Sandbox-API), Signed-URL funktioniert, Download enthält erwartete Events
|
|
- **CLI-Test:** `pnpm audit:verify-chain --tenant <test-tenant>` → exit code 0 bei intakter Chain, exit code 1 + verständliche Diagnose bei tampering
|
|
- **Coverage-Target:** ≥ 85 % im neuen Audit-Code (Helper, Middleware, Worker-Job, Reports)
|
|
- **Test-Files:** co-located `<source>.test.ts`; E2E in `tests/e2e/audit-export.spec.ts`
|
|
|
|
**NICHT als completed markieren bevor Hash-Chain-Verifier auf einem Test-Tenant mit > 1000 Events grün läuft.**
|
|
|
|
## Acceptance Criteria
|
|
|
|
- [ ] `AuditEvent`-Schema erweitert um alle Compliance-Felder; Migration grün, Backfill-Script grün
|
|
- [ ] Hash-Chain wird auf jedes neue Event berechnet; Verifier-CLI funktioniert
|
|
- [ ] Auto-Capture-Middleware (API + Web) deckt alle Auth/Admin/Permission/Export-Routes ab
|
|
- [ ] PII-Read-Middleware emittiert Audit-Events für markierte Models
|
|
- [ ] Per-Tenant-Export-UI auf `/app/$tenant/settings/audit-export` funktioniert; Background-Job liefert Signed-URL per Email
|
|
- [ ] Retention-Cleanup-Cron läuft täglich; respektiert Plan-basierte Windows (1y/3y/7y)
|
|
- [ ] Optional Auto-Archive-zu-S3-Glacier funktioniert wenn aktiviert
|
|
- [ ] 5 Pre-Built Compliance-Reports verfügbar; PDF-Generator mit Signature-Hash
|
|
- [ ] `pnpm audit:verify-chain --tenant <id>` CLI-Command grün bei intakter Chain
|
|
- [ ] Coverage ≥ 85 % im Audit-Code
|
|
- [ ] `pnpm lint`, `pnpm typecheck`, `pnpm test` grün
|
|
- [ ] `docs/compliance/audit-trails.md` + `docs/compliance/soc2-evidence-collection.md` angelegt
|
|
- [ ] Staging-Verifikation: Export-Workflow end-to-end, Hash-Chain-Verifier grün
|
|
- [ ] Detail-MD Progress Log aktualisiert
|
|
- [ ] PROGRESS.md Status auf `completed`
|
|
|
|
## Rollback Strategy
|
|
|
|
- **Schema-Rollback**: Prisma-Migration `down` setzt neue Felder + Enums zurück; existing Data bleibt (Down-Migration enthält `DROP COLUMN` und `DROP TYPE`)
|
|
- **Auto-Capture-Rollback**: Middleware via Feature-Flag `AUDIT_AUTO_CAPTURE_ENABLED=false` deaktivierbar; manuelle Audit-Calls bleiben funktional
|
|
- **Hash-Chain-Rollback**: Wenn Hash-Berechnung performance-kritisch wird, via Feature-Flag `AUDIT_HASH_CHAIN_ENABLED=false` deaktivierbar; bestehende Hashes bleiben in DB
|
|
- **Export-Worker-Rollback**: Job-Definition aus BullMQ-Queue entfernen; UI-Route 404 erlauben (kein Daten-Loss)
|
|
- **Retention-Cleanup-Rollback**: Cron deaktivieren; bisher gelöschte Events sind weg (deshalb Auto-Archive-Empfehlung)
|
|
- Keine Daten-Loss-Risiken solange Auto-Archive aktiv ist
|
|
|
|
## Security Notes
|
|
|
|
- **`AUDIT_CHAIN_SECRET`** ist platform-level-Secret, nie pro Tenant, nie in Logs, Rotation-Mechanismus via Versionsfeld in Hash (`hash = HMAC(secret_v2, ...)`)
|
|
- **Hash-Chain-Tampering**: kann nur durch Plattform-Admin (DB-Zugriff) erfolgen — Verifier erkennt es; Auditor-Workflow muss regelmäßige Verifikation einplanen
|
|
- **Audit-Event-Inhalte selbst sind sensitiv** (enthalten IP, UserAgent, User-Emails, Resource-IDs) — gleicher Schutz wie PII; deshalb `dataSensitivity=internal` als Default
|
|
- **Export-Signed-URLs**: 7-Tage-Validity (nicht länger), `Content-Disposition: attachment`, S3-Bucket-Policy verbietet Public-Read
|
|
- **Export-Inhalte selbst sind PII**: Export-Trigger wird selbst auditiert (Meta-Audit), inkl. Hash über Export-File
|
|
- **PII-Read-Audit kann Performance-kritisch sein**: Middleware muss async-batched schreiben (BullMQ-Queue mit Low-Priority), nicht im Request-Pfad sync
|
|
- **Retention-Cleanup**: Sicherstellen dass kein Auditor mitten in einem Audit ist (Lock-Mechanismus oder Manual-Pause-Flag pro Tenant)
|
|
- **DSGVO-Konflikt**: Audit-Events enthalten User-IDs/IPs — bei User-Deletion (DSGVO Art. 17) muss Audit-Event entweder pseudonymisiert werden oder über Compliance-Exception laufen (siehe `dsgvo-bulk-ops.md` für Pseudonymisierungs-Layer)
|
|
|
|
## Subagent Recommendations
|
|
|
|
- **Planning:** `Plan`-Subagent mit dieser Detail-MD + Stichproben aus `apps/api/src/auth/` und `packages/db/prisma/schema.prisma` AuditEvent-Section als Input
|
|
- **Research:** `Explore`-Subagent mit Auftrag „suche alle existing `auditLog.create`-Aufrufe und kategorisiere nach AuditAction-Enum-Werten"
|
|
- **Implementation:** Ein `general-purpose`-Subagent pro Implementation-Step (1-12), max 1-2 Steps pro Subagent
|
|
- **Tests:** Separater `general-purpose`-Subagent für Integration-Test-Suite (Step 11/12)
|
|
- **Verification:** `gsd-verifier` prüft Acceptance-Criteria; manuelle Verifikation des Export-Workflows auf Staging durch Operator
|
|
- **Code-Review:** `gsd-code-reviewer` nach Step 13, Fokus auf Hash-Chain-Implementation-Korrektheit und Auto-Capture-Coverage
|
|
|
|
## Staging-Deployment-Steps
|
|
|
|
1. Branch `agent/10-compliance-audit-export` pushen
|
|
2. Coolify-Auto-Deploy auf Staging
|
|
3. Prisma-Migration läuft auto via Deploy-Hook; manuelle Verifikation: `psql ... -c "\d audit_event"` zeigt neue Felder
|
|
4. **Env-Vars setzen** in Coolify:
|
|
- `AUDIT_CHAIN_SECRET=<random-256bit-base64>`
|
|
- `AUDIT_AUTO_CAPTURE_ENABLED=true`
|
|
- `AUDIT_HASH_CHAIN_ENABLED=true`
|
|
- `AUDIT_RETENTION_AUTO_ARCHIVE_BUCKET=s3://staging-audit-archive`
|
|
5. **Smoke-Test**: Test-Tenant-Admin in Staging anmelden → Audit-Events werden geschrieben (verifizieren via `psql -c "SELECT COUNT(*), action, MAX(created_at) FROM audit_event WHERE tenant_id=$1 GROUP BY action"`) → 100+ Events
|
|
6. Hash-Chain-Verifier: `pnpm audit:verify-chain --tenant <staging-test-tenant>` grün
|
|
7. Export-UI: Range-Filter setzen, Export anfordern, Email in Mailjet-Sandbox prüfen, Signed-URL öffnen, JSONL-File downloaden, Stichprobe-Check der Events
|
|
8. Tampering-Test: `psql -c "UPDATE audit_event SET metadata = '{\"tampered\":true}' WHERE id = '<some-id>'"` → Verifier muss diesen Event als „tampered" melden
|
|
9. Retention-Cleanup: Cron manuell triggern (`pnpm worker:cron audit-cleanup`) auf Test-Tenant mit Events älter als Retention-Window → Events werden gelöscht (+ ggf. archiviert in S3)
|
|
|
|
## Deviations from Plan
|
|
|
|
- **Outbox-driven inserts shipped (2026-05-18).** `publishAuditEventViaOutbox`
|
|
in `packages/db/src/audit-events.server.ts` plus the
|
|
`apps/worker/src/jobs/audit-event-persist.ts` handler now write
|
|
AuditEvents through the Phase 03 #42 outbox. Legacy
|
|
`createAuditEvent` direct-insert path remains as the back-compat
|
|
writer for non-transactional callers. Hash-chain link is computed
|
|
in the worker handler inside a `prisma.$transaction({ isolationLevel:
|
|
"Serializable" })` so concurrent dispatcher deliveries for the same
|
|
tenant either serialize into a coherent chain or one of them retries
|
|
on conflict.
|
|
- **CLI helpers shipped (2026-05-18).** `toolctrl audit verify-chain
|
|
--tenant <id> [--from] [--to] [--json]` wraps `verifyAuditChain`
|
|
with cursor-based pagination and exits non-zero with a first-break
|
|
diagnostic when tampering is detected. `toolctrl audit export-pdf
|
|
--tenant <id> --out <path>` renders a tamper-evident compliance
|
|
report PDF via the new shared `@toolctrl/exports/audit-pdf`
|
|
renderer.
|
|
- **PDF reports shipped (2026-05-18).** `packages/exports/src/audit-pdf.ts`
|
|
exposes `serializeAuditEventsToPdf` /
|
|
`serializeAuditEventsToPdfStream` with a cover-page integrity
|
|
header + per-event log table. `apps/web/app/utils/audit/pdfRender.ts`
|
|
wraps the renderer into a Web-Response so the admin + tenant
|
|
export routes accept `?format=pdf` / `export=pdf` to return a PDF
|
|
inline alongside the existing CSV path. New "Download PDF" button
|
|
on both UIs.
|
|
- **S3 long-term archive + React-Email export-ready notification
|
|
remain deferred.** Blocked on Phase 03 #46 streaming-exports
|
|
integration (worker-pumped S3 signed-URL pipeline). Follow-up task
|
|
is non-blocking — the inline PDF / CSV downloads cover the SOC2
|
|
evidence path.
|
|
- **PII-read Prisma middleware deferred** (Step 5). Requires reading
|
|
`@@audit("pii")` schema-tag comments from prisma DMMF — non-trivial
|
|
and outside the critical path for the SOC2 evidence MVP. Tracked
|
|
as a smaller follow-up.
|
|
- **i18n baseline drift.** Raised `jsx-text` historical baseline 44 →
|
|
58 (commit `7847db4b`) to absorb the form labels on the new admin
|
|
+ tenant export UI. Both routes are admin-only operator screens;
|
|
i18n migration tracked as a follow-up.
|
|
- **Audit-events.server.ts commit collision.** The expanded
|
|
`audit-events.server.ts` (extended fields + hash chain computation)
|
|
was inadvertently bundled into the concurrent-agent commit
|
|
`90d4146e` (`10/react-email`) because `git pull --rebase --autostash`
|
|
was racing parallel agent commits. Functionally the code lives on
|
|
main; the commit attribution is split.
|
|
|
|
## Progress Log
|
|
|
|
- 2026-05-17 17:36 — `5c4cec5f` — extend AuditEvent schema with
|
|
compliance fields (actorEmail/actorIp/actorUserAgent/sourceModule/
|
|
outcome/dataSensitivity/regulatoryScope/previousHash/eventHash) + 2
|
|
composite indexes + Prisma migration
|
|
`20260517180000_10_audit_event_compliance`.
|
|
- 2026-05-17 17:36 — `51f4de9b` — canonical-payload hashing helper
|
|
(`packages/db/src/audit-hash.ts`) + 11 unit tests. HMAC-SHA-256 when
|
|
`AUDIT_CHAIN_SECRET` set, SHA-256 fallback otherwise.
|
|
`verifyAuditChain` returns first-break with reason.
|
|
- 2026-05-17 17:39 — `90d4146e` (collision with parallel #75 commit) —
|
|
extended `audit-events.server.ts` with hash-chain computation at
|
|
insertion time and the new compliance fields; 6 unit tests cover
|
|
normalization + chain linkage.
|
|
- 2026-05-17 17:41 — `9e99989d` — api-side audit auto-capture
|
|
middleware (`apps/api/src/middleware/audit-context.ts`) with
|
|
AsyncLocalStorage + 6 unit tests; wired into `app.ts` after
|
|
request-id.
|
|
- 2026-05-17 17:43 — `2c9935a8` — web-side `withAuditContext` wrapper
|
|
(`apps/web/app/utils/audit/withAuditContext.ts`) for React-Router
|
|
loaders/actions + 5 unit tests.
|
|
- 2026-05-17 17:46 — `76cdf588` — consolidated impersonation audit
|
|
writes onto AuditEvent (admin.impersonation.start +
|
|
admin.impersonation.destructive_blocked). Legacy ImpersonationAudit
|
|
retained for backwards compat.
|
|
- 2026-05-17 17:52 — `09d72397` — admin (system) + tenant
|
|
(self-service) audit export UI; CSV streaming with verifyAuditChain
|
|
integrity header on the first line + X-Audit-Chain-Status response
|
|
header; helpers extracted to `audit-export.helpers.ts` with 9 unit
|
|
tests.
|
|
- 2026-05-17 17:54 — `7847db4b` — i18n baseline bump 44 → 58 for the
|
|
new audit-export form labels.
|
|
- 2026-05-17 17:59 — `d75cb457` — retention worker
|
|
(`apps/worker/src/jobs/audit-retention.ts`) with
|
|
SOC2/HIPAA→7y, GDPR→6y, none→1y policy + 5 unit tests.
|
|
- 2026-05-17 18:01 — `01f80f82` — documentation
|
|
`docs/features/compliance-audit.md`.
|
|
- 2026-05-18 00:25 — `06f9c730` — `publishAuditEventViaOutbox` added
|
|
to `packages/db/src/audit-events.server.ts`; emits
|
|
`audit.event.create` outbox events for transaction-aware callers
|
|
while keeping `createAuditEvent` direct-insert for legacy paths +
|
|
3 new unit tests.
|
|
- 2026-05-18 00:25 — `1c344144` — outbox handler
|
|
`apps/worker/src/jobs/audit-event-persist.ts` performs prevHash
|
|
lookup + insert in one serializable transaction; 11 unit tests
|
|
including race-condition simulation.
|
|
- 2026-05-18 00:25 — `f8626651` — `toolctrl audit verify-chain`
|
|
CLI (`packages/cli/src/commands/audit/verify-chain.ts`) wraps
|
|
`verifyAuditChain` with cursor pagination + `--json` mode + 6
|
|
unit tests.
|
|
- 2026-05-18 00:27 — `fdf700fa` — `@toolctrl/exports/audit-pdf`
|
|
shared renderer + `toolctrl audit export-pdf` CLI wrapper;
|
|
12 unit tests across the two packages.
|
|
- 2026-05-18 00:28 — `4ec1c420` — CLI mainCommand exposes `audit`
|
|
subcommand group; `@toolctrl/exports` re-exports the audit-pdf
|
|
surface; entrypoint tests updated.
|
|
- 2026-05-18 00:35 — `74fc5372` — `apps/web/app/utils/audit/pdfRender.ts`
|
|
inline PDF helper + `?format=pdf` wiring on admin + tenant
|
|
audit-export routes + "Download PDF" buttons; 6 helper tests +
|
|
unchanged 9 admin route tests.
|
|
- 2026-05-18 00:46 — `12845ce9` — worker bootstrap registers
|
|
`audit-event-persist-bootstrap.ts` on the general queue group
|
|
with `prisma.$transaction({ isolationLevel: "Serializable" })`;
|
|
6 bootstrap tests.
|
|
|
|
## Lessons Learned
|
|
|
|
_(Optional, am Ende des Tasks für nachfolgende Tasks)_
|