33 KiB
Boilerplate-Inventur: Integrationen, KI & Apps-Engine
Faktische Inventur des Boilerplate-Bereichs "Integrationen, KI & Apps-Engine". Quelle (NUR LESEN):
/Users/raffaelsheikh/Documents/GitHub/boilerplateErstellt fuer den Modul-Abgleich gegen HiveGrid 2.0. Alle Pfade absolut. Nichts erfunden — jeder Eintrag verweist auf konkrete Datei(en).
Inhaltsverzeichnis / abgedeckte Pakete
| Paket | npm-Name | Zweck (kurz) |
|---|---|---|
packages/ai |
@toolctrl/ai |
KI-Provider-Layer (Mistral/Scaleway/Aleph Alpha), Multi-Provider-Fallback, Embeddings/RAG-Pipeline, Cost-Tracking, Prompt-/Tool-Registry, Guards, Streaming |
packages/vector-store |
@toolctrl/vector-store |
Vektor-Store-Adapter (pgvector + Qdrant) hinter einem Interface, tenant-scoped Wrapper |
packages/search |
@toolctrl/search |
Typisierter Meilisearch-Wrapper, tenant-scoped Such-Index, Index-Schemas, Outbox-Publisher |
packages/plugins |
@toolctrl/plugins |
Feature-Manifest-Schema + Loader (Apps-/Feature-Inventur-Manifeste) |
packages/plugin-runtime |
@toolctrl/plugin-runtime |
WASM-Sandbox, Capability-Gateway, Manifest, Signatur, Hooks, Audit-Chain, Crash-Tracker, Migrations |
packages/plugin-sdk |
@toolctrl/plugin-sdk |
Typdefinitionen + Settings-Iframe-Client fuer Plugin-Autoren (defineHook, HostApi) |
packages/storage |
@toolctrl/storage |
S3/Scaleway-Storage-Adapter + lokaler Adapter, Signed-URLs, Virus-Scan (ClamAV/magic-bytes), Quarantaene |
packages/realtime |
@toolctrl/realtime |
SSE/Realtime-Event-Serialisierung + Client/Server-Helfer (Notifications) |
packages/mail |
@toolctrl/mail |
MailerSend-Provider, Queued-Mail-Provider, React-Email Transaktionstemplates |
packages/email-templates |
@toolctrl/email-templates |
React-basierte Sequenz-Email-Templates (welcome, trial-reminder, re-activation) |
packages/email-sequences |
@toolctrl/email-sequences |
Reine Logik: Sequenz-Engine, Condition-DSL, Step-Runner, Unsubscribe-Token |
packages/marketing-outbound |
@toolctrl/marketing-outbound |
Outbound-Webhook-Adapter (Slack, Discord, Mailchimp, HubSpot, Brevo, Custom) |
packages/queue |
@toolctrl/queue |
BullMQ-Wrapper: Queue-Namen, typisierte Producer/Jobs, Idempotenz, Prioritaeten, Redis-Connection |
Konvention durchgaengig: jedes Paket exportiert exports mit . -> ./src/index.ts (Source-Exports, kein Build-Schritt zwingend), Scripts meist test / typecheck (teils build, preview). Pakete sind bewusst dependency-arm und lesen nie process.env direkt — Boundary-Code (apps/api, apps/worker) reicht env-Snapshots herein (Env-Access-Enforcement-Tests).
1. @toolctrl/ai — KI-Provider-Layer, Fallback, RAG, Cost
Pfad: /Users/raffaelsheikh/Documents/GitHub/boilerplate/packages/ai
Deps: @ai-sdk/mistral, @ai-sdk/openai-compatible, ai (Vercel AI SDK), gpt-tokenizer, @toolctrl/entitlements
Subpath-Exports: ., ./cost, ./embeddings, ./embeddings/providers/{openai,scaleway,aleph-alpha,anthropic}
1.1 Provider-Adapter & Multi-Provider-Fallback
| Asset | Pfad | Was |
|---|---|---|
| Provider-Adapter-Interface + Registry | packages/ai/src/index.ts |
AIProviderAdapter, AIProviderName = "mistral" | "scaleway" | "aleph-alpha", getAIProviderAdapter(), getAIProviderAdapterForModel(), getPrimaryProvider(), getAIProviderDiagnostics(), Modell-Katalog aiModels (mit Preisen pro Mio Tokens), defaultAIModel = "mistral-large-latest", AIProviderAdapterError mit typisierten Codes |
| Mistral-Provider | packages/ai/src/providers/mistral.ts |
createMistralLanguageModel() via @ai-sdk/mistral createMistral, resolveMistralModel(env), MISTRAL_API_KEY/MISTRAL_MODEL |
| Scaleway-Provider (OpenAI-kompatibel) | packages/ai/src/providers/scaleway.ts |
createScalewayLanguageModel(), Default-BaseURL https://api.scaleway.ai/v1, Default-Modell mistral-small-3.2-24b-instruct-2506, SCALEWAY_API_KEY/SCALEWAY_BASE_URL/SCALEWAY_MODEL |
| Aleph-Alpha-Provider (OpenAI-kompatibel) | packages/ai/src/providers/aleph-alpha.ts |
createAlephAlphaLanguageModel(), Default-BaseURL https://api.aleph-alpha.com/v1, Default-Modell pharia-1-llm-7b-control, ALEPH_ALPHA_API_KEY/...BASE_URL/...MODEL |
| OpenAI-Compatible-Bruecke | packages/ai/src/providers/compatible.ts |
createCompatibleLanguageModel() ueber @ai-sdk/openai-compatible — gemeinsame Basis fuer Scaleway + Aleph Alpha |
| Health-Tracker + Fallback | packages/ai/src/fallback.ts |
AIProviderHealthTracker (Failure-Window 30 min), getFallbackProviderSelection(), fallbackProviderOrder = ["mistral","scaleway","aleph-alpha"] — durchlaeuft Provider in fester Reihenfolge, ueberspringt ungesunde/unkonfigurierte |
HiveGrid-Relevanz: Kern fuer jedes KI-Modul (Chatbot, Prompt-Flows, Generieren, RAG-Antworten). EU-Provider Scaleway/Aleph Alpha = DSGVO-Argument. Multi-Provider-Fallback deckt Querschnitt "KI-Verfuegbarkeit/Resilienz" ab.
1.2 Embeddings-Pipeline (RAG-Schreib-/Leseseite)
Pfad: packages/ai/src/embeddings — Subpath @toolctrl/ai/embeddings
| Asset | Pfad | Was |
|---|---|---|
| Provider-Interface + Typen | packages/ai/src/embeddings/types.ts |
EmbeddingProvider (embed/embedSingle, dimensions, maxBatchSize), EmbeddingProviderName = "openai" | "scaleway" | "aleph-alpha" | "anthropic", EmbeddingProviderError mit typisierten Codes, EmbeddingProviderEnv-Snapshot |
| Factory + Registry | packages/ai/src/embeddings/factory.ts |
createEmbeddingProvider(), registerEmbeddingProviderAdapter() (Adapter registrieren sich selbst), resolveEmbeddingProviderName(env) (Default openai->scaleway->aleph-alpha), resolveEmbeddingDimensions(env) |
| Embed-Adapter (opt-in subpaths) | packages/ai/src/embeddings/providers/{openai,scaleway,aleph-alpha,anthropic}.ts |
Scaleway-Default bge-multilingual-gemma2 (1024 dim), Batch 64, max 4096 Tokens; Anthropic ist bewusst Stub/Platzhalter |
| Schreibseite | packages/ai/src/embeddings/embed-and-upsert.ts |
embedAndUpsert() — chunked, embeds, upsert in VectorUpsertSink; erzwingt tenantId im Payload; deterministische buildChunkId(); optionale Kostenverfolgung ueber executeLLMCall (capability embedding) |
| Leseseite | packages/ai/src/embeddings/retrieve.ts |
retrieveRelevantContext() — Query embedden, Fan-out ueber Collections, Rank-Merge, min-Score 0.7, top-K 8; tenant-scoped via RagRetrievalStore.searchForTenant |
| Chunking | packages/ai/src/embeddings/chunking.ts |
splitForEmbedding(), countTokens(), DEFAULT_CHUNK_TOKENS/...OVERLAP_TOKENS |
| Outbox-Publisher (RAG-Index) | packages/ai/src/embeddings/outbox-publisher.ts |
publishRagUpsertEvent/publishRagDeleteEvent, ragEventType() -> rag.index.<entity>.<verb> (mapped auf BullMQ-Queue outbox.rag-index), Idempotenz via dispatchKey |
HiveGrid-Relevanz: Komplette RAG-Grundlage (Knowledge-Base-Suche, KI-Antworten mit Quellen, semantische Suche). Outbox-Pattern erlaubt async Reindex bei jedem Domain-Write. Direkt nutzbar fuer HiveGrid-Module "Wissensdatenbank/RAG" und "KI-Assistent".
1.3 Cost-Tracking, Quota & Guards
| Asset | Pfad | Was |
|---|---|---|
| Cost-Barrel | packages/ai/src/cost/index.ts |
Re-Export: countTokens, countCallCost, executeLLMCall, estimateLLMCost, Pricing-Tabelle (getPricingTable, LATEST_PRICING_VERSION), Spend-Cache, Quota |
| LLM-Wrapper | packages/ai/src/cost/executeLLMCall.ts |
executeLLMCall() — wrappt jeden Provider-Call, leitet Token/Kosten ab, emittiert Outbox-Event ai.cost.event.persist, Quota-Pre-Check + Soft-Degradation + Concurrency-Slot; setLLMCostPublisher/MetricsHook/ErrorHandler; Flag AI_COST_TRACKING_ENABLED=false deaktiviert; capabilities chat/chat-stream/embedding/completion/rerank/tool-call |
| Quota-Guard | packages/ai/src/cost/assertCredit.ts |
assertTenantAICreditAccess(), Concurrency-Counter (Redis-aehnlich), Soft-Degradation-Threshold |
| Spend-Cache | packages/ai/src/cost/spend-cache.ts |
createSpendCache(), Monatsspend-Caching (TTL) |
| Token-Zaehlung | packages/ai/src/cost/tokens.ts |
countTokens/countMessageTokens (gpt-tokenizer + Heuristik-Fallback) |
| Guards | packages/ai/src/guards/index.ts |
assertAICostBudget(), createFixedWindowAIRateLimiter() (pro userId), createAIAuditEvent()/redactAIAuditPayload() (PII-Redaction, prompt/tool_call/mutation), createReviewRequiredResult(), AIGuardError |
HiveGrid-Relevanz: Querschnitt "Billing/Usage-Metering" (KI-Kosten pro Tenant), "Rate-Limiting", "Audit/DSGVO" (Redaction). Persistiert in Prisma AICostEvent (siehe unten). Direkt fuer HiveGrid-Modul "KI-Kontingente/Abrechnung".
1.4 Prompt-/Tool-Registry, Streaming, Eval
| Asset | Pfad | Was |
|---|---|---|
| Prompt-Registry | packages/ai/src/prompts/index.ts |
createAIPromptRegistry() mit Versionierung, render(selection, variables), {{var}}-Templates, AIPromptRegistryError |
| Tool-Registry | packages/ai/src/tools/index.ts |
createAIToolRegistry() (scoped Tools fuer Vercel AI SDK ToolSet), toToolSet(selection), Secret-Redaction in Fehlern |
| Streaming | packages/ai/src/streaming/index.ts |
createAITextStream() ueber streamText (Vercel AI SDK), Guard-Evaluator, Prompt-/Tool-Registry-Integration, getAITextStreamDiagnostics() mit Redaction |
| Promptfoo-Provider + Evals | packages/ai/src/promptfoo-provider.ts, packages/ai/test/evals/, Repo-Root promptfoo.config.yaml |
Eval-Harness (promptfoo-Baseline-Test) |
HiveGrid-Relevanz: "Prompt-Flows / Prompt-Bibliothek" (Versionierung), "KI-Tools/Agenten" (scoped Tool-Registry), Streaming-Chat-UX.
2. @toolctrl/vector-store — Vektor-Store-Adapter (pgvector + Qdrant)
Pfad: /Users/raffaelsheikh/Documents/GitHub/boilerplate/packages/vector-store
Deps: @qdrant/js-client-rest
Subpath-Exports: ., ./factory, ./adapters/pgvector, ./adapters/qdrant, ./wrappers/tenant-scoped
| Asset | Pfad | Was |
|---|---|---|
| Adapter-Interface + Typen | packages/vector-store/src/types.ts |
VectorStoreAdapter (createCollection/upsert/search/delete/health), VectorItem mit Pflicht-payload.tenantId, VectorFilter/FilterClause (Qdrant-Filter-Sprache), VectorMetric = "cosine"|"euclidean"|"dot", VectorDimensionMismatchError |
| Factory (Adapter-Auswahl) | packages/vector-store/src/factory.ts |
createVectorStore() -> waehlt pgvector (Default) oder Qdrant (adapter: oder qdrantUrl); resolveAdapter(); Namespace-Pflicht; gibt tenant-scoped Store zurueck |
| pgvector-Adapter | packages/vector-store/src/adapters/pgvector.ts |
PgvectorAdapter ueber PrismaLike ($queryRawUnsafe/$executeRawUnsafe), erstellt vec_<namespace>_<collection>-Tabellen, HNSW-Index (m, efConstruction), Payload-Filter via payload ->> 'key' |
| Qdrant-Adapter | packages/vector-store/src/adapters/qdrant.ts |
QdrantAdapter ueber QdrantClientLike, Namespace-Praefix |
| Tenant-Scope-Wrapper | packages/vector-store/src/wrappers/tenant-scoped.ts |
withTenantScope() -> TenantScopedVectorStore; searchForTenant() injiziert tenantId-must-Filter zwingend; bare searchVectors() wirft; listVectorIds() scoped nach tenant/sourceType/sourceId |
| Konformanztest | packages/vector-store/src/adapters/conformance.test.ts |
Erzwingt byte-fuer-byte gleiches Verhalten beider Adapter |
Prisma: Kein Prisma-Modell — pgvector legt eigene vec_*-Tabellen via Raw-SQL an. Migration: prisma/migrations/20260517085146_04_enable_pgvector/migration.sql (aktiviert pgvector-Extension).
HiveGrid-Relevanz: Speicher-Backend fuer RAG/semantische Suche. pgvector = "kein Extra-Service" (laeuft auf Haupt-Postgres), Qdrant = Upgrade-Pfad. Tenant-Isolation built-in. Direkt fuer HiveGrid "RAG"/"semantische Suche".
3. @toolctrl/search — Meilisearch Such-Index
Pfad: /Users/raffaelsheikh/Documents/GitHub/boilerplate/packages/search
Deps: meilisearch, zod
Subpath-Exports: ., ./schemas, ./testing
| Asset | Pfad | Was |
|---|---|---|
| Client-Adapter | packages/search/src/client.ts |
createSearchClient() — kapselt Meilisearch-SDK (nie re-exportiert), SearchClient/IndexHandle (ensure/search/upsert/delete/deleteByFilter), meiliFactory-Injection fuer Tests |
| Query-Layer (tenant-scoped) | packages/search/src/query.ts |
searchInIndex, searchWithFallback, bulkIndex, deleteFromIndex, health, TenantScopeError — jede Suche erzwingt tenantId-Filter |
| Index-Helper | packages/search/src/index-helper.ts |
createSearchIndex(), validateIndexSchema(), IndexSchemaValidationError |
| Index-Schemas | packages/search/src/schemas/ (kb-articles.ts, crm-contacts.ts, rows.ts, entities.ts, users.ts, index.ts) |
ALL_SEARCH_INDEXES, BULK_REINDEX_SUPPORTED_INDEXES, getSearchIndexSchema() |
| Outbox-Publisher | packages/search/src/outbox-publisher.ts |
publishSearchUpsertEvent/publishSearchDeleteEvent/publishSearchBulkReindexEvent, searchEventType() |
| Sanitize | packages/search/src/sanitize.ts |
sanitizeDocument() |
HiveGrid-Relevanz: Volltextsuche fuer KB-Artikel, CRM-Kontakte, generische Entity-Rows, User. Outbox-getriebener Reindex. Querschnitt "Suche" parallel zur Vektorsuche (hybrid moeglich).
4. Apps-Engine / Plugin-System (Kern-Fokus)
Wichtige Unterscheidung im Boilerplate:
@toolctrl/plugins= Feature-/App-Manifest-Inventur (Klassifizierung von Features als core/plugin-candidate, Integration-Surfaces) — kein Laufzeit-Sandbox.@toolctrl/plugin-runtime+@toolctrl/plugin-sdk= echtes WASM-Plugin-Laufzeitsystem mit Capability-Modell (das "Apps-Engine"-Adapter-/Sandbox-Pattern).- Prisma
appsEngine.prisma= separate Data-Sync-"App"-Engine (App/DataConnection/AppSyncRun) fuer externe Datenquellen-Integrationen.
4.1 @toolctrl/plugins — Feature-Manifest-Schema + Loader
Pfad: /Users/raffaelsheikh/Documents/GitHub/boilerplate/packages/plugins (keine Deps)
| Asset | Pfad | Was |
|---|---|---|
| Manifest-Schema | packages/plugins/src/manifest-schema.ts |
FeatureManifest (schemaVersion "1.0", classification core/plugin-candidate/remove-candidate/mixed, integration.surfaces admin/tenant/public/api-Routes, integration.workers queues/jobs, externalProviders, permissions, auditEvents, billingImpacts, uat, boundaries, pruning), validateFeatureManifestShape() (handgeschriebener Validator, kein zod) |
| Loader | packages/plugins/src/manifest-loader.ts |
loadFeatureManifests() |
HiveGrid-Relevanz: Methodik-/Governance-Werkzeug — beschreibt, welche Features Plugin-Kandidaten sind und welche Surfaces/Worker/Provider sie brauchen. Nuetzlich als Vorlage fuer HiveGrid-Modul-/Plugin-Klassifizierung, weniger als Laufzeit.
4.2 @toolctrl/plugin-runtime — WASM-Sandbox + Capability-Gateway
Pfad: /Users/raffaelsheikh/Documents/GitHub/boilerplate/packages/plugin-runtime
Deps: @toolctrl/logger, @toolctrl/shared-store, semver, zod
Subpath-Exports: ., ./capabilities, ./hooks/registry, ./manifest
README: packages/plugin-runtime/README.md
| Asset | Pfad | Was |
|---|---|---|
| WASM-Sandbox | packages/plugin-runtime/src/runtime.ts |
createPluginSandbox() — Node-22 WebAssembly, Memory-Cap (Default 64 MB), Fuel-Budget (Default 1.000.000), JSON-Payload-ABI (toolctrl_payload_len/...byte), invoke(hookName, payload), typisierte Failures (compile/instantiate/fuel_exhausted/memory_limit/trap). Upgrade-Pfad: Wasmer/Wasmtime hinter gleichem Interface |
| Capability-Enum | packages/plugin-runtime/src/capabilities.ts |
PLUGIN_CAPABILITY_NAMES (http.outbound, db.read/write, storage.read/write, log, audit, tenant.info, secret.read), Risk-Metadata, PLUGIN_AVAILABLE_HOST_CAPABILITY_NAMES = ["log","audit"] (nur diese in Prod aktiv), productionAvailable-Flags |
| Capability-Gateway | packages/plugin-runtime/src/capability-gateway.ts |
createCapabilityGateway() — deny-by-default; prueft manifest-deklariert + tenant-granted + Scope; CapabilityDeniedError mit Gruenden (not_declared/not_granted/scope_violation/no_host_implementation); laedt Fuel pro Call; onCall-Hook fuer Audit |
| Manifest (Plugin) | packages/plugin-runtime/src/manifest.ts |
PluginManifestSchema (zod): Reverse-DNS-id, SemVer, entry (.wasm), wasmSha256, Ed25519 signature/signedBy, allowedHosts (SSRF-Allowlist, kein localhost/IP/.internal), capabilities mit scope (allowedHosts/keyPattern/tablePrefix), hooks (name->export), optional ui.settings (HTML + SHA256/Integrity), min/maxHostVersion; parsePluginManifest() (safe), manifestDeclaresCapability() |
| Signatur | packages/plugin-runtime/src/signature.ts |
verifyPluginSignature() (Ed25519), parseTrustedPublicKeys(), canonicalPluginSignaturePayload() |
| Capability-Grants/Scope | packages/plugin-runtime/src/capability-grants.ts, .../capability-scope.ts |
PluginCapabilityGrant, isCapabilityGrantScopeSatisfied(), isCapabilityCallWithinScope() |
| Hooks-Registry | packages/plugin-runtime/src/hooks/registry.ts |
zod-validierte Hook-Payloads: tenant.created, user.created, entity.{created,updated,deleted}, workflow.{created,updated,deleted,executed,failed}; registerHook(), validatePluginHookPayload(), Prod-Available-Liste |
| Hook-Emit | packages/plugin-runtime/src/hooks/emit.ts |
emitPluginHook(tx, def, payload, publish) — validiert + publiziert plugin.hook.<name> Outbox-Event in der DB-Transaktion (Fan-out auf Worker-Seite), PluginHookValidationError |
| Audit (Capability-Use) | packages/plugin-runtime/src/audit.ts |
createPluginCapabilityUseRecorder(), hashCapabilityPayload() (kein Roh-Payload), Batch-Sink |
| Audit-Chain (Tamper-Evidence) | packages/plugin-runtime/src/audit-chain.ts |
Per-Tenant Hash-Chain (prevHash/rowHash), stampPluginCapabilityUseChain(), verifyPluginCapabilityUseChain() |
| Crash-Tracker | packages/plugin-runtime/src/crash-tracker.ts |
createCrashTracker() — auto-disable Plugin nach N Crashes |
| Migrations | packages/plugin-runtime/src/migrations.ts |
validatePluginMigration()/applyPluginMigration() — Plugin-eigene SQL mit Keyword-Blocklist + Statement-Limit + tablePrefix-Scoping |
| Version-Check | packages/plugin-runtime/src/version-check.ts |
isPluginCompatibleWithHost() (semver-Range gegen Host) |
| Production-Gate | packages/plugin-runtime/src/production-gate.ts |
assertPluginRuntimeProductionEnabled(), Env PLUGIN_RUNTIME_PRODUCTION_GATE_ENV |
4.3 @toolctrl/plugin-sdk — Autoren-Typen + Settings-Client
Pfad: /Users/raffaelsheikh/Documents/GitHub/boilerplate/packages/plugin-sdk (keine Deps)
README + Beispiele: packages/plugin-sdk/README.md, packages/plugin-sdk/examples/{hello-world,simple-hook-listener,capability-use,async-task}/ (jeweils plugin.toml + handler.ts)
| Asset | Pfad | Was |
|---|---|---|
SDK-Typen + defineHook |
packages/plugin-sdk/src/index.ts |
HookHandler/TypedHookHandler, defineHook(), PluginContext, HostApi (log/audit/tenant/http/storage/secret), PluginCapabilityName/PluginHookName (gespiegelt aus runtime), Hook-Payload-Typen, ABI-Konstanten (PLUGIN_PAYLOAD_ABI_VERSION), Settings-Iframe-Protokoll: createPluginSettingsClient() (postMessage get/set, Timeout, Channel) |
Prisma (Plugins): prisma/schema/plugins.prisma
Plugin(tenantId, pluginId, version, manifest Json, wasmSha256, signaturePublicKey, enabled)TenantPluginCapabilityGrant(per-tenant Consent pro Capability, scopeJson) — Install darf erstenabledwerden, wenn alle deklarierten Capabilities Grants habenPluginUserConfig(per-user opt-in + configJson)PluginInvocation(Audit pro Invoke: outcome/durationMs/fuelUsed)PluginCapabilityUse(per-Capability-Call Audit mitpayloadHash, Hash-ChainprevHash/rowHash)
Queue: Queue-Name toolctrl.plugin-invoke, Job plugin.invoke (siehe Queue-Abschnitt).
HiveGrid-Relevanz (4.2/4.3): Das ist das eigentliche Apps-Engine/Plugin-Adapter-Pattern fuer HiveGrid: sicheres, sandboxed Drittanbieter-/Erweiterungssystem mit Capability-Consent, Signatur-Verifikation, Hook-Lifecycle, Audit-Chain und Crash-Schutz. Direkt relevant fuer HiveGrid-Module "Apps/Marketplace", "Integrationen", "Erweiterbarkeit". Achtung: in Prod sind nur log/audit-Capabilities freigeschaltet — http/db/storage/secret sind Foundation-Stubs (productionAvailable: false).
4.4 Prisma Apps-Engine (Data-Sync) — appsEngine.prisma
Pfad: prisma/schema/appsEngine.prisma
| Modell | Was |
|---|---|
App |
Installierte App pro Tenant (type-Identifier z.B. "sample-data-sync", config JSON, isActive) |
DataConnection |
Verbindung zu externer Datenquelle (type "rest-api"/"oauth2"/"webhook", verschluesselte Credentials in config) |
AppSyncRun |
Sync-Lauf (status pending/running/completed/partial/failed/..., executionMode manual/scheduled/webhook, items processed/created/updated/failed) |
AppSyncRunLog |
Log-Eintraege pro Lauf |
AppSyncItem |
Einzel-Item-Mapping (sourceId/destinationId, action, rollback-Daten) |
HiveGrid-Relevanz: Separate "Apps-Engine" fuer externe Datensynchronisation (ETL-aehnlich, idempotent, mit Rollback + Audit). Relevant fuer HiveGrid-Module "Integrationen/Datenimport" und externe Connector-Apps — komplementaer zum WASM-Plugin-System (4.2).
5. @toolctrl/storage — S3/Scaleway-Storage, Signed-URLs, Virus-Scan
Pfad: /Users/raffaelsheikh/Documents/GitHub/boilerplate/packages/storage
Deps: @aws-sdk/client-s3, clamscan, file-type
| Asset | Pfad | Was |
|---|---|---|
| S3-/Local-Adapter | packages/storage/src/index.ts |
createS3StorageAdapter() (endpoint/region/bucket/credentials, forcePathStyle, publicBaseUrl) ueber @aws-sdk/client-s3 — Scaleway-/jeder-S3-kompatibel; createLocalStorageAdapter() (Dev, Pfad-Traversal-Schutz); StorageAdapter (putObject/deleteObject), StorageAdapterError mit typisierten Codes + Secret-Redaction |
| Signed-URLs | packages/storage/src/signed/index.ts |
createSignedUploadUrl/createSignedDownloadUrl (PUT/GET, expiry, contentType, checksumSha256) |
| Virus-Scan-Service | packages/storage/src/scan/index.ts |
createStorageScanService() (scan -> bei infected quarantaene), ScanVerdict clean/infected/error |
| ClamAV-Scanner | packages/storage/src/scan/clamav-scanner.ts |
createClamAVStorageScanner() ueber clamscan, ClamAVUnreachableError |
| Magic-Bytes-Adapter | packages/storage/src/scan/magic-bytes-adapter.ts |
createMagicBytesScanAdapter() ueber file-type (Content-Type-Verifikation) |
| Allow-List | packages/storage/src/scan/allow-list.ts |
DEFAULT_ALLOW_LIST, resolveAllowList() |
| S3-Quarantaene | packages/storage/src/scan/s3-quarantine.ts |
createS3StorageQuarantineAdapter() |
| Access-Control | packages/storage/src/access/index.ts |
Storage-Zugriffspruefung |
HiveGrid-Relevanz: Datei-Uploads/Anhaenge (KB, CRM, Apps), DSGVO-EU-Storage via Scaleway-S3. Virus-Scan + magic-bytes-Pruefung = Sicherheits-Querschnitt fuer jedes Upload-Modul. Signed-URLs fuer direkten Client-Upload/Download.
6. @toolctrl/realtime — SSE/Realtime-Events
Pfad: /Users/raffaelsheikh/Documents/GitHub/boilerplate/packages/realtime (keine Deps)
README: packages/realtime/README.md
| Asset | Pfad | Was |
|---|---|---|
| Event-Typen | packages/realtime/src/events.ts |
REALTIME_EVENT_NAMES (notification.snapshot/update, keepalive, error), RealtimeEvent-Union, validateRealtimeEvent(), Notification-Stream-Item-Typen |
| Server (SSE-Serialisierung) | packages/realtime/src/server.ts |
serializeRealtimeEvent() (SSE id/event/data, retry), serializeRealtimeComment(), Newline-Sanitizing |
| Client | packages/realtime/src/client.ts |
Client-seitige Event-Verarbeitung |
| Guards | packages/realtime/src/guards.ts |
Type-Guards fuer Events |
HiveGrid-Relevanz: Live-Benachrichtigungen (In-App-Notifications) via SSE — kein WebSocket-Server, sondern SSE-Stream. Querschnitt "Realtime/Notifications". Hinweis: WebSocket im engeren Sinn ist hier nicht vorhanden; Realtime = SSE.
7. E-Mail-Stack
7.1 @toolctrl/mail — Provider + Templates
Pfad: /Users/raffaelsheikh/Documents/GitHub/boilerplate/packages/mail
Deps: @react-email/components, @react-email/render, @toolctrl/brand, @toolctrl/queue, react, react-dom
README: packages/mail/README.md; Preview-Server: packages/mail/scripts/preview-server.ts
| Asset | Pfad | Was |
|---|---|---|
| MailerSend-Provider | packages/mail/src/mailersend.ts |
createMailerSendProvider()/...FromEnv(), API https://api.mailersend.com, Timeout, MailMetricsHook, Templates/Personalization/Attachments/Tracking |
| Queued-Mail-Provider | packages/mail/src/queue.ts |
createQueuedMailProvider() — Versand via @toolctrl/queue (async) statt direkt |
| Mail-Typen | packages/mail/src/types.ts |
MailProvider, SendMailInput, MailAddress, MailAttachment, MailProviderError |
| React-Email Render | packages/mail/src/templates-react/render.ts |
renderTransactionalEmail(), reactTransactionalTemplateNames, Brand-Helfer |
| Transaktionstemplates | packages/mail/src/templates-react/transactional/ |
Welcome, MagicLink, PasswordReset, Invitation, InvoiceSent, InvoiceTenant, SubscriptionSuccess/Failed, TenantNotification |
| Layout-Primitives | packages/mail/src/templates-react/_layout/ |
BaseLayout, Header, Footer, Button (assertSafeUrl/InvalidActionUrlError), InfoBox, Eyebrow, theme |
7.2 @toolctrl/email-templates — Sequenz-Templates
Pfad: /Users/raffaelsheikh/Documents/GitHub/boilerplate/packages/email-templates
Subpath-Exports: ./sequences/{welcome,trial-reminder,re-activation,generic-step}
| Asset | Pfad | Was |
|---|---|---|
| Render + Theme | packages/email-templates/src/render.ts, .../theme.ts |
React-Email Render-Layer |
| Sequenz-Templates | packages/email-templates/src/sequences/ |
welcome, trial-reminder, re-activation, generic-step |
7.3 @toolctrl/email-sequences — Sequenz-Engine (reine Logik)
Pfad: /Users/raffaelsheikh/Documents/GitHub/boilerplate/packages/email-sequences (keine Deps)
Subpath-Exports: ., ./types, ./engine, ./step-runner, ./conditions, ./unsubscribe-token
| Asset | Pfad | Was |
|---|---|---|
| Engine | packages/email-sequences/src/engine.ts |
startSequenceRun()/advanceSequenceRun() — Store + Queue injiziert (kein Prisma/BullMQ direkt); SequenceRunStatus, idempotent ueber (sequenceId, triggerEventId) |
| Step-Runner | packages/email-sequences/src/step-runner.ts |
runStep() — rendert + sendet einen Schritt |
| Condition-DSL | packages/email-sequences/src/conditions.ts |
evaluateCondition()/evaluateAllConditions(), resolvePath() |
| Unsubscribe-Token | packages/email-sequences/src/unsubscribe-token.ts |
signUnsubscribeToken()/verifyUnsubscribeToken()/buildUnsubscribeUrl(), Secret-Pflicht |
Prisma: prisma/schema/emailSequences.prisma — EmailSequence, EmailSequenceRun, EmailSequenceStepDelivery. prisma/schema/email.prisma (App-Email-Config).
HiveGrid-Relevanz (7.x): Transaktions-Mails (Auth, Billing, Notifications) + Marketing-/Lifecycle-Sequenzen (Onboarding, Trial-Reminder, Reaktivierung). EU-Provider MailerSend. Querschnitt "Kommunikation/E-Mail" fuer praktisch jedes HiveGrid-Modul; Sequenz-Engine direkt fuer "Marketing/Funnel/Lifecycle".
8. @toolctrl/marketing-outbound — Outbound-Webhook-Adapter
Pfad: /Users/raffaelsheikh/Documents/GitHub/boilerplate/packages/marketing-outbound (keine Deps)
Subpath-Exports: ./adapters/{slack,discord,mailchimp,hubspot,brevo,custom-webhook}
| Asset | Pfad | Was |
|---|---|---|
| Adapter-Interface + Registry | packages/marketing-outbound/src/index.ts |
adapterRegistry (slack/discord/mailchimp/hubspot/brevo/webhook), resolveAdapter(type), ADAPTER_TYPES |
| Shared-Typen + SSRF-Guard | packages/marketing-outbound/src/types.ts |
MarketingOutboundAdapter<TConfig> (parseConfig/send), MarketingLeadPayload/MarketingLeadEventType (captured/qualified/converted...), adapterFetch() (Timeout + SSRF-Allowlist, nur HTTPS, kein RFC1918/metadata), maskEmailIfRequested() (PII), MarketingAdapterConfigError |
| Konkrete Adapter | packages/marketing-outbound/src/adapters/*.ts |
je buildXPayload() + Adapter-Objekt (Slack-Blocks, Discord-Embed, Mailchimp/HubSpot/Brevo-Payloads, Custom-Webhook) |
Prisma: prisma/schema/marketingLeads.prisma — MarketingIntegration (type, configJson verschluesselt, triggerEvents String[], enabled, lastDeliveredAt/lastError), Lead, LeadEvent, TenantAcquisition.
HiveGrid-Relevanz: Lead-/Marketing-Integrationen mit Drittsystemen (CRM, Chat, ESP). Worker dispatcht Outbox-Events nach MarketingIntegration.type. SSRF-/PII-Schutz im Adapter. Direkt fuer HiveGrid-Module "Marketing/Funnel" und "Integrationen (Outbound-Webhooks)".
9. @toolctrl/queue — BullMQ + Idempotenz + Prioritaeten
Pfad: /Users/raffaelsheikh/Documents/GitHub/boilerplate/packages/queue
Deps: bullmq
Subpath-Exports: ., ./runtime (CJS runtime.cjs)
README: packages/queue/README.md
| Asset | Pfad | Was |
|---|---|---|
| Queue-Namen | packages/queue/src/names.ts |
queueNames: toolctrl.email, .notifications, .maintenance, .ai-chatbot, .browser-automation, .file-scan, .exports, .imports, .plugin-invoke; isQueueName() |
| Job-Payloads (typisiert) | packages/queue/src/payloads.ts |
AiJob (ai.chat-completion/ai.embedding/ai.browser-task), PluginInvokeJob (plugin.invoke), TransactionalMailJob, WebhookDeliveryJob, NotificationJob, CleanupJob, FileScanJob, ExportJob/ImportJob, Workflow-Outbox-Events; QueueJobPayloads-Map |
| Producer + Idempotenz | packages/queue/src/producers.ts |
enqueueJob(), build*Job()-Builder, buildIdempotentJobId() (jobId/idempotencyKey/export-/importJobId/metadata.idempotencyKey), buildRetryJobOptions(), buildIdempotentJobId, setEnqueueObserver() |
| Prioritaeten | packages/queue/src/priority.ts |
JOB_PRIORITY_{CRITICAL,HIGH,NORMAL,LOW}, resolveJobPriorityForTenant() (Plan-basiert), priorityFromPlan/Override, setTenantPlanLookup() |
| Redis-Connection | packages/queue/src/connection.ts |
parseRedisUrl(), resolveRedisConnectionOptions(), ...FromProcessEnv() |
| Queue-Gruppen | packages/queue/src/groups.ts |
WORKER_QUEUE_GROUPS, getGroupForQueue(), getQueueNamesForGroup() |
| Runtime (BullMQ-Factory) | packages/queue/src/runtime.ts / runtime.cjs |
createQueue/createWorker/createQueueEvents (typisiert) |
Prisma: prisma/schema/queue.prisma — QueueFailedJob (DLQ-Spiegel), ExportJob + ExportJobStatus-Enum. Outbox-Pattern: prisma/schema/outbox.prisma.
HiveGrid-Relevanz: Zentrale asynchrone Verarbeitung fuer KI-Jobs, Plugin-Invokes, Mail, Notifications, File-Scan, Exporte/Importe. Idempotenz ueber jobId/idempotencyKey + plan-basierte Prioritaeten = Querschnitt fuer alle HiveGrid-Worker-Module. AI/Plugin/Mail-Queues binden die obigen Pakete an den Worker an.
Querschnitts-Konventionen (faktisch beobachtet)
- Tenant-Isolation ist ueberall am Seam verankert:
payload.tenantId-Pflicht (vector-store),searchForTenant/tenantId-Filter (search, vector-store),tenantIdPflichtargument (embeddings), per-tenant Grants/Audit-Chain (plugin-runtime). - Outbox-Pattern fuer asynchrone Seiteneffekte: RAG-Reindex (
rag.index.*), Such-Reindex, Plugin-Hooks (plugin.hook.*), AI-Cost (ai.cost.event.persist), Marketing-Dispatch — alle in DB-Transaktion committet, Worker fan-out. - Kein direkter
process.env-Zugriff in den Paketen (Env-Snapshots vom Boundary) — Env-Access-Enforcement-Tests. - Secret-/PII-Redaction durchgaengig in Fehlern/Audit (ai/guards, ai/tools, ai/streaming, storage, marketing-outbound).
- Adapter-/Factory-Pattern als roter Faden:
getAIProviderAdapter,createEmbeddingProvider+Registry,createVectorStore+resolveAdapter,createSearchClient+meiliFactory,adapterRegistry(marketing),createS3StorageAdapter/createLocalStorageAdapter, Capability-hostFns-Map (plugin-runtime). - Tests: jedes Paket hat
*.test.tsneben den Quellen; Eval-Harness inpackages/ai/test/evals+ Repo-Rootpromptfoo.config.yaml; CI/verify ueberpnpmWorkspace-Scripts (test/typecheck/build), Repo-Rootpackage.json+lefthook.yml.
Wichtige Abgrenzungen / Luecken (faktisch)
- WebSocket: Es gibt kein WebSocket-Paket; Realtime laeuft ueber SSE (
@toolctrl/realtime). - Plugin-Capabilities in Prod: Nur
log+auditsind hostseitig verfuegbar (PLUGIN_AVAILABLE_HOST_CAPABILITY_NAMES); http/db/storage/secret/tenant.info sind deklarierbar, aberproductionAvailable: false(Foundation). - WASM-Fuel: Keine echte Per-Instruktions-Meterung (V8-Limit) — Fuel wird pro Capability-Call/Invoke geladen; Wall-Clock-Timeout liegt beim Aufrufer.
- Anthropic-Embeddings: Bewusster Stub/Platzhalter (
packages/ai/src/embeddings/providers/anthropic.ts). - Zwei "Apps-Engine"-Bedeutungen: WASM-Plugin-System (
plugin-runtime/plugin-sdk) vs. Data-Sync-Apps (prisma/schema/appsEngine.prisma,App/DataConnection/AppSyncRun) — beim Modul-Abgleich nicht verwechseln.