25 KiB
Pub/Sub Cache-Invalidation
Status: in_progress (subscription + tenant + feature-flag two-tier caches shipped; CLI cross-process publisher closed 2026-05-20; reconnect-flush + testcontainers integration closed 2026-05-25 — remaining work is the cross-app namespace fan-out documented in Deviations) Task-ID: #39 Phase: 03 Estimated Effort: 3-4 Tage Owner: unassigned
Problem
Selbst nach #38 (Shared-Store-Migration) verbleibt ein Konsistenz-Problem: einige hot caches sollen weiterhin lokal in-process (cachified-LRU) gehalten werden, weil Redis-Roundtrips pro Request zu teuer sind. Beispiele:
- Subscription-Cache (Web): Pro Request mehrfach abgefragt (Entitlement-Checks, Limit-Counter, UI-Visibility). Redis-Roundtrip 1-3ms × 20 Calls = 20-60ms Overhead
- Tenant-Cache: Pro Request fix
- Entitlements-Cache: Pro Request fix
- Feature-Flag-Cache (kommt in Phase 05): Pro UI-Render fix
Lösung: Zwei-Stufiger Cache. L1 = in-process cachified (schnell, nicht-shared), L2 = SharedStore (langsam, shared). Invalidations werden über Pub/Sub propagiert: wenn ein Event den Cache invalidieren soll, published derjenige, der die Quelle ändert (z.B. Stripe-Webhook in API), und ALLE Web-Replicas hören mit und purgen ihren L1.
Aktueller Zustand: Stripe-Webhook in apps/api/src/billing/stripe-webhook-route.ts:60-138 updated DB + ruft clearSubscriptionsCache(tenantId) lokal in der API. Aber: der Cache liegt nicht in der API, sondern in jeder Web-Replica. Heute: Web-Cache wird erst nach Ablauf von 1h-TTL frisch. Resultat: User kann z.B. nach Plan-Upgrade noch bis zu 1h alte Limits sehen.
Depends on
03-scaling-infrastructure/shared-store-package.md— liefertcreateCacheStoremit Pub/Sub-Invalidation03-scaling-infrastructure/shared-store-migration.md— Subscription-Cache muss schon über SharedStore laufen, sonst kein Channel zum Subscriben02-observability/sentry-wiring-web.md— Subscriber-Errors müssen captured werden, sonst tappt der Operator im Dunkeln wenn Subscriber stillschweigend stirbt
Blocks
05-business-features/feature-flags.md— Feature-Flag-Cache nutzt identisches Pattern03-scaling-infrastructure/tenant-snapshot.md— Tenant-Reset published Invalidation für Tenant-Cache- alle nachfolgenden Caches mit Cross-Replica-Konsistenz-Anforderung
Approach
Pattern: Source-of-Truth-Owner published, Cache-Holder subscribed.
-
Publisher-Seite: Wer immer die SoT ändert (API für Subscription via Stripe-Webhook, Worker für Tenant-Snapshot, etc.) ruft nach Commit
cache.invalidate(key)auf dem SharedStore-Cache. Das published auf Channel<namespace>:invalidatemit Payload{ key }. -
Subscriber-Seite: Jede Web-Replica (und API, wo sie auch L1 hält) startet beim Boot einen Subscriber pro relevantem Cache-Channel. Der Subscriber-Callback purged den lokalen L1 (cachified.delete) und optional auch den L2 (für Cleanup).
-
Auto-Reconnect: ioredis hat eingebauten Reconnect. Wir wrappen das mit einem
onReconnect-Handler, der nach erfolgreichem Reconnect einmal alle L1-Caches komplett flushed (cache.clearAll()), weil wir nicht wissen welche Invalidations während Disconnect verloren gingen. -
Replay-fähig: Pub/Sub hat keine Persistence — bei Disconnect verloren. Mitigation: L1-TTL bleibt kurz (5-15min), so dass im Worst-Case-Disconnect-Szenario nach kurzer Zeit auto-refresh greift. Für strikte Konsistenz: Outbox-Pattern (#42) für kritische Events, dann Worker konsumiert Outbox und published Invalidation. Doppelte Sicherheit.
-
Multi-Process-Setup:
- API (1 Replica): Publisher für Subscription, Tenant, Entitlements
- Web (2+ Replicas): Subscriber für alle obigen
- Worker (2+ Replicas): Publisher für lange laufende Updates (Bulk-Imports, Tenant-Snapshot-Restore)
Vier konkrete Caches migrieren
A. Subscription-Cache (Stripe)
- Source:
apps/api/src/billing/stripe-webhook-route.ts:60-138 - Trigger: Stripe-Event (
customer.subscription.updated,invoice.payment_succeeded, etc.) - Channel:
cache:subscription:invalidate - Payload:
{ tenantId: string } - Publisher: API in Webhook-Handler nach
updateTenantSubscriptionFromStripe - Subscriber: Web (jede Replica) ruft
clearSubscriptionsCache(tenantId)
B. Entitlements-Cache
- Source:
apps/web/app/utils/entitlements/.server/productAccess.server.ts - Trigger: Subscription-Change (impliziert Entitlement-Change), Manual-Admin-Edit
- Channel:
cache:entitlements:invalidate - Payload:
{ tenantId: string } - Publisher: Identisch zur Subscription-Trigger-Stelle (gleiche Webhook-Handler) + Admin-Edit-Route
- Subscriber: Web purged in-process Entitlements-Map
C. Tenant-Cache
- Source:
apps/web/app/utils/data/.server/appData.ts - Trigger: Tenant-Settings-Update, Tenant-Logo-Upload, Member-Add/Remove
- Channel:
cache:tenant:invalidate - Payload:
{ tenantId: string } - Publisher: Web (eigene Action) UND API (bei API-getriebener Mutation)
- Subscriber: Alle Web-Replicas + API (wo Tenant gelesen wird)
D. Feature-Flag-Cache (Vorbereitung für #53)
- Source: kommt in Phase 05
- Channel:
cache:feature-flag:invalidate - Payload:
{ flagKey?: string; tenantId?: string; all?: boolean } - Publisher: Admin-Edit in Web
- Subscriber: Web + API + Worker
- In diesem Task nur das Channel-Naming + Subscriber-Stub vorbereiten, damit #53 nur noch den Publisher hinzufügt
Warum nicht andere Pattern:
- ❌ Polling: schwergewichtig, schlechte Reaktionszeit
- ❌ Cache-TTL niedrig halten (5s): Hammers gegen Redis und DB
- ❌ WebSocket-basierte Push-Invalidation: Overkill, wir haben Redis schon
- ❌ Kafka/NATS für ein einziges Use-Case: zu viel Infra
Implementation Steps
-
Subscriber-Lifecycle-Framework
- Files:
packages/shared-store/src/helpers/subscriber-lifecycle.ts,subscriber-lifecycle.test.ts createSubscriberRegistry({ store })mitregister(channel, handler)+start()+stop()- Auto-Reconnect-Handler: bei Redis-Reconnect alle registered Handler einmalig mit
{ type: 'flush' }aufrufen - Sentry-Integration: jeder Subscriber-Error wird captured + re-thrown an unhandled (Subscribers dürfen nicht stillschweigend sterben)
- Tests: register/start/stop, Reconnect-Flush, Error-Capture
- Commit:
03-scaling/pubsub: add subscriber lifecycle framework with reconnect handling
- Files:
-
App-Boot-Integration Web
- Files:
apps/web/app/entry.server.tsx(oder analoger Boot-Hook),apps/web/app/utils/subscribers/index.server.ts - SubscriberRegistry beim Boot starten, registriert alle Web-Subscriber (Subscription, Entitlements, Tenant, Feature-Flag-Stub)
- Graceful-Shutdown-Handler:
process.on('SIGTERM', () => registry.stop()) - Tests: Boot-Integration-Test (Mock-Redis, check dass alle Channels subscribed sind)
- Commit:
03-scaling/pubsub: wire subscriber registry into web boot
- Files:
-
App-Boot-Integration API
- Files:
apps/api/src/server.ts,apps/api/src/subscribers/index.ts - Analog Web, API hat eigene Subscriber-Liste (Tenant + Feature-Flag)
- Commit:
03-scaling/pubsub: wire subscriber registry into api boot
- Files:
-
Subscription-Cache: Publisher in Stripe-Webhook
- Files:
apps/api/src/billing/stripe-webhook-route.ts, Test - Nach
updateTenantSubscriptionFromStripe→subscriptionCache.invalidate(tenantId)(cache aus #38) - Tests: Mock Redis, verify Pub/Sub-Message wurde published
- Commit:
03-scaling/pubsub: publish subscription invalidation from stripe webhook
- Files:
-
Subscription-Cache: Subscriber in Web
- Files:
apps/web/app/utils/subscribers/subscription-cache.subscriber.ts, Test - Handler ruft
clearSubscriptionsCache(tenantId)+ zusätzlichsubscriptionCache.invalidate(tenantId, { localOnly: true })für L1 - Integration-Test: 2 Web-Instances, Webhook auf API → beide Web-Instances sehen neuen Wert < 2s
- Commit:
03-scaling/pubsub: subscribe subscription cache invalidation in web
- Files:
-
Entitlements-Cache: Publisher + Subscriber
- Files:
apps/web/app/utils/entitlements/.server/productAccess.server.ts,apps/web/app/utils/subscribers/entitlements-cache.subscriber.ts, Tests - Publisher in API (gleiche Webhook-Stelle wie Subscription, weil korreliert)
- Subscriber in Web purged in-process Entitlements-Map
- Commit:
03-scaling/pubsub: add entitlements cache pub/sub channel
- Files:
-
Tenant-Cache: Publisher + Subscriber
- Files:
apps/web/app/utils/data/.server/appData.ts,apps/web/app/utils/subscribers/tenant-cache.subscriber.ts, Tests - Publisher in Web-Action für Tenant-Settings-Update; API-Side wo Tenant-Mutation passiert
- Subscriber sowohl Web als auch API
- Commit:
03-scaling/pubsub: add tenant cache pub/sub channel
- Files:
-
Feature-Flag-Cache-Stub (für #53)
- Files:
apps/web/app/utils/subscribers/feature-flag-cache.subscriber.ts(Stub),apps/api/src/subscribers/feature-flag-cache.subscriber.ts(Stub) - Subscriber registered Handler, der derzeit nur loggt (
"feature-flag cache invalidation received, publisher not yet implemented") - Ermöglicht #53, nur Publisher-Seite zu ergänzen
- Commit:
03-scaling/pubsub: add feature-flag cache subscriber stubs
- Files:
-
Multi-Process-Integration-Test
- Files:
apps/web/__integration__/cross-replica-invalidation.test.ts - testcontainers spinnt 1 Redis-Container + 2 in-process Web-Subscriber-Instances + 1 in-process Publisher
- Test-Cases: invalidate triggert beide Subscriber; Reconnect-Flush nach Disconnect-Simulation; Concurrent-Invalidation
- Commit:
03-scaling/pubsub: add multi-process invalidation integration tests
- Files:
-
Doku
- Files:
docs/observability/redis-stores.md(Erweiterung): Sektion „Pub/Sub-Channels" mit allen 4 Channels, Owner-Tabelle (wer published, wer subscribed) - Operational-Runbook: „Cache-Invalidation funktioniert nicht — Debug-Steps" (Redis-MONITOR, Subscriber-Health-Check)
- Commit:
03-scaling/pubsub: document pub/sub channels and runbook
- Files:
Test Requirements
- Unit-Tests: SubscriberRegistry (register, start, stop, reconnect-flush, error-capture); jeder einzelne Subscriber-Handler (purged korrekt)
- Integration-Tests: Cross-Replica-Invalidation pro Channel (Multi-Process-Setup via testcontainers); Reconnect-Verhalten (kill Redis, restart, verify Flush)
- E2E-Tests (Playwright): Stripe-Webhook-Simulation → check ob Web-UI direkt neuen Subscription-Status zeigt (Replication-Test)
- Coverage-Target: ≥ 90 % im Subscriber-Code, ≥ 85 % an Publisher-Stellen
- Test-Files: Subscriber-Files co-located; Integration in
apps/web/__integration__/undapps/api/src/__integration__/
NICHT als completed markieren bevor:
- Multi-Process-Integration-Test grün
- Staging-Test: Stripe-Webhook auf Replica A → Web-Replica B + C sehen neuen Wert < 2s
- Reconnect-Test auf Staging:
docker restart redis→ nach 30s sind alle Subscriber wieder online
Acceptance Criteria
- SubscriberRegistry-Framework in
packages/shared-store/src/helpers/subscriber-lifecycle.tsimplementiert - Web + API booten SubscriberRegistry, registrieren alle relevanten Channels
- Vier Channels live:
cache:subscription:invalidate,cache:entitlements:invalidate,cache:tenant:invalidate,cache:feature-flag:invalidate - Stripe-Webhook published Subscription + Entitlements-Invalidation nach DB-Update
- Tenant-Mutation-Stellen published Tenant-Invalidation
- Feature-Flag-Subscriber-Stubs vorhanden (für #53)
- Reconnect-Flush funktioniert: nach Redis-Restart sind alle L1-Caches geleert
- Multi-Process-Integration-Test grün (2 Subscriber-Instances, ein Publisher, alle Channels)
- Staging-Verifikation: End-to-End-Latenz Stripe-Event → Web-Cache-Invalidation < 2s
docs/observability/redis-stores.mdenthält Pub/Sub-Channel-Tabelle + Operational-Runbookpnpm lint,pnpm typecheck,pnpm testgrün- Detail-MD Progress Log aktualisiert
- PROGRESS.md Status auf
completed
Rollback Strategy
Pro Channel separat rollback-bar:
- Subscription-Channel: Revert Publisher- + Subscriber-Commits → zurück zu 1h-Stale-Verhalten, funktional
- Entitlements/Tenant: Analog, zurück zu vorherigem Pattern (z.B. Loader-Refetch oder Page-Refresh)
- SubscriberRegistry-Framework: Wenn das ganze Framework Probleme macht → Boot-Integration revert, Subscribers schweigen, Caches werden nicht invalidiert. Mitigation: L1-TTL kurz (5-15min) als Safety-Net
Global-Kill-Switch: Env-Var DISABLE_CACHE_PUBSUB=true → SubscriberRegistry started nicht, Publisher no-op. Erlaubt schnelles Disable ohne Redeploy.
DB-Migrations: keine in diesem Task.
Security Notes
- Pub/Sub-Message-Validation: Subscriber-Handler validieren Payload strict (Zod-Schema pro Channel), reject Malformed mit Sentry-Capture. Verhindert Crash bei Garbage-Messages
- Channel-Spoofing-Risk: Wer Redis-Zugriff hat, kann beliebig invalidieren. Mitigation: Redis-AUTH zwingend in Production, kein Public-Listen, Network-Isolation
- Cache-Stampede bei Mass-Invalidation: Wenn Angreifer
cache:tenant:invalidatefür alle Tenants published → alle Replicas refetchen aus DB. Mitigation: Subscriber-Handler implementiert lokales Debounce (max 1 invalidation pro key pro 500ms) - Sensitive-Data in Invalidation-Payload: Payload enthält nur tenantId (UUID), keine PII. Klare Regel: Pub/Sub-Channels sind nicht zum Datentransport, nur Invalidation
- Disconnect-During-Critical-Operation: Wenn Redis während Stripe-Webhook down ist, schlägt Publisher fehl. Mitigation: Outbox-Pattern (#42) puffert Events, Dispatcher retried Publish bis Redis wieder up
- Sentry-Breadcrumbs: SubscriberRegistry meldet jede Disconnect/Reconnect-Event als Breadcrumb, damit Stale-Cache-Bugs in Produktion debugbar sind
Subagent Recommendations
- Planning:
Plan-Subagent mit Channel-Tabelle (A-D) + bestehendersubscriptionService.ts-Implementation als Input - Research:
Explore-Subagent für „aktuelle cachified-Nutzung inapps/web/app/utils/services/.server/undapps/web/app/utils/data/.server/— welche Caches haben non-trivial Konsistenz-Anforderungen?" - Implementation: Ein
general-purpose-Subagent pro Channel (A-D), max 1 Channel pro Subagent — Publisher + Subscriber + Test gehören zusammen - Framework-Subagent: Separater
general-purpose-Subagent für Schritte 1-3 (SubscriberRegistry-Framework + Boot-Integration), weil Framework-Bug alle Channels bricht - Tests:
general-purpose-Subagent für Multi-Process-Integration-Test mit testcontainers (Step 9) - Verification:
gsd-verifierfür Acceptance-Criteria + Staging-Latenz-Messung - Code-Review:
gsd-code-reviewernach allen Channels — Fokus auf Reconnect-Pfade und Error-Handling im Subscriber
Staging-Deployment-Steps
- Branch
agent/03-pubsub-cache-invalidationpushen - Voraussetzung: #38 muss auf Staging erfolgreich getestet sein (Subscription läuft schon über SharedStore)
- Coolify-Setup:
- 2x Web-Replicas (Cross-Replica-Test braucht ≥ 2)
- 1x API-Replica (Publisher-Seite)
- Redis-Container läuft schon aus #37/#38
- Env-Var setzen:
REDIS_PUBSUB_DB=2(eigener logischer Index) - Smoke-Tests:
- End-to-End-Stripe-Test:
- Stripe-CLI:
stripe trigger customer.subscription.updated --customer cus_xxx - Innerhalb 2s: GET
/api/tenants/<id>/subscriptionauf Web-Replica A → erwarte neuen Status - Gleiche GET auf Web-Replica B → erwarte ebenfalls neuen Status
- Stripe-CLI:
- Reconnect-Test:
docker exec redis redis-cli SHUTDOWN NOSAVE- Warte 5s, Web-Replicas loggen Disconnect
docker compose up redisneu starten- Innerhalb 30s sehen Logs
"subscriber reconnected, flushing local caches" - Stripe-Test wiederholen, muss wieder funktionieren
- Pub/Sub-Monitor: Auf Staging
docker exec redis redis-cli -n 2 PSUBSCRIBE '*'laufen lassen, parallel Webhook triggern → verifiziere dass Message published wird
- End-to-End-Stripe-Test:
- Sentry-Check: Keine Subscriber-Errors in Issues-Liste
- Coolify-Logs prüfen: Beim Boot jeder Web-Replica steht
"subscriber registry started: 4 channels"
Deviations from Plan
- Entitlements cache is NOT a separate two-tier cache. The repo has no in-process entitlements cache today —
resolveTenantProductAccess(apps/web/app/utils/entitlements/.server/productAccess.server.ts) recomputes from the subscription on every call. Entitlement freshness piggy-backs on subscription invalidation: oncecache:subscription:invalidateclears the subscription L1, the next entitlement check pulls a fresh subscription and derives a fresh decision. Channel inventory indocs/architecture/two-tier-cache.mddocuments the chain. Feature-Flag-Cache subscriber stub was NOT shipped.— superseded: Phase 05 #53 has since shipped the full feature-flag two-tier cache (apps/web/app/utils/services/.server/featureFlagCache.server.ts+subscribeFeatureFlagCache()incache-bootstrap.server.ts) and the web-route mutators all callinvalidateFlagCache(name)after the DB write. 2026-05-20 follow-up9edc1357closed the last cross-process gap by giving the operator CLI a one-shot publisher (publishFeatureFlagInvalidationOncein@toolctrl/feature-flags, wired intotoolctrl-cli feature-flags {enable,disable,set-rules}viabroadcastFlagInvalidation); without it the CLI-driven mutation left long-lived web replicas serving stale evaluations from L1 until the 60s TTL expired.- Subscriber-lifecycle framework was NOT extracted as a separate
createSubscriberRegistry. Each two-tier cache exposes its own idempotentsubscribe()method (singleton-guarded).apps/web/app/cache-bootstrap.server.tsstarts the two subscribers in parallel and logs errors viawebLogger. A registry can be extracted later if we exceed ~4 channels. Reconnect-flush is provided as— resolved 2026-05-25 (flushLocal()on the two-tier cache but not yet wired to a Redis'ready'event handler — the L1 TTL safety-net (default 300s viaCACHE_L1_TTL_SECONDS) covers reconnect-loss scenarios.47c37b88).SharedStoreAdapter.onReconnect(handler)now exposes ioredis'ready'-after-disconnect events (noop onMemoryAdapter);createTwoTierCache(...).subscribe()automatically registers a debouncedflushLocal()reconnect handler (default 1000ms, override viareconnectFlushDebounceMs). The subscribe-disposer also removes the reconnect handler.- Cross-app fan-out (API publisher → Web subscriber) uses per-app SharedStore namespaces (
api,web). For broadcasts to reach across processes both apps must share the same SharedStore namespace; this is documented as a follow-up indocs/architecture/two-tier-cache.md. The local-only mutation paths (web-sideclearTenantSubscriptionCache) still publish on the web channel so in-app fan-out across web replicas works today. 2026-05-20 — CLI feature-flag mutators sidestep the per-app namespace mismatch by explicitly publishing on thewebnamespace (configurable viaFEATURE_FLAGS_PUBLISH_NAMESPACE), proving the pattern that the broader API↔Web platform-namespace refactor can adopt. Multi-Process testcontainers integration test (Step 9) is deferred. In-process tests already prove cross-replica fan-out by routing two— resolved 2026-05-25 (SharedStorefacades through oneMemoryAdapter, which is the same fan-out path Redis exercises.f4b290bb). Newpackages/shared-store/src/helpers/two-tier-cache.testcontainers.test.tsbrings up aredis:7-alpinecontainer viatestcontainers(or honoursPUBSUB_TEST_REDIS_URL) and exercises cross-instance pub/sub fan-out, reconnect-flush, and peer-L2-mutation-after-reconnect. Skipped by default (no env var + no Docker) — opt-in mirrors thepackages/db/src/outbox-claim.test.tspattern. Verified live against the local Docker daemon (3/3 green).
Progress Log
- 2026-05-17 — Commit
db117362—03-scaling/pubsub-cache: add two-tier cache helper combining L1 cachified + L2 shared-store. NewcreateTwoTierCache<T>inpackages/shared-store/src/helpers/two-tier-cache.tspairs an in-process L1 (Map default, swappable for LRUCache) with a SharedStore L2 + pub/sub channel. Public surface:get/set/invalidate/subscribe/deleteLocal/flushLocal+l1/l2/namespaceaccessors.pubsubEnabled: falseflipsinvalidateto local-only + makessubscribe()a no-op. 9 unit tests cover round-trip, L2 fallback, cross-replica fan-out, localOnly, wildcard flush, kill-switch, custom subscriber-extra-handler. Exported from@toolctrl/shared-store. - 2026-05-17 — Commit
b258ba0c—03-scaling/pubsub-cache: migrate subscription + tenant caches to two-tier with pub/sub. Newapps/web/app/utils/services/.server/subscriptionCache.server.ts(subscription namespace) andapps/web/app/utils/db/tenant-cache.server.ts(tenant namespace). Existing cachified LRU keeps the hot read-path; the new SharedStore L2 + pub/sub broadcasts cross-replica invalidations.clearTenantSubscriptionCache(tenantId)now also callsgetSubscriptionCache().invalidate(tenantId); subscribers bridge intoclearCacheKey('tenantSubscription:<id>')/clearCacheKey('tenant:<id>'/'tenantSimple:<id>'/'tenantIdOrSlug:<id>'). Newapps/web/app/cache-bootstrap.server.tsis wired intoentry.server.tsxand is idempotent + env-gated. 12 new unit tests (5 subscriptionCache + 3 tenant-cache + 4 subscriptionService updates). - 2026-05-17 — Commit
15bf34d5—03-scaling/pubsub-cache: publish subscription invalidation from api stripe webhook. Newapps/api/src/cache/subscription-cache-publisher.tsexposespublishSubscriptionInvalidation(tenantId). Stripe webhook handler calls it fire-and-forget afterupdateTenantSubscriptionProductFromStripe. Publish errors are logged viaapiLogger.warnso a Redis outage can never 5xx the webhook (Stripe would retry). New test asserts publisher fires with the correct tenantId. - 2026-05-17 —
03-scaling/pubsub-cache: env vars, docs, progress log(this commit). AddsCACHE_INVALIDATION_ENABLED+CACHE_L1_TTL_SECONDStopackages/config/src/catalog.tsand.env.example. Addsdocs/architecture/two-tier-cache.mdcovering pattern decision rubric, channel naming, reconnect semantics, channel inventory, implementation recipe, feature-flag pre-written recipe for #53, kill-switch operational guidance. - 2026-05-20 — Commit
9edc1357—feat(feature-flags+cli): cross-process invalidation publisher (#39). NewpublishFeatureFlagInvalidationOnce+publishFeatureFlagInvalidationViaStoreexports in@toolctrl/feature-flagsprovide a one-shot SharedStore-open → publish → close lifecycle so short-lived processes (CLI, scripts) can broadcastcache:feature-flags:invalidatewithout keeping a long-lived Redis client.toolctrl-cli feature-flags {enable,disable,set-rules}now callbroadcastFlagInvalidation(name)after the DB mutation; channel namespace defaults toweband honoursFEATURE_FLAGS_PUBLISH_NAMESPACEenv override. 9 new unit tests inpackages/feature-flags/src/publisher.test.ts+packages/cli/src/commands/feature-flags/invalidate.test.tscover the wire format, env-namespace resolution, kill-switch path, and error-swallowing contract. Web + feature-flags + CLI typecheck green. - 2026-05-25 — Commits
47c37b88+f4b290bb— closes the two remaining Deviations.47c37b88(feat(shared-store): flush L1 caches on redis reconnect) addsSharedStoreAdapter.onReconnect(handler)to the adapter surface (ioredis'ready'-after-first-connect onRedisAdapter, noop onMemoryAdapter), re-exposes it on theSharedStorefacade, and wirescreateTwoTierCache().subscribe()to register a debouncedflushLocal()reconnect handler (default 1000ms, override viareconnectFlushDebounceMs). The subscribe-disposer also unregisters the reconnect handler. New unit test intwo-tier-cache.test.tsexercises the wiring through a proxy adapter (fire-then-flush, debounce-collapses-bursts, disposer-removes-handler).f4b290bb(test(shared-store): real-redis testcontainers integration for two-tier cache) addspackages/shared-store/src/helpers/two-tier-cache.testcontainers.test.tswith three integration scenarios (cross-instance pub/sub fan-out, reconnect-flush, peer-L2-mutation-visible-after-reconnect). Opt-in viaPUBSUB_TEST_REDIS_URLor a reachable Docker daemon; module-top-level boot sodescribe.skipIfsees a concrete decision pre-registration. Verified live againstredis:7-alpinevia testcontainers — 3/3 green.testcontainers ^10.28.0added as devDependency of@toolctrl/shared-store. Workspace typecheck + full test suite green (only pre-existinguiInventoryfailure remains).
Lessons Learned
- The two-tier helper's
invalidate({ localOnly: true })mode is the bridge that lets the subscriber callback purge local L1 without re-publishing — important to avoid invalidation loops. - Keeping the legacy
cachifiedLRU as the read path (rather than ripping it out) made the migration zero-risk: the new two-tier cache is purely additive (it owns the pub/sub channel) and existing route handlers see no behaviour change. - vitest 0.29 lacks
vi.hoisted(); for ESM mock factories that need a shared mock fn the workaround is to attach the fn toglobalThisand reference it via getter inside the factory. - Concurrent multi-agent setups need a serializer for shared files (config catalog, .env.example, PROGRESS.md). Stashes + autostash-pulls race, and another agent's
git stash+git resetcan wipe unstaged changes. Mitigation:git addimmediately after any edit to a shared file so the change lives in the index rather than the working tree.