94 lines
8.7 KiB
Markdown
94 lines
8.7 KiB
Markdown
# Two-Tier Cache Pattern
|
|
|
|
**Status:** active
|
|
**Owner:** platform-scaling
|
|
**Related:** [`packages/shared-store/README.md`](../../packages/shared-store/README.md), `docs/improvement-plan/03-scaling-infrastructure/pubsub-cache-invalidation.md`
|
|
|
|
## Why
|
|
|
|
A SharedStore-only cache pays one Redis round-trip per read (~1-3 ms on a healthy network). Hot paths that hit the same key 10-30 times per request (entitlement checks, limit counters, UI feature gating) accumulate enough overhead that we need an in-process L1 — but an in-process L1 alone goes stale across replicas. The two-tier cache combines both:
|
|
|
|
| Tier | Backing | Latency | Consistency |
|
|
| --- | --- | --- | --- |
|
|
| L1 | `Map` / `LRUCache` in process | sub-microsecond | per-replica |
|
|
| L2 | `SharedStore` (Redis or memory) | 1-3 ms | cross-replica |
|
|
| Pub/Sub | Redis `PUBLISH` on `cache:<ns>:invalidate` | < 50 ms typical fan-out | best-effort, see safety net |
|
|
|
|
A mutating call invalidates its local L1+L2 and publishes the key on the namespace channel. Every replica's subscriber receives the message and drops the key from its L1 (and optionally the L2). Reads probe L1 first; on miss, fall back to L2; on L2 miss, refresh from the source-of-truth (typically Postgres) and populate both tiers.
|
|
|
|
## Decision rubric
|
|
|
|
Use a two-tier cache when **all three** hold:
|
|
|
|
1. The key is read more than ~5x per request, or by every request (subscription, tenant, entitlements, feature flag).
|
|
2. Source-of-truth mutations are rare relative to reads (Stripe webhook firing seconds-to-minutes apart, admin edits).
|
|
3. Stale reads have user-visible consequences (showing the wrong plan limit, denying access after an upgrade).
|
|
|
|
Otherwise, prefer:
|
|
|
|
- **In-process LRU only** when consistency doesn't matter (route-stable computed values, request-scoped caches).
|
|
- **SharedStore cache only** when the read-rate is low and consistency must be strict (idempotency keys, session lookups).
|
|
- **No cache** when the source is already fast (cached Prisma result, in-memory blueprint).
|
|
|
|
## Channel naming
|
|
|
|
Channels follow `shared-store:<store-namespace>:ch:cache:<cache-namespace>:invalidate`. Concretely, the web SharedStore (`namespace: "web"`) emits subscription invalidations on `shared-store:web:ch:cache:subscription:invalidate`. The API SharedStore (`namespace: "api"`) emits on `shared-store:api:ch:cache:subscription:invalidate`.
|
|
|
|
For cross-app fan-out (Stripe webhook in API → web replicas), both apps must use the same SharedStore namespace so they share a channel. The current implementation uses per-app namespaces (`api`, `web`) but both connect to the same Redis logical DB index (default `1`). The actual channel string is namespaced under the writing app's SharedStore prefix. **If you need cross-app fan-out**, either:
|
|
|
|
1. Use a dedicated shared namespace (e.g. `namespace: "platform"`) for both apps' two-tier caches, or
|
|
2. Have the publishing app PUBLISH the message via a SharedStore configured with the receiving app's namespace explicitly.
|
|
|
|
The Stripe webhook → subscription pattern in this codebase publishes on `shared-store:api:ch:cache:subscription:invalidate`. For a production multi-replica web deploy to actually receive that broadcast, configure both `apps/api/src/cache/subscription-cache-publisher.ts` and `apps/web/app/utils/services/.server/subscriptionCache.server.ts` to construct their SharedStores with the same namespace (recommended: introduce `getCacheSharedStore({ namespace: "cache" })` for both apps). The current implementation broadcasts within the API process and applies the same `clearTenantSubscriptionCache` semantics on the web side via local mutation-paths that already publish; the cross-process broadcast is the work item documented under Deviations in the detail-md.
|
|
|
|
## Reconnect semantics
|
|
|
|
ioredis auto-reconnects on connection loss with exponential backoff. While disconnected, the subscriber is offline and any PUBLISH issued by another replica is lost forever (pub/sub has no persistence). Mitigations layered in order:
|
|
|
|
1. **L1 TTL is the floor.** Default `CACHE_L1_TTL_SECONDS=300` (5 min). Any missed invalidation self-heals within five minutes because the L1 entry expires. Lower the TTL for hotter paths if stale-window tolerance shrinks; raise it for marketing-content-grade caches.
|
|
2. **Reconnect flush.** Each two-tier cache exposes `flushLocal()` so a higher-level boot hook can `cache.flushLocal()` on a Redis reconnect event. The minimal implementation is "TTL is enough"; a more aggressive deployment can wire `subscriber.on('ready', () => cache.flushLocal())` in app boot.
|
|
3. **Outbox fallback.** For strictly-consistent flows (billing, entitlement enforcement), the source of truth writes an outbox event (see `docs/architecture/outbox-pattern.md`); the outbox dispatcher consumes the event and re-issues the invalidation. This guarantees at-least-once delivery and survives Redis outages — at the cost of one DB round-trip and ~1 second dispatch latency.
|
|
|
|
## Channel inventory
|
|
|
|
| Channel | Publisher | Subscriber | Payload | Defined in |
|
|
| --- | --- | --- | --- | --- |
|
|
| `cache:subscription:invalidate` | `apps/api/src/billing/stripe-webhook-route.ts` (after `updateTenantSubscriptionProductFromStripe`); `apps/web/.../subscriptionService.ts` (`clearTenantSubscriptionCache`, `clearAllSubscriptionsCache`) | `apps/web/app/utils/services/.server/subscriptionCache.server.ts` → bridges into `clearCacheKey('tenantSubscription:<tenantId>')` | tenantId (string) or `"*"` for wildcard | `subscriptionCache.server.ts` + `subscription-cache-publisher.ts` |
|
|
| `cache:tenant:invalidate` | `apps/web/.../tenant-cache.server.ts` (`publishTenantInvalidation`) | `apps/web/.../tenant-cache.server.ts` → clears `tenant:`, `tenantSimple:`, `tenantIdOrSlug:` cachified keys | tenantId (string) or `"*"` | `tenant-cache.server.ts` |
|
|
| `cache:entitlements:invalidate` | _piggy-backs on subscription_ — entitlements derive from the subscription and recompute on the next read after the subscription L1 is flushed | _piggy-backs on subscription_ | n/a — chained | n/a (chain) |
|
|
| `cache:feature-flags:invalidate` | `packages/feature-flags` publishers, including CLI feature-flag enable/disable/rule updates | web feature-flag cache subscribers through the shared two-tier cache bootstrap | `{ flagKey?, tenantId?, all? }` | `packages/feature-flags/src/publisher.ts` |
|
|
|
|
## Implementing a new two-tier cache
|
|
|
|
1. Create a `<namespace>-cache.server.ts` module per app (api/web) that wraps `createTwoTierCache({ store: getSharedStore(), namespace, ttlSeconds, pubsubEnabled, logger })`.
|
|
2. Expose two functions: `publish<Namespace>Invalidation(key)` (for mutators) and `subscribe<Namespace>Cache()` (idempotent boot hook).
|
|
3. The subscriber callback bridges into the existing local cache representation (`clearCacheKey()` for cachified-LRU, `Map.delete()` for ad-hoc maps).
|
|
4. Add the boot hook to `apps/web/app/cache-bootstrap.server.ts` (or equivalent API bootstrap) so the subscriber starts before request traffic.
|
|
5. Add publisher call-sites at every mutation that owns the SoT (database write, admin action, scheduled job).
|
|
6. Write a co-located test that drives the two-tier helper directly against a `MemoryAdapter` and asserts the cachified-bridge fires.
|
|
|
|
## Feature-Flag Cache Notes
|
|
|
|
Feature-flag invalidation is now implemented through the shared two-tier cache
|
|
publisher path. CLI feature-flag mutations publish to
|
|
`cache:feature-flags:invalidate` after the audit write, and web subscribers
|
|
flush L1 entries immediately instead of waiting for the TTL window. Keep new
|
|
feature-flag mutators on the same publisher path.
|
|
|
|
## Kill switch
|
|
|
|
`CACHE_INVALIDATION_ENABLED=false` (env var) flips both publishers and subscribers into local-only mode. The cache still works — read-through still populates L1+L2 — but invalidate becomes a local-only delete + no PUBLISH, and subscribers do not start. The system degrades to "every replica has its own L1 that expires after `CACHE_L1_TTL_SECONDS`". Use this switch:
|
|
|
|
- During a Redis incident to stop publisher errors from spamming Sentry
|
|
- During a deployment when subscribers can't keep up with traffic patterns
|
|
- During investigation of a stale-cache bug to confirm whether the publisher or subscriber path is at fault
|
|
|
|
Always re-enable after the incident; the L1-only mode has no replica-consistency floor below the L1 TTL.
|
|
|
|
## Tests
|
|
|
|
- Unit tests for `createTwoTierCache` cover round-trip, fallback, cross-replica invalidation, `localOnly`, wildcard flush, pubsub kill-switch — `packages/shared-store/src/helpers/two-tier-cache.test.ts`
|
|
- Subscription cache integration: `apps/web/app/utils/services/.server/subscriptionCache.server.test.ts`
|
|
- Tenant cache integration: `apps/web/app/utils/db/tenant-cache.server.test.ts`
|
|
- Stripe webhook publish path: `apps/api/src/billing/stripe-webhook-route.test.ts` (asserts `publishSubscriptionInvalidation(tenantId)` is called after the DB update)
|