203 lines
9.1 KiB
Markdown
203 lines
9.1 KiB
Markdown
# AI-Cost Billing — Stripe Overage (Phase 3)
|
||
|
||
> Status: daily Stripe overage reporting is worker-wired on
|
||
> `worker-general` (Stripe SDK 19.x + Billing Meter Events). Stripe meter
|
||
> setup and tenant/plan configuration are operator-owned. Monthly
|
||
> reconciliation is worker-wired on `worker-general` when Redis and
|
||
> `STRIPE_SK` are configured; the CLI remains an offline preview path.
|
||
> See "Operator runbook" below.
|
||
|
||
## Overview
|
||
|
||
The Phase 3 engine converts per-tenant AI overage (every cent of provider
|
||
spend beyond the plan's `aiQuotaCentsPerMonth`) into Stripe invoice line
|
||
items, with a markup multiplier so the boilerplate finances its own
|
||
hosting + engineering overhead.
|
||
|
||
End-to-end flow:
|
||
|
||
```
|
||
executeLLMCall(...) ─► outbox ─► ai-cost.persist (worker)
|
||
├─ resolveTenantAIQuota
|
||
├─ resolveTenantAIOverage
|
||
├─ if projectedSpend > quota:
|
||
│ isOverage = true
|
||
│ markedUpCostCents = costCents × mult
|
||
└─ write AICostEvent
|
||
│
|
||
▼
|
||
ai-cost-stripe-report (nightly cron)
|
||
├─ aggregate WHERE isOverage = true
|
||
│ AND stripeReportedAt IS NULL
|
||
├─ round UP to plan.aiOverageBlockCents
|
||
├─ stripe.billing.meterEvents
|
||
│ .create({
|
||
│ event_name,
|
||
│ payload: { stripe_customer_id, value }
|
||
│ }, { idempotencyKey })
|
||
└─ stamp stripeReportedAt = now
|
||
│
|
||
▼
|
||
Stripe invoice (next period close)
|
||
│
|
||
▼
|
||
ai-cost-reconciliation (monthly worker cron)
|
||
compares internal vs
|
||
stripe.billing.meters
|
||
.listEventSummaries
|
||
alert when diff > 1%
|
||
```
|
||
|
||
## Plan markup defaults
|
||
|
||
| Plan | Multiplier | Block (c) | Default state |
|
||
| -------- | ---------- | --------- | ----------------------------------------------- |
|
||
| Starter | 1.0× | 10 | No overage (quota = 0 forces per-tenant unlock) |
|
||
| Pro | 2.0× | 10 | Overage on once Stripe meter is configured |
|
||
| Business | 1.5× | 10 | Overage on once Stripe meter is configured |
|
||
|
||
Defaults live in
|
||
[`packages/entitlements/src/ai-overage.ts`](../../packages/entitlements/src/ai-overage.ts).
|
||
Per-tenant overrides live on `Tenant.aiQuotaOverrideJson` (the same Json
|
||
column used by Phase 2 quotas; the override parser is shape-tolerant so
|
||
both blobs coexist).
|
||
|
||
## Operator runbook
|
||
|
||
### 1. Create the Stripe meter / price
|
||
|
||
This repo runs Stripe SDK 19 with Billing Meter Events. Setup:
|
||
|
||
1. Stripe dashboard → Products → New product.
|
||
2. Add a **metered** recurring price backed by a Billing Meter. Choose either:
|
||
- **Per unit** (e.g. `0.001 EUR / cent`) — simplest, what Phase 3
|
||
unit-tests assume.
|
||
- **Tiered (graduated)** — preferred when bigger tenants should get
|
||
volume discounts. The reporter submits `quantity = cents` either
|
||
way.
|
||
3. Copy the Billing Meter event name (default `ai-overage`) and attach the
|
||
price to the subscriptions of the relevant plan (Subscription → Add
|
||
product → choose the metered price you created).
|
||
4. Admin → AI Overage Pricing (`/admin/settings/pricing/ai-overage`) —
|
||
paste the meter event name into the Stripe Meter field for that plan.
|
||
The action handler audit-logs the change.
|
||
5. Stripe → API keys → set `STRIPE_SK` (test mode for staging,
|
||
`sk_live_...` for production).
|
||
|
||
### 2. Configure killswitch + cron
|
||
|
||
- `.env`: ensure `AI_OVERAGE_SUBMISSION_ENABLED` is unset or `true`.
|
||
- Worker bootstrap: `apps/worker/src/index.ts` registers
|
||
`registerAICostStripeReportCron(...)` on `worker-general` when Redis is
|
||
configured. The maintenance handler reads the killswitch and bails early
|
||
when set.
|
||
- Monthly reconciliation: `apps/worker/src/index.ts` registers
|
||
`registerAICostReconciliationCron(...)` on `worker-general` when Redis,
|
||
`STRIPE_SK`, and `AI_COST_RECONCILIATION_ENABLED!=false` are present. It
|
||
runs on the dedicated `toolctrl.ai-cost-reconciliation` queue on the 5th
|
||
of each month at 03:00 UTC.
|
||
|
||
### 3. Verify on staging
|
||
|
||
```
|
||
toolctrl-cli ai-cost show-billing --tenant <tenantId>
|
||
toolctrl-cli ai-cost report-overage --tenant <tenantId> # dry-run
|
||
toolctrl-cli ai-cost reconcile --tenant <tenantId> --month 2026-05
|
||
```
|
||
|
||
End-to-end smoke:
|
||
|
||
1. Set `Tenant.aiQuotaOverrideJson = { aiQuotaCentsPerMonth: 100 }` on
|
||
the test tenant.
|
||
2. Drive `executeLLMCall` until total cost exceeds 100c. Confirm
|
||
`AICostEvent.isOverage = true` and `markedUpCostCents = 2x cost` for
|
||
Pro.
|
||
3. Trigger the nightly cron. Confirm a meter event appears on the
|
||
Stripe dashboard for the configured meter/customer.
|
||
4. Wait until the next Stripe invoice period — verify the overage
|
||
appears as an invoice line item.
|
||
|
||
## Idempotency
|
||
|
||
The reporter constructs the Stripe idempotency key deterministically:
|
||
|
||
```
|
||
ai-overage:<tenantId>:<periodKey>:<roundedCents>:<eventCount>
|
||
```
|
||
|
||
Retries with identical inputs collapse to a single Stripe-side charge.
|
||
The cron's per-tenant failure isolation logs the failure, leaves
|
||
`stripeReportedAt` NULL, and a later run retries.
|
||
|
||
## Reconciliation procedure
|
||
|
||
The worker reconciliation path sums internal overage
|
||
`markedUpCostCents WHERE createdAt IN <prior-month>` and compares it to
|
||
Stripe Billing Meter event summaries for the same customer + meter window.
|
||
Diffs above 1% are logged through the worker notifier. The CLI preview still
|
||
sums internal totals offline and is useful before enabling `STRIPE_SK` in a
|
||
worker environment.
|
||
|
||
Remaining precision gap: the reporter rounds each submission up to the
|
||
tenant's `aiOverageBlockCents`, but the database does not yet persist a
|
||
submission ledger with the rounded quantity per Stripe meter event. The
|
||
worker therefore reconciles raw expected overage cents against Stripe's
|
||
meter summaries; exact penny-level reconciliation needs the planned
|
||
`AIOverageSubmission` audit table.
|
||
|
||
Manual investigation steps on mismatch:
|
||
|
||
1. `toolctrl-cli ai-cost reconcile --tenant <tenantId> --month <m>` —
|
||
inspect internal totals offline.
|
||
2. Stripe dashboard → Billing Meter → event summaries → diff the reported
|
||
quantities.
|
||
3. If a row in `AICostEvent` is missing `stripeReportedAt` but Stripe
|
||
shows it, the most common cause is a previous `markEventsReported`
|
||
crashed between the Stripe success and DB stamp; rerun the cron and
|
||
stamp manually if necessary.
|
||
|
||
## Security
|
||
|
||
Per the Phase-3 spec security notes:
|
||
|
||
- Tenant UI shows `markedUpCostCents` only — provider `costCents` is
|
||
stripped from the loader payload.
|
||
- Every markup mutation flows through the audit log
|
||
(`createAdminLog` → `AuditEvent`).
|
||
- Submission carries no PII — only aggregate cents.
|
||
|
||
## Deviations from plan
|
||
|
||
- **Stripe SDK migration complete.** Older iterations used legacy
|
||
subscription-item usage records. The current implementation uses
|
||
`stripe.billing.meterEvents.create(...)` behind the
|
||
`StripeMeterEventSubmitter` interface and persists only
|
||
`aiOverageStripeMeterId` for plan configuration.
|
||
- **Plan defaults edit-via-code.** The admin UI surfaces the
|
||
per-plan defaults read-only and audit-logs would-be changes; the
|
||
canonical numbers stay in `standardAIOverageDefaults` until a
|
||
follow-up moves them into a DB table. Per-tenant overrides on
|
||
`Tenant.aiQuotaOverrideJson` are fully live today and recommended
|
||
for enterprise deals.
|
||
- **Per-tenant override field name.** Phase-3 spec called for a
|
||
dedicated `Tenant.aiMarkupOverride Float?` column; we instead extend
|
||
the existing `aiQuotaOverrideJson` blob with new optional keys. This
|
||
keeps the migration surface zero and matches the Phase-2 override
|
||
parser style.
|
||
|
||
## Stripe Meters API (SDK 19, current)
|
||
|
||
The repo runs Stripe SDK 19 (`apiVersion: "2025-10-29.clover"`).
|
||
Overage submission goes through `stripe.billing.meterEvents.create`
|
||
with payload `{ stripe_customer_id, value: rounded }` and the
|
||
operator-provisioned Meter event name (default `ai-overage`). The
|
||
production wireup lives in
|
||
`apps/worker/src/jobs/ai-cost-stripe-submitter.ts`; the cron is
|
||
registered by `apps/worker/src/jobs/ai-cost-stripe-report-cron.ts`.
|
||
Per-tenant configuration uses `aiOverageStripeMeterId` only; the
|
||
legacy `aiOverageStripeSubscriptionItemId` field was retired.
|
||
|
||
For reconciliation use `stripe.billing.meters.listEventSummaries`
|
||
with `value_grouping_window=hour` and an explicit `customer` +
|
||
`start_time` / `end_time` window.
|