Track HiveGrid Cloud planning reference
This commit is contained in:
1 parent
b3d5ea6764
commit
e341c5938a
5784 files changed
+1104929
-3
No files matched your search
@@ -0,0 +1,202 @@
|
||||
# AI-Cost Billing — Stripe Overage (Phase 3)
|
||||
|
||||
> Status: daily Stripe overage reporting is worker-wired on
|
||||
> `worker-general` (Stripe SDK 19.x + Billing Meter Events). Stripe meter
|
||||
> setup and tenant/plan configuration are operator-owned. Monthly
|
||||
> reconciliation is worker-wired on `worker-general` when Redis and
|
||||
> `STRIPE_SK` are configured; the CLI remains an offline preview path.
|
||||
> See "Operator runbook" below.
|
||||
|
||||
## Overview
|
||||
|
||||
The Phase 3 engine converts per-tenant AI overage (every cent of provider
|
||||
spend beyond the plan's `aiQuotaCentsPerMonth`) into Stripe invoice line
|
||||
items, with a markup multiplier so the boilerplate finances its own
|
||||
hosting + engineering overhead.
|
||||
|
||||
End-to-end flow:
|
||||
|
||||
```
|
||||
executeLLMCall(...) ─► outbox ─► ai-cost.persist (worker)
|
||||
├─ resolveTenantAIQuota
|
||||
├─ resolveTenantAIOverage
|
||||
├─ if projectedSpend > quota:
|
||||
│ isOverage = true
|
||||
│ markedUpCostCents = costCents × mult
|
||||
└─ write AICostEvent
|
||||
│
|
||||
▼
|
||||
ai-cost-stripe-report (nightly cron)
|
||||
├─ aggregate WHERE isOverage = true
|
||||
│ AND stripeReportedAt IS NULL
|
||||
├─ round UP to plan.aiOverageBlockCents
|
||||
├─ stripe.billing.meterEvents
|
||||
│ .create({
|
||||
│ event_name,
|
||||
│ payload: { stripe_customer_id, value }
|
||||
│ }, { idempotencyKey })
|
||||
└─ stamp stripeReportedAt = now
|
||||
│
|
||||
▼
|
||||
Stripe invoice (next period close)
|
||||
│
|
||||
▼
|
||||
ai-cost-reconciliation (monthly worker cron)
|
||||
compares internal vs
|
||||
stripe.billing.meters
|
||||
.listEventSummaries
|
||||
alert when diff > 1%
|
||||
```
|
||||
|
||||
## Plan markup defaults
|
||||
|
||||
| Plan | Multiplier | Block (c) | Default state |
|
||||
| -------- | ---------- | --------- | ----------------------------------------------- |
|
||||
| Starter | 1.0× | 10 | No overage (quota = 0 forces per-tenant unlock) |
|
||||
| Pro | 2.0× | 10 | Overage on once Stripe meter is configured |
|
||||
| Business | 1.5× | 10 | Overage on once Stripe meter is configured |
|
||||
|
||||
Defaults live in
|
||||
[`packages/entitlements/src/ai-overage.ts`](../../packages/entitlements/src/ai-overage.ts).
|
||||
Per-tenant overrides live on `Tenant.aiQuotaOverrideJson` (the same Json
|
||||
column used by Phase 2 quotas; the override parser is shape-tolerant so
|
||||
both blobs coexist).
|
||||
|
||||
## Operator runbook
|
||||
|
||||
### 1. Create the Stripe meter / price
|
||||
|
||||
This repo runs Stripe SDK 19 with Billing Meter Events. Setup:
|
||||
|
||||
1. Stripe dashboard → Products → New product.
|
||||
2. Add a **metered** recurring price backed by a Billing Meter. Choose either:
|
||||
- **Per unit** (e.g. `0.001 EUR / cent`) — simplest, what Phase 3
|
||||
unit-tests assume.
|
||||
- **Tiered (graduated)** — preferred when bigger tenants should get
|
||||
volume discounts. The reporter submits `quantity = cents` either
|
||||
way.
|
||||
3. Copy the Billing Meter event name (default `ai-overage`) and attach the
|
||||
price to the subscriptions of the relevant plan (Subscription → Add
|
||||
product → choose the metered price you created).
|
||||
4. Admin → AI Overage Pricing (`/admin/settings/pricing/ai-overage`) —
|
||||
paste the meter event name into the Stripe Meter field for that plan.
|
||||
The action handler audit-logs the change.
|
||||
5. Stripe → API keys → set `STRIPE_SK` (test mode for staging,
|
||||
`sk_live_...` for production).
|
||||
|
||||
### 2. Configure killswitch + cron
|
||||
|
||||
- `.env`: ensure `AI_OVERAGE_SUBMISSION_ENABLED` is unset or `true`.
|
||||
- Worker bootstrap: `apps/worker/src/index.ts` registers
|
||||
`registerAICostStripeReportCron(...)` on `worker-general` when Redis is
|
||||
configured. The maintenance handler reads the killswitch and bails early
|
||||
when set.
|
||||
- Monthly reconciliation: `apps/worker/src/index.ts` registers
|
||||
`registerAICostReconciliationCron(...)` on `worker-general` when Redis,
|
||||
`STRIPE_SK`, and `AI_COST_RECONCILIATION_ENABLED!=false` are present. It
|
||||
runs on the dedicated `toolctrl.ai-cost-reconciliation` queue on the 5th
|
||||
of each month at 03:00 UTC.
|
||||
|
||||
### 3. Verify on staging
|
||||
|
||||
```
|
||||
toolctrl-cli ai-cost show-billing --tenant <tenantId>
|
||||
toolctrl-cli ai-cost report-overage --tenant <tenantId> # dry-run
|
||||
toolctrl-cli ai-cost reconcile --tenant <tenantId> --month 2026-05
|
||||
```
|
||||
|
||||
End-to-end smoke:
|
||||
|
||||
1. Set `Tenant.aiQuotaOverrideJson = { aiQuotaCentsPerMonth: 100 }` on
|
||||
the test tenant.
|
||||
2. Drive `executeLLMCall` until total cost exceeds 100c. Confirm
|
||||
`AICostEvent.isOverage = true` and `markedUpCostCents = 2x cost` for
|
||||
Pro.
|
||||
3. Trigger the nightly cron. Confirm a meter event appears on the
|
||||
Stripe dashboard for the configured meter/customer.
|
||||
4. Wait until the next Stripe invoice period — verify the overage
|
||||
appears as an invoice line item.
|
||||
|
||||
## Idempotency
|
||||
|
||||
The reporter constructs the Stripe idempotency key deterministically:
|
||||
|
||||
```
|
||||
ai-overage:<tenantId>:<periodKey>:<roundedCents>:<eventCount>
|
||||
```
|
||||
|
||||
Retries with identical inputs collapse to a single Stripe-side charge.
|
||||
The cron's per-tenant failure isolation logs the failure, leaves
|
||||
`stripeReportedAt` NULL, and a later run retries.
|
||||
|
||||
## Reconciliation procedure
|
||||
|
||||
The worker reconciliation path sums internal overage
|
||||
`markedUpCostCents WHERE createdAt IN <prior-month>` and compares it to
|
||||
Stripe Billing Meter event summaries for the same customer + meter window.
|
||||
Diffs above 1% are logged through the worker notifier. The CLI preview still
|
||||
sums internal totals offline and is useful before enabling `STRIPE_SK` in a
|
||||
worker environment.
|
||||
|
||||
Remaining precision gap: the reporter rounds each submission up to the
|
||||
tenant's `aiOverageBlockCents`, but the database does not yet persist a
|
||||
submission ledger with the rounded quantity per Stripe meter event. The
|
||||
worker therefore reconciles raw expected overage cents against Stripe's
|
||||
meter summaries; exact penny-level reconciliation needs the planned
|
||||
`AIOverageSubmission` audit table.
|
||||
|
||||
Manual investigation steps on mismatch:
|
||||
|
||||
1. `toolctrl-cli ai-cost reconcile --tenant <tenantId> --month <m>` —
|
||||
inspect internal totals offline.
|
||||
2. Stripe dashboard → Billing Meter → event summaries → diff the reported
|
||||
quantities.
|
||||
3. If a row in `AICostEvent` is missing `stripeReportedAt` but Stripe
|
||||
shows it, the most common cause is a previous `markEventsReported`
|
||||
crashed between the Stripe success and DB stamp; rerun the cron and
|
||||
stamp manually if necessary.
|
||||
|
||||
## Security
|
||||
|
||||
Per the Phase-3 spec security notes:
|
||||
|
||||
- Tenant UI shows `markedUpCostCents` only — provider `costCents` is
|
||||
stripped from the loader payload.
|
||||
- Every markup mutation flows through the audit log
|
||||
(`createAdminLog` → `AuditEvent`).
|
||||
- Submission carries no PII — only aggregate cents.
|
||||
|
||||
## Deviations from plan
|
||||
|
||||
- **Stripe SDK migration complete.** Older iterations used legacy
|
||||
subscription-item usage records. The current implementation uses
|
||||
`stripe.billing.meterEvents.create(...)` behind the
|
||||
`StripeMeterEventSubmitter` interface and persists only
|
||||
`aiOverageStripeMeterId` for plan configuration.
|
||||
- **Plan defaults edit-via-code.** The admin UI surfaces the
|
||||
per-plan defaults read-only and audit-logs would-be changes; the
|
||||
canonical numbers stay in `standardAIOverageDefaults` until a
|
||||
follow-up moves them into a DB table. Per-tenant overrides on
|
||||
`Tenant.aiQuotaOverrideJson` are fully live today and recommended
|
||||
for enterprise deals.
|
||||
- **Per-tenant override field name.** Phase-3 spec called for a
|
||||
dedicated `Tenant.aiMarkupOverride Float?` column; we instead extend
|
||||
the existing `aiQuotaOverrideJson` blob with new optional keys. This
|
||||
keeps the migration surface zero and matches the Phase-2 override
|
||||
parser style.
|
||||
|
||||
## Stripe Meters API (SDK 19, current)
|
||||
|
||||
The repo runs Stripe SDK 19 (`apiVersion: "2025-10-29.clover"`).
|
||||
Overage submission goes through `stripe.billing.meterEvents.create`
|
||||
with payload `{ stripe_customer_id, value: rounded }` and the
|
||||
operator-provisioned Meter event name (default `ai-overage`). The
|
||||
production wireup lives in
|
||||
`apps/worker/src/jobs/ai-cost-stripe-submitter.ts`; the cron is
|
||||
registered by `apps/worker/src/jobs/ai-cost-stripe-report-cron.ts`.
|
||||
Per-tenant configuration uses `aiOverageStripeMeterId` only; the
|
||||
legacy `aiOverageStripeSubscriptionItemId` field was retired.
|
||||
|
||||
For reconciliation use `stripe.billing.meters.listEventSummaries`
|
||||
with `value_grouping_window=hour` and an explicit `customer` +
|
||||
`start_time` / `end_time` window.
|
||||
Reference in new issue
Block a user