# AI Cost Tracking — Phase 1

**Status:** Phase-1 (foundation) shipped. Phases 2 (quota enforcement)
and 3 (Stripe overage billing) build on the pipeline below.

## Architecture

Three-layer pipeline that attributes every LLM call to a tenant +
user and persists the cost asynchronously via the transactional
outbox:

```
LLM call-site
    │
    ▼
executeLLMCall          ← packages/ai/src/cost/executeLLMCall.ts
  ├─ countTokens()      ← packages/ai/src/cost/tokens.ts
  ├─ countCallCost()    ← packages/ai/src/cost/cost.ts
  └─ publishOutboxEvent ← @toolctrl/db/outbox
                        │
                        ▼
                 OutboxEvent (PENDING)
                        │  dispatcher polls (1s)
                        ▼
                 BullMQ outbox.ai-cost queue
                        │
                        ▼
                 ai-cost.persist handler  ← apps/worker/src/jobs/ai-cost-persist.ts
                                                bootstrapped by worker-general
                        │
                        ▼
                 AICostEvent (append-only row)
```

`apps/worker/src/index.ts` registers the `outbox.ai-cost` subscriber on
`worker-general` when Redis is configured. Set
`AI_COST_PERSIST_ENABLED=false` to pause persistence during incident response;
leave it enabled in staging/production so the outbox does not report cost
events as dispatched while no consumer writes `AICostEvent` rows.

The `apps/worker/src/jobs/ai-cost-aggregate.ts` rollup helper upserts
`TenantAICostSummary` rows so dashboard top-line lookups stay O(1).
`apps/worker/src/index.ts` registers the dedicated
`toolctrl.ai-cost-aggregate` scheduler/worker on the `general` worker
group when Redis is configured. Set `AI_COST_AGGREGATE_ENABLED=false`
or `0` to disable it during incident response.

## Why a wrapper instead of provider-native usage?

- **Multi-provider consistency.** OpenAI, Anthropic, Mistral and
  Scaleway expose usage telemetry in slightly different shapes. The
  wrapper normalises to one schema.
- **Hot-path latency.** Synchronous cost-write would block the LLM
  stream. The outbox lets us commit the cost-event in the surrounding
  transaction and persist async.
- **PII hygiene.** The wrapper closes over the prompt/response text;
  only counts + metadata leak into the outbox payload (see "Security"
  below).

## API

```ts
import { executeLLMCall } from "@toolctrl/ai/cost";

const result = await executeLLMCall({
  provider: "openai",
  model: "gpt-4o-mini",
  capability: "chat",
  ctx: { tenantId, userId, requestId: crypto.randomUUID() },
  run: async () => {
    const response = await openai.chat.completions.create({...});
    return {
      output: response.choices[0].message.content ?? "",
      usage: {
        inputTokens: response.usage?.prompt_tokens,
        outputTokens: response.usage?.completion_tokens,
      },
    };
  },
});
```

When the provider does not return usage numbers, populate
`inputForCount` / `outputForCount` so the wrapper falls back to
`countTokens`.

## Pricing tables

Pricing is checked-in (`packages/ai/src/cost/pricing/<version>.ts`)
and **immutable once shipped**. Adding a price change:

1. Copy the latest `v<YYYY-MM-DD>.ts` to a new file with today's date.
2. Edit the rows that changed.
3. Update `LATEST_PRICING_VERSION` in `pricing/index.ts`.
4. Append the new entry to `PRICING_VERSIONS`.

Historical events keep referring to their original pricing version
via `AICostEvent.pricingVersion` — so a 6-month-old event still
renders with the cents it was originally billed at.

Phase-1 ships covers OpenAI (10 models), Anthropic (6), Mistral (7),
Scaleway (5) and Aleph-Alpha (5).

## Operational toggles

| Env | Default | Effect |
|---|---|---|
| `AI_COST_TRACKING_ENABLED=false` | enabled | Skip outbox publish entirely; LLM hot-path keeps working |

## Idempotency

Every event carries a `requestId` which the publisher writes as the
outbox `dispatchKey` AND which is uniquely indexed on `AICostEvent`.
Re-deliveries collapse to a no-op via the existing outbox
dedup-on-conflict path.

## Tenant isolation

- `AICostEvent.tenantId` is nullable to support admin-only chats; rows
  cascade-delete with the Tenant.
- Dashboard loaders strictly filter by `tenantId`.
- Admin analytics endpoints are gated behind the `admin` role.

## Security

- The outbox payload contains **counts + metadata only**. Prompt and
  response text never leave the wrapper closure.
- CSV-exports cap at 100k rows; larger exports go through the
  background-job queue (planned).
- Pricing tables are git-versioned — no DB override path.
- Cost wrapper adds <5ms in the LLM hot-path (tokenizer is sync;
  outbox insert is one round-trip co-located with existing tx).

## Spec

Full spec, acceptance criteria and migration plan:
[`docs/improvement-plan/05-business-features/ai-cost-phase-1.md`](../improvement-plan/05-business-features/ai-cost-phase-1.md).