# Test Coverage Policy

**Status:** completed; no-regression gate in maintenance
**Spec:** `docs/improvement-plan/10-compliance-polish/test-coverage.md`
**Owner:** Engineering quality / release maintenance

This document explains how test coverage is measured, gated, and reported in the
ToolCTRL boilerplate. Per-path thresholds are encoded as data in
`scripts/coverage-targets.json` and enforced by `scripts/verify-test-coverage.mjs`.
CI workflows run the gate on GitHub PRs and `push` to `main`, while the
Gitea coverage workflow currently runs on pull requests. Keep
`.github/workflows/coverage.yml`, `.gitea/workflows/coverage.yml`, and
this policy aligned when trigger coverage changes.

## Why

- **Per-path thresholds** reflect risk: auth, billing, tenant-isolation,
  shared-store, and outbox modules carry materially higher correctness and
  security risk than view components or marketing pages. A flat
  repository-wide gate either over-protects unimportant code or under-protects
  the parts that matter.
- **No-regression baseline** is active. Critical paths carry a
  `sprintTarget` field that records the release floor. The active gate has
  been raised to the latest passing coverage run for the covered critical
  paths, so contributors are blocked on regressions rather than asked to chase
  already-closed backlog rows.

## Tooling

- **Provider:** V8 native coverage via `@vitest/coverage-v8` and
  `vitest@3.2.4`.
- **Reporters:** `text-summary`, `json-summary`, `html`, `lcov`. Configured in
  `vitest.config.ts` so `scripts/verify-test-coverage.mjs` can use a stable
  `vitest run --coverage` invocation.
- **Output:** `coverage/` (git-ignored). `coverage-summary.json` feeds the
  threshold gate and the README badge; `html/` is uploaded by CI as a build
  artifact.

## Local commands

```bash
# Run vitest with coverage and enforce per-path thresholds.
pnpm run verify:test-coverage

# Print the shields.io badge URL for the current coverage-summary.json.
pnpm coverage:badge

# Render the per-path table as a markdown PR comment.
node scripts/coverage-summary.mjs --pr-comment
```

## Critical paths

The following modules carry tighter thresholds because regressions there have
disproportionate impact. Each entry in `scripts/coverage-targets.json` with
`"criticalPath": true` and a `sprintTarget` belongs to this list.

| Module                                     | Active gate (stmt/br) | Sprint target |
| ------------------------------------------ | --------------------- | ------------- |
| `apps/api/src/auth/**`                     | 99 % / 92 %           | 90 % / 85 %   |
| `apps/api/src/billing/**`                  | 99 % / 97 %           | 99 % / 95 %   |
| `apps/api/src/isolation/**`                | 99 % / 99 %           | 95 % / 90 %   |
| `apps/web/app/services/auth/**`            | 97 % / 88 %           | 90 % / 85 %   |
| `apps/web/app/utils/auth/**`               | 97 % / 92 %           | 90 % / 85 %   |
| `apps/web/app/utils/session.server.ts`     | 99 % / 98 %           | 95 % / 90 %   |
| `apps/web/app/utils/stripe.server.ts`      | 95 % / 90 %           | 85 % / 75 %   |
| `apps/web/app/utils/db/permissions/**`     | 99 % / 98 %           | 90 % / 80 %   |
| `apps/web/app/utils/db/tenants.db.server.ts` | 98 % / 95 %         | 95 % / 90 %   |
| `packages/entitlements/src/**`             | 96 % / 89 %           | 90 % / 80 %   |
| `packages/db/src/outbox*`                  | 92 % / 88 %           | 90 % / 80 %   |
| `packages/shared-store/src/**`             | 91 % / 90 %           | 90 % / 80 %   |

The active gate is the **no-regression baseline** — it is the floor below
which CI will fail. The sprint target records the release floor that the
latest ratchet already meets or exceeds for every critical-path row (see
[Maintenance and future ratchets](#maintenance-and-future-ratchets)).

## How to add a coverage gate for a new module

1. Open `scripts/coverage-targets.json`.
2. Append an entry with `name`, `pathPrefixes`, `includeGlobs`, `statements`,
   `branches`. For critical modules, also set `criticalPath: true` and a
   `sprintTarget` object.
3. Run `pnpm run verify:test-coverage` locally. The gate prints the actual
   coverage; set the active gate at or slightly below that number so the
   commit lands green.
4. Open a PR. The Coverage workflow will post a sticky comment showing the
   new row in the per-path table.

## How to bump or lower a threshold

Bumps **up** (tighter gate) — most common after writing new tests:

1. Run `pnpm run verify:test-coverage` and read the actual coverage value.
2. Edit the relevant entry in `scripts/coverage-targets.json`.
3. Reference the commit that added the new tests in the PR description.

Bumps **down** (looser gate) — should be rare and reviewed:

1. Open the PR with a justification. Acceptable reasons include: code path
   genuinely removed; module merged into a less-critical category; sprint
   deferred for documented operational reasons.
2. Request review from a code owner.
3. **Never** lower a `criticalPath` gate without compensating mitigation
   (e.g. a corresponding E2E test, a stricter integration suite). Coverage
   floor for critical paths is a security control.

## What triggers a CI failure

`pnpm run verify:test-coverage` exits non-zero in any of these cases:

- vitest itself failed to run (test failure, config error, OOM)
- `coverage/coverage-summary.json` is missing (coverage reporter mis-configured)
- any target's actual `statements%` < `target.statements` **or**
  actual `branches%` < `target.branches`

The GitHub workflow uploads `coverage/` as `coverage-report-<run_id>` even on
failure so contributors can drill into the HTML report.

## Type-only packages

Packages that intentionally emit no runtime JavaScript, such as
`@toolctrl/api-types`, must not add fake exports or `expect(true)` tests just
to satisfy statement coverage. Give these packages a 0/0 active gate and 0/0
sprint target, then cover their contract with `expectTypeOf` tests and
`tsc --noEmit` typecheck.

## Maintenance and future ratchets

Task #76 is completed. Keep the gate useful as a no-regression control:

1. Confirm `docs/testing/coverage-sprint-backlog.md` still has no
   critical-path or standard-tier positive-gap rows before release closeouts.
2. Keep each active `statements`/`branches` gate at or above the relevant
   `sprintTarget`.
3. Treat `sprintTarget` fields as explicit release-floor annotations unless a
   future cleanup removes them consistently from docs and gate data.
4. Run `pnpm run verify:test-coverage` after threshold changes — it must pass
   with the new gates.
5. Track additional Playwright/E2E coverage separately; it is not a blocker
   for the completed per-module coverage gate.

## Out of scope (deferred to follow-up tasks)

- **Coverage trend dashboard** (`docs/testing/coverage-trend.md`) — nightly
  cron-updated markdown table. Not implemented in this phase.
- **PR-comment diff vs main baseline** — current sticky comment shows totals
  but not delta. A future PR can add baseline-fetch + delta rendering similar
  to bundle-budget.
- **E2E coverage via Playwright** (5 golden-path flows) — tracked separately.
- **Additional critical-path test writing** — see
  `docs/testing/coverage-sprint-backlog.md` for the ordered backlog.

## References

- Spec: `docs/improvement-plan/10-compliance-polish/test-coverage.md`
- Gate runner: `scripts/verify-test-coverage.mjs`
- Targets data: `scripts/coverage-targets.json`
- Summary/badge: `scripts/coverage-summary.mjs`
- CI: `.github/workflows/coverage.yml`, `.gitea/workflows/coverage.yml`