Track HiveGrid Cloud planning reference

hivegrid-cloud/docs/testing/coverage-baseline-2026-05-25.md

@@ -0,0 +1,138 @@
+# Coverage Baseline — 2026-05-25
+
+> Historical baseline only. This file records the first 2026-05-25 coverage snapshot and is no longer the current gap source of truth. Use `docs/testing/coverage-sprint-backlog.md` plus `scripts/coverage-targets.json` for current coverage status and active gates.
+
+**Captured by:** Task E follow-up to #76 (HANDOFF-2026-05-25)
+**Source:** `pnpm exec vitest run --coverage` (4 pre-existing failing tests excluded so the v8 reporter could emit `coverage/coverage-summary.json`)
+**Wall-clock runtime:** 136 seconds (~2.3 min) on local dev machine — fits well inside the documented 5-10 min budget.
+
+## Total
+
+| metric     |  total |
+| ---------- | -----: |
+| statements | 76.57% |
+| branches   | 77.76% |
+| functions  | 66.22% |
+| lines      | 76.57% |
+
+## Historical Critical-path modules — sorted by smallest gap to `sprintTarget`
+
+The table below is preserved to explain the original remediation order. Several rows were subsequently closed and gate-raised; do not use this table as the current open-backlog list.
+
+Negative gap = module already past sprintTarget (graduation candidate).
+"Gap" columns show `sprintTarget - actual` for statements / branches.
+
+| Module                                       | stmt % | br %  | fn %  | gate (stmt/br) | sprint (stmt/br) | gap stmt | gap br | status              |
+| -------------------------------------------- | -----: | ----: | ----: | -------------: | ---------------: | -------: | -----: | ------------------- |
+| `apps/api/src/auth/**`                       |  99.2  |  92.2 |  88.9 |          70/60 |            90/85 |    -9.2  |  -7.2  | GRADUATED           |
+| `apps/web/app/utils/auth/**`                 |  94.7  |  86.0 |  95.2 |          60/50 |            90/85 |    -4.7  |  -1.0  | GRADUATED           |
+| `packages/entitlements/src/**`               |  97.0  |  90.8 | 100.0 |          60/50 |            90/80 |    -7.0  | -10.8  | GRADUATED           |
+| `packages/db/src/outbox*`                    |  93.0  |  88.6 | 100.0 |          60/50 |            90/80 |    -3.0  |  -8.6  | GRADUATED           |
+| `packages/shared-store/src/**`               |  94.4  |  91.0 |  97.5 |          60/50 |            90/80 |    -4.4  | -11.0  | GRADUATED           |
+| `apps/api/src/billing/**`                    |  89.8  |  62.1 | 100.0 |          60/50 |            85/75 |    -4.8  |  12.9  | branches gap remains |
+| `apps/api/src/isolation/**`                  | 100.0  |  66.7 | 100.0 |          70/60 |            95/90 |    -5.0  |  23.3  | branches gap remains |
+| `apps/web/app/services/auth/**`              |  78.7  |  85.8 |  82.0 |          60/50 |            90/85 |    11.3  |  -0.8  | statements gap remains |
+| `apps/web/app/utils/session.server.ts`       |  90.0  |  88.6 |  87.5 |          60/50 |            95/90 |     5.0  |   1.4  | small gap           |
+| `apps/web/app/utils/db/permissions/**`       |  15.9  |  60.9 |   7.8 |          50/40 |            90/80 |    74.1  |  19.1  | LARGE GAP           |
+| `apps/web/app/utils/stripe.server.ts`        |   3.5  |   0.0 |   0.0 |          50/40 |            85/75 |    81.5  |  75.0  | LARGE GAP           |
+| `apps/web/app/utils/db/tenants.db.server.ts` |   2.9  |   0.0 |   0.0 |          50/40 |            95/90 |    92.1  |  90.0  | LARGE GAP           |
+
+## Standard-tier modules — gap to sprintTarget
+
+Negative gap = already past sprintTarget.
+
+| Module                          | stmt % | br %  | gate  | sprint | gap stmt | gap br | status     |
+| ------------------------------- | -----: | ----: | ----: | -----: | -------: | -----: | ---------- |
+| `apps/api/src` (overall)        |  78.5  |  78.1 | 70/60 |   none |        - |    -   | -          |
+| `apps/worker/src/**`            |  80.9  |  76.8 | 50/40 |   none |        - |    -   | passes     |
+| `apps/hocuspocus/src/**`        |  58.9  |  90.7 | 40/30 |  70/60 |    11.1  | -30.7  | branches past |
+| `apps/mobile/src/**`            |  45.3  |  75.6 | 40/30 |  70/60 |    24.7  | -15.6  | branches past |
+| `packages/ai/src/**`            |  81.9  |  81.3 | 40/30 |  80/70 |    -1.9  | -11.3  | GRADUATED  |
+| `packages/cli/src/**`           |  65.9  |  79.1 | 40/30 |  80/70 |    14.1  |  -9.1  | branches past |
+| `packages/config/src/**`        |  99.0  |  95.7 | 50/40 |  80/70 |   -19.0  | -25.7  | GRADUATED  |
+| `packages/db/src/**` (boundary) |  94.1  |  81.6 | 60/50 |   none |        - |    -   | -          |
+| `packages/feature-flags/src/**` |  94.3  |  86.4 | 50/40 |  80/70 |   -14.3  | -16.4  | GRADUATED  |
+| `packages/foundation/src/**`    |  87.7  |  86.4 | 70/60 |   none |        - |    -   | passes     |
+| `packages/i18n/src/**`          |  95.2  |  75.7 | 50/40 |  80/70 |   -15.2  |  -5.7  | GRADUATED  |
+| `packages/mail/src/**`          |  98.4  |  92.7 | 75/65 |   none |        - |    -   | passes     |
+| `packages/plugin-runtime/src/**`|  92.7  |  89.0 | 50/40 |  80/70 |   -12.7  | -19.0  | GRADUATED  |
+| `packages/queue/src/**`         |  93.5  |  96.1 | 40/30 |  80/70 |   -13.5  | -26.1  | GRADUATED  |
+| `packages/exports/src/**`       |  90.7  |  85.4 | 50/40 |  80/70 |   -10.7  | -15.4  | GRADUATED  |
+| `packages/imports/src/**`       |  91.8  |  87.5 | 40/30 |  80/70 |   -11.8  | -17.5  | GRADUATED  |
+| `packages/realtime/src/**`      |  90.5  |  82.7 | 50/40 |  80/70 |   -10.5  | -12.7  | GRADUATED  |
+| `packages/search/src/**`        |  88.6  |  71.5 | 40/30 |  80/70 |    -8.6  |  -1.5  | GRADUATED  |
+| `packages/storage/src/**`       |  83.4  |  84.7 | 50/40 |  80/70 |    -3.4  | -14.7  | GRADUATED  |
+| `packages/vector-store/src/**`  |  94.6  |  90.4 | 50/40 |  80/70 |   -14.6  | -20.4  | GRADUATED  |
+| `packages/metrics/src/**`       |  98.9  |  90.5 | 50/40 |  80/70 |   -18.9  | -20.5  | GRADUATED  |
+| `packages/form-widget/src/**`   |  96.7  |  88.2 | 50/40 |  80/70 |   -16.7  | -18.2  | GRADUATED  |
+| `packages/email-sequences/src/**`|  84.5 |  78.0 | 50/40 |  80/70 |    -4.5  |  -8.0  | GRADUATED  |
+| `packages/email-templates/src/**`|  96.3 |  76.7 | 30/20 |  70/60 |   -26.3  | -16.7  | GRADUATED  |
+| `packages/error-reporter/src/**`|  78.3  |  82.6 | 50/40 |  80/70 |     1.7  | -12.6  | tiny stmt gap |
+| `packages/observability/src/**` | 100.0  | 100.0 | 40/30 |  80/70 |   -20.0  | -30.0  | GRADUATED  |
+| `packages/marketing-outbound/src/**`| 85.8 | 65.9 | 30/20 |  70/60 |   -15.8  |  -5.9  | GRADUATED  |
+| `packages/api-client-ts/src/**` |  15.2  |  85.9 | 30/20 |  70/60 |    54.8  | -25.9  | LARGE stmt gap (branches past) |
+| `packages/plugins/src/**`       |  98.7  |  72.6 | 40/30 |  75/65 |   -23.7  |  -7.6  | GRADUATED  |
+| `packages/plugin-sdk/src/**`    | 100.0  | 100.0 | 40/30 |  75/65 |   -25.0  | -35.0  | GRADUATED  |
+| `packages/ui-app/src/**`        |  59.0  |  69.2 | 40/30 |  70/60 |    11.0  |  -9.2  | branches past |
+| `packages/ui-mock/src/**`       |  97.9  |  68.6 | 40/30 |  70/60 |   -27.9  |  -8.6  | GRADUATED  |
+| `packages/ui-subframe/src/**`   | 100.0  | 100.0 | 30/20 |  50/40 |   -50.0  | -60.0  | GRADUATED  (visual via lost-pixel) |
+| `packages/view-models/src/**`   |  86.7  |  71.4 | 40/30 |  75/65 |   -11.7  |  -6.4  | GRADUATED  |
+| `packages/url/src/**`           |  98.3  |  96.0 | 50/40 |  80/70 |   -18.3  | -26.0  | GRADUATED  |
+| `packages/logger/src/**`        | 100.0  | 100.0 | 50/40 |  80/70 |   -20.0  | -30.0  | GRADUATED  |
+| `packages/brand/src/**`         | 100.0  |  93.8 | 50/40 |  80/70 |   -20.0  | -23.8  | GRADUATED  |
+| `packages/api-types/src/**`     |   0.0  | 100.0 | 30/20 |  60/50 |    60.0  | -50.0  | stmt gap (file is type-only) |
+
+## Methodology notes
+
+- Coverage is captured by `vitest@3.2.4` with `@vitest/coverage-v8`.
+  Reporter list: `text-summary`, `json-summary`, `html`, `lcov` (see
+  `vitest.config.ts`).
+- 4 pre-existing test failures were excluded for this baseline run so the
+  v8 reporter could emit `coverage-summary.json`. They are tracked
+  separately and are out of scope for this baseline:
+  - `apps/web/app/application/ui/uiInventory.test.ts` (acknowledged in
+    HANDOFF as the allowed pre-existing failure)
+  - `apps/api/src/app.test.ts` — Stripe webhook integration test fails
+    when running outside the `TOOLCTRL_CORE_TESTS_ONLY=1` filter
+  - `packages/config/src/env-examples.test.ts` — `.env.example` drift
+    against the cataloged env schema (plugin keys uncataloged)
+  - `packages/cli/src/commands/tenant/json-codec.test.ts` — JSON
+    snapshot replacer expectation mismatch (`Buffer` serializer)
+- The repo-wide `pnpm test` filters down to `apps/web/app/`, `scripts/`,
+  selected `tools/`, and a handful of `packages/db` boundary tests — so
+  the failures above only surface under the unfiltered coverage run.
+  This is an existing coverage-scope gap noted in
+  `docs/testing/coverage-sprint-backlog.md`.
+
+## Targeted-wins decision for this session
+
+The task brief says: "If after recon you find that no critical-path
+module has a gap < 5pp to its sprintTarget, write that finding into the
+baseline doc and SKIP the two targeted-wins step entirely."
+
+Among critical-path modules with a positive gap (i.e., still below
+sprintTarget):
+
+- `apps/web/app/utils/session.server.ts` — gap 5.0 pp statements /
+  1.4 pp branches. **Statements gap is exactly 5pp, not below it**;
+  branches gap is below 5pp but moving branches alone does not lift
+  statements past sprintTarget.
+- `apps/api/src/billing/**` — branches gap 12.9 pp.
+- `apps/api/src/isolation/**` — branches gap 23.3 pp.
+- `apps/web/app/services/auth/**` — statements gap 11.3 pp (branches
+  already past).
+
+Decision: skip the test-writing step in this session. Instead, **graduate
+gates** for the modules already past sprintTarget is the high-leverage
+move — it prevents regression without padding test files. See
+`docs/testing/coverage-sprint-backlog.md` for the ordered work-list of
+remaining positive-gap modules.
+
+The two gates graduated in this session are:
+
+1. `apps/api/src/auth/**` — bumped from 70/60 to 99/92 (current 99.2/92.2)
+2. `packages/entitlements/src/**` — bumped from 60/50 to 96/90 (current 97.0/90.8)
+
+These two were picked as "smallest absolute gap to sprintTarget among
+critical-path graduations" — both are securely past with multiple-pp
+headroom, the safest candidates for an immediate gate tighten.

hivegrid-cloud/docs/testing/coverage-sprint-backlog.md

@@ -0,0 +1,147 @@
+# Coverage Sprint Backlog — #76 Follow-up Work
+
+**Generated:** 2026-05-25
+**Baseline source:** [`coverage-baseline-2026-05-25.md`](./coverage-baseline-2026-05-25.md)
+**Current overall coverage:** 77.68% statements / 79.34% branches / 75.46% functions / 77.68% lines
+
+**2026-05-29 closeout check:** `pnpm run verify:test-coverage` passes with
+the current per-module gates. The historical baseline remains useful for
+trend context, but the active closeout state is now "no positive sprint
+gaps".
+
+**Rule:** Every entry in the "Critical-path positive-gap" table below must
+reach its `sprintTarget` before a future coverage ratchet can be treated as
+done. There are no positive-gap rows in the latest baseline, so #76 is closed;
+remaining work is gate maintenance and opportunistic threshold ratcheting
+rather than broad test writing.
+
+## Pick-the-next-module workflow
+
+For a follow-up session that wants to pick mechanically:
+
+1. Read this file.
+2. Grab the top row of the "Critical-path positive-gap" table.
+3. Open the HTML coverage report (`coverage/lcov-report/index.html`) and
+   drill into that module.
+4. Identify uncovered lines/branches; write tests against real code paths
+   (no mocks-only padding).
+5. Re-run `pnpm exec vitest run --coverage --exclude ...` (see baseline doc
+   for the exclude flags). Confirm achievement.
+6. Bump the entry in `scripts/coverage-targets.json` for that module to the
+   new achieved level (round DOWN by 0.5-1pp for headroom). Never weaken
+   existing thresholds.
+7. Commit with `test(<area>): close coverage gap for <module>`. Move the
+   row to the "Graduated" table below.
+
+## Critical-path positive-gap modules — priority order
+
+Sorted by sprint-priority (per detail-MD section d): Auth > Tenant-Isolation
+
+> Billing > Permissions > Entitlements > DB.
+
+No critical-path modules currently have a positive sprint gap.
+
+## Standard-tier modules with positive gap
+
+No standard-tier runtime modules currently have a positive sprint gap.
+
+## Graduated modules (already at or past sprintTarget)
+
+Where the gap is negative the module has met or exceeded the sprintTarget.
+Rows marked `(raised)` had their active gate raised on 2026-05-25 to lock
+the achievement in.
+
+| Module                                | stmt now | br now | sprint | Active gate now |
+| ------------------------------------- | -------: | -----: | -----: | --------------: |
+| `apps/api/src/auth/**` (raised)       |     99.2 |   92.2 |  90/85 |       **99/92** |
+| `apps/web/app/services/auth/**` (raised) |  97.4 |   88.5 |  90/85 |       **97/88** |
+| `apps/web/app/utils/session.server.ts` (raised) | 100.0 | 99.1 | 95/90 | **99/98** |
+| `apps/web/app/utils/auth/**` (raised) | 97.7 | 92.7 | 90/85 | **97/92** |
+| `apps/web/app/utils/stripe.server.ts` (raised) | 95.4 | 91.1 | 85/75 | **95/90** |
+| `apps/web/app/utils/db/tenants.db.server.ts` (raised) | 98.5 | 96.1 | 95/90 | **98/95** |
+| `apps/web/app/utils/db/permissions/**` (raised) | 99.8 | 99.0 | 90/80 | **99/98** |
+| `apps/api/src/billing/**` (raised)    |     99.3 |   98.0 |  99/95 |       **99/97** |
+| `packages/entitlements/**` (raised)   |     97.0 |   89.9 |  90/80 |       **96/89** |
+| `apps/api/src/isolation/**` (raised)  |    100.0 |  100.0 |  95/90 |       **99/99** |
+| `packages/db/src/outbox*` (raised)    |     93.0 |   88.6 |  90/80 |       **92/88** |
+| `packages/shared-store/**` (raised)   |     91.3 |   90.7 |  90/80 |       **91/90** |
+| `packages/error-reporter/**` (raised) |     81.5 |   86.9 |  80/70 |       **81/86** |
+| `apps/mobile/src/**` (raised)         |     79.2 |   80.2 |  70/60 |       **79/79** |
+| `apps/hocuspocus/src/**` (raised)     |     89.2 |   92.9 |  70/60 |       **88/90** |
+| `packages/cli/src/**` (raised)        |     80.4 |   83.6 |  80/70 |       **80/83** |
+| `packages/ui-app/src/**` (raised)     |     95.1 |   73.7 |  70/60 |       **94/73** |
+| `packages/api-client-ts/src/**` (raised) |  70.9 | 80.9 | 70/60 | **70/80** |
+| `packages/plugins/src/**` (raised) | 100.0 | 86.8 | 98/80 | **99/86** |
+| `packages/ai/src/**` (raised)         |     81.6 |   81.0 |  80/70 |       **81/81** |
+| `packages/config/src/**` (raised)     |     99.0 |   95.4 |  80/70 |       **98/95** |
+| `packages/feature-flags/src/**` (raised) |  94.3 |   86.4 |  80/70 |       **94/86** |
+| `packages/i18n/src/**` (raised)       |     95.2 |   75.7 |  80/70 |       **95/75** |
+| `packages/plugin-runtime/src/**` (raised) | 92.7 | 88.5 | 80/70 | **92/88** |
+| `packages/queue/src/**` (raised)      |     93.5 |   96.0 |  80/70 |       **93/95** |
+| `packages/exports/src/**` (raised)    |     90.7 |   83.6 |  80/70 |       **90/83** |
+| `packages/imports/src/**` (raised)    |     91.8 |   87.5 |  80/70 |       **91/87** |
+| `packages/realtime/src/**` (raised)   |     91.6 |   85.3 |  80/70 |       **91/85** |
+| `packages/search/src/**` (raised)     |     88.6 |   71.7 |  80/70 |       **88/71** |
+| `packages/storage/src/**` (raised)    |     82.8 |   84.2 |  80/70 |       **82/84** |
+| `packages/vector-store/src/**` (raised) |   94.6 |   90.4 |  80/70 |       **94/90** |
+| `packages/metrics/src/**` (raised)    |     98.9 |   90.2 |  80/70 |       **98/90** |
+| `packages/form-widget/src/**` (raised) |    96.7 |   86.5 |  80/70 |       **96/86** |
+| `packages/email-sequences/src/**` (raised) | 84.5 | 78.0 | 80/70 | **84/77** |
+| `packages/email-templates/src/**` (raised) | 96.9 | 86.5 | 70/60 | **96/86** |
+| `packages/observability/src/**` (raised) | 100.0 | 100.0 | 80/70 | **99/99** |
+| `packages/marketing-outbound/src/**` (raised) | 85.8 | 65.9 | 70/60 | **85/65** |
+| `packages/plugin-sdk/src/**` (raised) |    100.0 |  100.0 |  75/65 |       **99/99** |
+| `packages/ui-mock/src/**` (raised)    |     97.9 |   68.6 |  70/60 |       **97/68** |
+| `packages/view-models/src/**` (raised) |    86.7 |   71.4 |  75/65 |       **86/71** |
+| `packages/url/src/**` (raised)        |     98.3 |   95.9 |  80/70 |       **98/95** |
+| `packages/logger/src/**` (raised)     |    100.0 |  100.0 |  80/70 |       **99/99** |
+| `packages/brand/src/**` (raised)      |    100.0 |   93.8 |  80/70 |       **99/93** |
+| `packages/api-types/src/**` (type-only) |   0.0 | 100.0 | 0/0 | **0/0** |
+
+## Out-of-scope notes
+
+- `packages/ui-subframe/**` keeps an active 30/20 gate while its sprint
+  target remains 50/40 by design; visual confidence lives in Storybook
+  lost-pixel (#62). Do NOT force the active gate to 50/40 with shallow
+  unit tests; close that gap with the visual regression contract.
+- **The latest full coverage run puts
+  `apps/web/app/utils/stripe.server.ts` past both sprint targets**. The
+  active gate is raised to 95/90 so future changes cannot drop back to the
+  old mock-only baseline.
+- **The latest full coverage run graduates `packages/cli/src/**` at
+  80.4% statements / 83.6% branches**. The active gate is raised to 80/83
+  so the new AI-cost, feature-flag, chatbot, backup-create, DSGVO,
+  sequence, leads, and KB article coverage does not regress.
+- **The latest full coverage run graduates `packages/ui-app/src/**` at
+  95.1% statements / 73.7% branches**. The active gate is raised to 94/73
+  after covering the public a11y-template test factory.
+- **The latest full coverage run graduates `packages/api-client-ts/src/**`
+  at 70.9% statements / 80.9% branches**. The active gate is raised to
+  70/80 after covering generated body/path/query/auth/client/SSE helpers and
+  catching a cookie header separator regression.
+- **The latest full coverage run graduates `packages/plugins/src/**` at
+  100.0% statements / 86.8% branches**. The active gate is raised to 99/86
+  after adding branch-edge coverage for manifest schema and loader behavior.
+- `packages/api-types/src/**` reports 0% statements because the source
+  contains only `export type ... = ...` aliases (no runtime code). Its
+  sprint target is intentionally 0/0 and guarded by `expectTypeOf` tests plus
+  package typecheck instead of fake runtime assertions.
+- **The latest `pnpm run verify:test-coverage` execution had 399 passing
+  files, 2 skipped files, 3597 passing tests, and 50/50 passing coverage
+  targets**.
+- **2026-05-27 focused recon:** the four historical coverage-scope failures
+  called out in `coverage-baseline-2026-05-25.md` now pass when run together:
+  `apps/web/app/application/ui/uiInventory.test.ts`, `apps/api/src/app.test.ts`,
+  `packages/config/src/env-examples.test.ts`, and
+  `packages/cli/src/commands/tenant/json-codec.test.ts` (193 tests passing).
+- **2026-05-29 closeout verification:** `pnpm run verify:test-coverage`
+  passes with the current gate set; #76 is closed in
+  `docs/improvement-plan/PROGRESS.md`.
+
+## References
+
+- Spec: `docs/improvement-plan/10-compliance-polish/test-coverage.md`
+- Policy: `docs/architecture/test-coverage.md`
+- Baseline: `docs/testing/coverage-baseline-2026-05-25.md`
+- Gate runner: `scripts/verify-test-coverage.mjs`
+- Targets data: `scripts/coverage-targets.json`

hivegrid-cloud/docs/testing/e2e-readiness-handoff.md

@@ -0,0 +1,206 @@
+# E2E Readiness — Handoff (2026-06-01)
+
+Single-source pickup doc for the boiler e2e fix effort. Target: `boiler.saas-ctrl.com`
+(staging on Hetzner `staging-1`, 46.225.5.93). Companion memory files:
+`boiler-deploy-seed.md`, `boiler-e2e-remaining-tail.md` (in the agent memory dir).
+
+## Current status
+
+Full local run against boiler (`set -a; source .env.e2e-readiness.boiler; set +a; npx playwright test`):
+
+| Run | pass / fail / skip |
+|-----|--------------------|
+| baseline (start of effort) | 108 / 54 / 46 |
+| full-run-8 | 157 / 10 / 41 |
+| **full-run-9 (latest)** | **163 / 15 / 30** |
+
+Headline: **+55 pass vs baseline**. Fail rose 10→15 only because the workflow-execution
+fix (below) **un-skipped** ~11 previously-gated workflow tests — they now run and some
+fail on detail contracts. Not regressions.
+
+All code fixes are on `main` (latest pushed commit `379a5b25`). The hard infrastructure
+is solved; what remains are detail contracts + one real UI bug.
+
+### 2026-06-01 FINAL RESULT — merged to `main` and deployed to boiler
+
+Full single-worker run **with `--retries=2` (CI-equivalent) = 178 pass / 0 fail / 2 flaky
+(pass on retry) / 28 skip**. The 2 flaky are serial-only and pass in isolation + on retry:
+read-only plugin permissions (`app.plugins.use`) and a registration login-rate-limit test
+(10/60s). All real product bugs and test-contract issues below are fixed. Commits on `main`:
+`71686c25` (getWorkflowExecution 500→404), `ef7f4ee4` (run-dropdown Menu.Item Fragment 500),
+`626abd9d` (prune stale role→permission grants), `5261bf98` (z-50/CSRF/canvas), `87b997fd`
++ `f93ba964` + `8de012ef` (e2e test contracts). Re-seed after every deploy (`/tmp` ephemeral).
+
+### 2026-06-01 follow-up (investigation detail)
+
+Re-investigated the plugin-settings + workflow-run mobile-a11y specs against boiler with
+fresh re-seeds. The earlier read-only diagnosis was partly wrong; running with traces
+corrected it. Net: 8 of the 13 remaining were genuine **test-contract / single-fetch**
+bugs (now fixed + locally verified passing), 3 are **product** bugs (fixed in code, need a
+web deploy to validate), and 3 are **staging environment / RBAC / seed** issues (not code).
+
+**Fixed in tests — verified GREEN locally (no deploy needed):**
+- All RR7 single-fetch interceptors now match `<path>.data` and return a **turbo-stream**
+  body, not `application/json` (faked JSON breaks the decoder so `useActionData()` is empty
+  and the UI never renders). New helper `singleFetchActionData(payload, status?)` in
+  `tests/e2e/utils/navigation.ts` calls the real `turbo-stream` `encode({data})` (byte-
+  identical to the proven `singleFetchActionError`). Fixed: plugin consent/lifecycle/revoke/
+  stale-review (`capturePluginPost` + inline 409/400 interceptors; also matches the
+  `fetch(".")` parent-dir for the iframe write), and workflow manual/api/stream waiting-block
+  continuations + manual-error.
+- Stale-review (`:173`): the readiness plugin has **2+ missing capabilities** (audit + a hook
+  subscription); approval needs **every** checkbox confirmed. Test now checks all (mirrors
+  the consent test). Approve-button-disabled was the real blocker, not the interceptor.
+- Invalid-JSON (`:780`): Monaco auto-closed a typed `{` into a valid `{}` AND a bare wrapper
+  click left keystrokes unrouted. Fixed by focusing the `.monaco-editor` edit context, then
+  `insertText("not valid json")` (non-bracket, paste-like), asserting the model value.
+
+**Product bugs — fixed in code, NEED a web deploy + re-seed to validate:**
+- Run dropdown obstructed by ReactFlow (`:303`, was `:315`): toolbar lacked a stacking
+  context. Added `relative z-50` to the builder toolbar in `workflows.$id.index.view.tsx`.
+- Stream waiting-block continuation POST missing CSRF (`:567`): `WorkflowStreamProgress.tsx`
+  `onSubmitWaitingBlock` never called `appendCSRFToken(form)` (manual/api views do). Added it.
+- Execution-detail builder canvas hidden on mobile (`:617`): ReactFlow needs a *definite*
+  height; `workflows.$id.executions.view.tsx` used indefinite `min-h`/`flex-1`. Applied the
+  proven `h-[70vh] lg:h-auto lg:min-h-0 lg:flex-1` (same as the index view).
+
+**Staging environment / RBAC / seed — NOT code bugs (CI lane is the arbiter):**
+- `:175` builder "stays usable" → e2e tenant user is redirected to
+  `/unauthorized/app.settings.roles.set/<tenant>` (HTTP 200, so `skipIfUnavailable` doesn't
+  skip). The seeded user is missing role permissions on staging (the "roles flag seed" gap).
+- `:461` settings iframe → the readiness plugin is **Disabled** (state drifted across the
+  serial run; the settings detail route is correctly unreachable while disabled). Needs
+  per-test enable/restore or a `beforeEach` re-seed of the plugin's enabled state.
+- `:669` read-only user sees mutation controls → the `User` role carries a **stale**
+  `app.plugins.manage` grant. `syncDefaultRolesAndPermissions` is **add-only** (never prunes
+  role→permission links), so the 2026-05-26 plugin-permission split never propagated to
+  tenants seeded before it. Immediate: prune the stale grant on staging. Durable: make the
+  sync prune links no longer in `defaultPermissionSeedDescriptors[].inRoles`.
+
+## What was fixed this effort
+
+### Real product bugs
+1. **Outbox BullMQ jobId contained `:`** (`2b6ff2bf`). `jobId: \`outbox:${id}\`` — BullMQ
+   rejects `:` ("Custom Id cannot contain :"), so EVERY async outbox dispatch failed and
+   dead-lettered (workflow.execution.requested, workflow.trigger.event, plugin.hook.*,
+   audit.event.create). Unit tests mocked `queue.add` so it never surfaced. Fixed to `-`
+   in `apps/worker/src/outbox/bootstrap.ts` AND `workflow-execution-stalled-reconciler.ts`
+   `buildOutboxJobId` (must agree). `outbox-receiver.ts` idempotency key keeps `:` (not a
+   BullMQ id). This was breaking ALL async delivery in production, not just tests.
+2. **Language switch was a no-op** (earlier, root.tsx) — setLocale didn't write the `_i18n`
+   cookie remixI18Next.getLocale reads.
+3. Misc a11y/contrast cluster (rose-500→600, focus-return, etc.) — see git log.
+
+### Coolify config changes (staging only — applied via API, NOT in repo)
+The workflow-execution + plugin tests were blocked by deployment config, not code:
+- **worker-general `WORKFLOW_TRIGGER_EXECUTOR_MODE=native`** (was `http`). In http mode the
+  worker POSTs to the web's `/internal/api/workflows/executions/<id>/start`, but that route
+  is **deliberately** gated out of `routes.ts` (`internal/api/__workflows/**`, enforced by a
+  test — do NOT un-gate it). native mode runs safe-core workflows (manual/log/if/switch/
+  httpRequest/email/rowGet…) directly in the worker via prisma. This made workflow execution
+  work (the outbox-drain canary goes green).
+- **`PLUGIN_BINARY_BASE_PATH=/tmp/e2e-plugins`** on boilerplate-web + worker-general. The
+  image default `/app/var/plugins` doesn't exist and is unwritable for uid 1000 `node`; the
+  seed writes the plugin bundle to `/tmp/e2e-plugins`. Set this so the web reads it. Made the
+  plugin settings iframe render.
+- **boilerplate-web `custom_network_aliases=boilerplate-web`** so the cluster-internal URL
+  `http://boilerplate-web:8080` resolves (the public fqdn returns curl `000` from inside —
+  hairpin NAT). (Only needed if you ever switch the worker back to http mode.)
+
+### Test fixes (on main)
+- `tests/e2e/utils/navigation.ts` `singleFetchActionError()` — builds the real RR7 single-fetch
+  turbo-stream error envelope (`text/x-script` + body `[{"_1":2},"data",{"_3":4},"error","msg"]\n`
+  + header **`X-Remix-Response: yes`** — without that header the client renders a 500 boundary).
+- `tests/e2e/utils/axe.ts` — `settleAnimations()` waits on Web Animations `finished` before
+  scanning (kills transient color-contrast flakes); filters `scrollable-region-focusable` on
+  `role="menu"` (Radix false positive).
+- input-editor 320px: focus Monaco edit context + assert the editor MODEL value (DOM is
+  virtualised). executions-mobile: assert the `.filter({visible:true})` Executions label.
+  readinessPluginId passed into the iframe `evaluate`. signout race, portals skip, drawer reopen.
+
+## Coolify / infra access (found in the `infra` repo, ~/Documents/GitHub/infra)
+
+- API: `https://saas-ctrl.com/api/v1`, Bearer = `~/.cli-tokens/coolify-token` (works).
+- App UUIDs: web `mrf8lj70kmy9hqprx9dd6w7q`, api `e109dzc3xzjxi54llfje4tob`,
+  worker-general `wzshv933n2nhy0cvsxu8bqmm` (drains outbox + runs workflow executor),
+  worker-plugin `o10eqbjd7hd49d2iuyjataqo`, -import/-mail/-webhook.
+- Env API gotcha: `POST /applications/{uuid}/envs {key,value}` and PATCH-by-key BOTH create →
+  dupes. Always GET uuids and `DELETE /applications/{uuid}/envs/{env_uuid}` extras.
+- Redeploy: `GET /applications/{uuid}/deploy` (add `?force=true` to re-pull image).
+- SSH host: `boilerplate-staging` (root@46.225.5.93, key `~/.ssh/node1`).
+- DB query: throwaway `postgres:16-alpine` on the `coolify` docker network; strip
+  `?pgbouncer=true` from `$DATABASE_URL`; columns are quoted camelCase.
+
+## Re-seed (REQUIRED after every web deploy — /tmp is ephemeral)
+```
+WEB=$(ssh boilerplate-staging "docker ps --format '{{.Names}}\t{{.Image}}' | grep /web: | cut -f1 | head -1")
+ssh boilerplate-staging "docker exec -w /tmp -e E2E_SEED_ALLOW_PRODUCTION=1 -e E2E_LOCAL_READINESS_SEED=1 \
+  -e PLUGIN_BINARY_BASE_PATH=/tmp/e2e-plugins $WEB node /myapp/packages/db/dist/e2e-readiness-seed.server.js"
+```
+
+## Remaining 15 failures — categorized with pointers
+
+### A. Workflow run-detail (7) — newly running after the execution fix
+File `tests/e2e/workflow-builder-run.mobile-a11y.spec.ts`. The execution CORE works (drain
+canary + execution-status + status-toggle-contract pass with `E2E_REQUIRE_WORKFLOW_OUTBOX_DRAIN=1`).
+These fail on continuation/error contracts — likely 1-2 shared root causes in the run flow:
+- `:462` manual waiting-block continuation posts raw input
+- `:539` API waiting-block continuation posts JSON input
+- `:579` stream run waiting-block continuation
+- `:629` manual run result deep-links to execution details (gated by E2E opt-in)
+- `:751` manual run remains on-page when execution action returns an error
+- `:796` manual run rejects invalid JSON through the real action
+- `:900` keeps builder and manual run reachable at 320px
+START HERE: run one (e.g. `:796`) with `E2E_REQUIRE_WORKFLOW_OUTBOX_DRAIN=1`, read the failure;
+the waiting-block ones share `continueExecution`/wait-for-input handling. Native executor
+supports a subset of blocks (see `docs/worker.md`) — confirm wf_waiting_123's blocks are in it,
+else native fails-loud on unsupported graph semantics (would explain waiting-block fails).
+
+### B. Plugin-settings POST/capability contracts (5)
+File `tests/e2e/plugin-settings.mobile-a11y.spec.ts`. The iframe now RENDERS (binary path fixed).
+- `:49` capability consent review — "capability grant POST captured: null" (the grant POST
+  never fires → needs the capability in a pending-review state in the seed, or the approve
+  control isn't rendering).
+- `:173` stale capability review error
+- `:332` lifecycle controls non-mutating interception contract
+- `:461` settings iframe sandbox/CSP — readinessPluginId test bug fixed; now fails on the
+  settings-write POST not being captured (line ~546) — likely needs `E2E_ALLOW_PLUGIN_SETTINGS_WRITE`
+  opt-in OR the parent doesn't forward the iframe message (channelId/pluginId validation).
+- `:617` capability revoke controls non-mutating form contract
+START HERE: these likely share a capability-grant/seed-state root. Check what capability state
+the seed creates vs what the consent-review UI needs (a pending grant request).
+
+### C. Real UI bug (1) — run dropdown obstructed by ReactFlow
+`tests/e2e/workflow-builder-run.mobile-a11y.spec.ts:315` "run dropdown stays keyboard reachable".
+Diagnosed: the `ui.workflows.run.dropdown.button` (HeadlessUI Menu.Button in
+`apps/web/app/modules/workflowEngine/components/workflows/buttons/WorkflowRunDropdown.tsx`) is
+visible but `click()` TIMES OUT and `aria-expanded` stays false even on a DOM click — the
+ReactFlow canvas overlays it (captures pointer + keyboard) on the builder page at mobile width.
+This is a genuine product bug (z-index / pointer-events / layout on the builder), needs a fix +
+web deploy. NOT a test bug.
+
+### D. Test-isolation (1) — status toggle
+`tests/e2e/workflow-builder-actions.spec.ts:155` "persists workflow status toggles". The toggle
+now PERSISTS (native mode made the status mutation work), so the persisted state leaks between
+runs and the test's read-initial/restore logic races (error flips between "expected true got
+false" and vice-versa across runs). Fix: make the test reset wf_123 status deterministically, or
+the seed should pin a known status each run.
+
+### E. Cross-suite flake (1)
+"guards seeded tenant account membership before dashboard" — passes in isolation AND its own
+full file; only fails in the full serial run (cumulative login rate-limit, 10/60s). CI lane's
+retries absorb it. Not a real fail.
+
+## How to run / verify
+- Full local: `cd boilerplate; set -a; source .env.e2e-readiness.boiler; set +a; npx playwright test --reporter=json > /tmp/run.json`
+- Workflow execution tests need `export E2E_REQUIRE_WORKFLOW_OUTBOX_DRAIN=1`.
+- The authoritative "everything on" run is the CI readiness lane (`.gitea/workflows/e2e-readiness.yml`,
+  `workflow_dispatch` with the operator opt-ins: workflow_outbox_drain, plugin_settings_write,
+  plugin_lifecycle_mutation) which sets `REQUIRE_E2E_READINESS=1` (fail-closed).
+
+## Do-not-repeat notes
+- Do NOT un-gate `internal/api/__workflows/**` in routes.ts (a test enforces the exclusion; the
+  intended path is native executor mode).
+- Do NOT use a global `<style>` to freeze CSS transitions in the axe helper — it perturbs later
+  Radix close-animation focus timing. Use the Web Animations `finished` wait (already in place).
+- Coolify env POST/PATCH both CREATE — always dedupe by deleting extra env uuids.

hivegrid-cloud/docs/testing/e2e-readiness.md

@@ -0,0 +1,153 @@
+# E2E Readiness Fixtures
+
+Use the local readiness seed when a fresh database should exercise the authenticated Playwright suite instead of silently skipping plugin, workflow, billing, and dashboard paths. The same fail-closed contract is wired into `.github/workflows/e2e-readiness.yml` and `.gitea/workflows/e2e-readiness.yml` for scheduled or manually dispatched staging/readiness runs.
+
+This is a local test-readiness runbook. It does not replace the manual,
+redacted staging evidence contract in [Staging, Live, and UAT Readiness](../staging-live-uat.md), and it does not mark the plugin runtime production-ready before the external security sign-off described in [Plugins](../features/plugins.md).
+These fixtures prove local authenticated browser/a11y coverage for the named UI
+paths; they do not prove production readiness for plugin execution, workflow
+workers, provider delivery, staging UAT, or operator-owned infrastructure.
+Run the browser command below only as an operator-approved local/readiness
+browser run; it is intentionally not part of automatic docs/static readiness
+verification.
+If `E2E_BASE_URL` is unset, empty, or only whitespace, the Playwright wrapper
+probes the local default `http://localhost:3000`.
+
+```bash
+E2E_LOCAL_READINESS_SEED=1 pnpm run seed:e2e-readiness
+set -a; source .env.e2e-readiness; set +a
+pnpm run verify:a11y-coverage
+REQUIRE_E2E_A11Y=1 REQUIRE_AUTH_E2E=1 REQUIRE_E2E_READINESS=1 E2E_A11Y_STRICT=1 E2E_BROWSER_MATRIX=1 E2E_ALLOW_WORKFLOW_RUN=1 pnpm run verify:e2e-a11y
+```
+
+The seed is idempotent, refuses `NODE_ENV=production`, and writes `.env.e2e-readiness` with deterministic credentials, `E2E_NO_TENANT_EMAIL` / `E2E_NO_TENANT_PASSWORD` for the disposable no-tenant account-creation flow, `E2E_PLUGIN_READONLY_EMAIL` / `E2E_PLUGIN_READONLY_PASSWORD`, account slug/name, tenant dashboard/settings/profile paths, governance/policies path, API overview/keys/docs/logs paths, admin email sequence path, plugin settings path, workflow builder/API-run/manual-run paths, a separate webhook workflow builder/manual-run path, and a waiting-workflow execution path. The seeded workflow ids are fixed (`wf_123`, `wf_webhook_123`, and `wf_waiting_123`) so the local env file and scheduled `.github`/`.gitea` readiness workflows target the same staging paths. The seed also creates one successful manual execution (`wfe_123`) with a successful log block run (`wfbe_log_123`) so fresh strict-readiness databases can render workflow execution list/detail mobile and a11y checks before any mutating run drains through a worker. It also creates a paused wait-for-input execution (`wfe_waiting_123`) so waiting-state execution details and continuation form contracts can be inspected deterministically without claiming that the worker completed the continuation. The no-tenant fixture user is reset with no default tenant, no memberships, and no roles so `E2E_ALLOW_NO_TENANT_ACCOUNT_CREATE=1` exercises the real account onboarding path instead of reusing the seeded tenant. The read-only plugin user joins the seeded tenant, has that tenant as its default tenant, and is reset to only the default `User` app role, which includes `app.plugins.use` but not `app.plugins.manage`. It also writes `E2E_ALLOW_WORKFLOW_WEBHOOK_RUN=0` plus an empty `E2E_WORKFLOW_WEBHOOK_TARGET_URL` so a sourced readiness env never accidentally calls an external endpoint. It also writes `E2E_REQUIRE_WORKFLOW_OUTBOX_DRAIN=0`, `E2E_WORKFLOW_OUTBOX_DRAIN_EXPECTED_STATUS=success`, `E2E_WORKFLOW_OUTBOX_DRAIN_ASSERTION=operator-gated`, and an operator-gated readiness note because successful workflow execution requires the general worker/outbox path to be running. It also seeds one active owned subscription plan for billing/cancellation checks and a second public unowned upgrade plan so checkout-catalog smokes always have an enabled submit button without mutating the active subscription. It also seeds deterministic admin email sequences, one non-secret API key, and one `/v1/e2e-readiness` API log row so authenticated API overview, docs, logs, governance, email-sequence dashboard, and key-management mobile/a11y checks can run without production telemetry. The default manual workflow is log-only and enables `E2E_ALLOW_WORKFLOW_RUN=1`; the API-run page stays form-contract tested without invoking the backend API directly, and the webhook workflow stays contract-tested by default so it can validate HTTP/Webhook payload wiring without external secrets. The plugin fixture intentionally has one granted capability and one missing capability so consent, revoke, manifest, and settings iframe checks have meaningful state.
+
+Workflow worker/outbox-drain readiness is now available as an explicit operator-gated browser probe. Keep `E2E_REQUIRE_WORKFLOW_OUTBOX_DRAIN` unset or `0` for normal local, staging, and scheduled readiness runs unless the general worker is running and subscribed to the workflow/outbox path. When an operator sets `E2E_REQUIRE_WORKFLOW_OUTBOX_DRAIN=1`, the workflow mobile/a11y spec requires seeded tenant credentials plus `E2E_WORKFLOW_RUN_MANUAL_PATH`, submits the log-only manual workflow, follows the returned execution details link, and polls until the execution reaches `E2E_WORKFLOW_OUTBOX_DRAIN_EXPECTED_STATUS` (`success` by default). This probe intentionally waits for the domain `WorkflowExecution`, not merely for the `OutboxEvent` row to become `DONE`: outbox `DONE` only proves handoff to BullMQ, while `outbox.workflow` job state and `WorkflowExecution.status` prove whether the worker bridge actually ran the workflow. A drained execution that ends in `error` is a failed readiness signal unless an operator explicitly sets `E2E_WORKFLOW_OUTBOX_DRAIN_EXPECTED_STATUS=error` for a negative-path probe. Use `E2E_WORKFLOW_OUTBOX_DRAIN_TIMEOUT_MS` to tune the default 90-second poll window for slower readiness environments.
+
+Plugin `workflow.executed` hook readiness is a separate operator-gated browser probe layered on top of the workflow outbox drain. Keep `E2E_REQUIRE_PLUGIN_HOOK_AUDIT` unset or `0` for normal local, staging, and scheduled readiness runs; the plugin hook lifecycle spec then only exercises the non-mutating audit-trail view and the mutation-gated deactivate→re-enable lifecycle flow. Setting `E2E_REQUIRE_PLUGIN_HOOK_AUDIT=1` requires a running general worker, the plugin runtime, and outbox drain, so it reuses the same workflow outbox-drain readiness gate (it internally calls `requireWorkflowOutboxDrainReadiness`, which still needs `E2E_REQUIRE_WORKFLOW_OUTBOX_DRAIN=1`). When enabled, the spec requires `E2E_TENANT_EMAIL`, `E2E_TENANT_PASSWORD`, `E2E_PLUGIN_SETTINGS_PATH`, and `E2E_WORKFLOW_RUN_MANUAL_PATH`; any missing value fails with a clear configuration error instead of skipping. The probe grants the seeded readiness plugin's `audit` capability (mutation-gated additionally behind `E2E_ALLOW_PLUGIN_LIFECYCLE_MUTATION=1`), runs the seeded log-only manual workflow to fire `workflow.executed`, waits for the worker to drain the outbox and the execution to terminate, then polls the plugin capability-use audit trail until a success row appears for the hook-driven `audit`/`log` capability. It restores the seeded baseline (revoking the `audit` grant; re-running the readiness seed also resets capability state) so reruns stay idempotent. Use `E2E_PLUGIN_HOOK_AUDIT_TIMEOUT_MS` to tune the default 90-second audit poll window for slower readiness environments. This operator opt-in is owned by the readiness lane, not the seed, so the seed's `buildE2eReadinessEnv` does not emit `E2E_REQUIRE_PLUGIN_HOOK_AUDIT` or `E2E_PLUGIN_HOOK_AUDIT_TIMEOUT_MS`.
+
+Real webhook/HTTP execution is a separate opt-in. Keep `E2E_ALLOW_WORKFLOW_WEBHOOK_RUN` unset or `0` for normal local, staging, and scheduled readiness runs; the Playwright spec intercepts the form POST and verifies the payload contract only. For a disposable operator-approved run that may call an HTTP endpoint, set both `E2E_ALLOW_WORKFLOW_WEBHOOK_RUN=1` and `E2E_WORKFLOW_WEBHOOK_TARGET_URL=<operator-owned-url>` after sourcing `.env.e2e-readiness`. When that opt-in is enabled, the wrapper and spec require `E2E_TENANT_EMAIL`, `E2E_TENANT_PASSWORD`, `E2E_WORKFLOW_WEBHOOK_BUILDER_PATH`, `E2E_WORKFLOW_WEBHOOK_RUN_MANUAL_PATH`, and `E2E_WORKFLOW_WEBHOOK_TARGET_URL`; any missing value fails with a clear configuration error instead of skipping.
+
+The scheduled GitHub/Gitea readiness workflows keep those operator-owned
+probes off by default. Manual `workflow_dispatch` exposes explicit inputs for
+plugin lifecycle mutation, plugin settings writes, workflow outbox drain, and
+real webhook execution; the wrapper fails before Playwright starts when a
+selected input is missing its required path, credential, expected status, or
+target URL contract.
+
+Mutating provisioning checks stay opt-in and are separate from normal strict
+readiness. Set `REQUIRE_E2E_MUTATING_READINESS=1` together with the relevant
+`E2E_ALLOW_*` opt-ins only for disposable local/readiness databases where the
+suite may create a new user, tenant, no-tenant account, or mutate plugin state.
+Keep those values unset for non-mutating staging smoke runs.
+
+For a disposable local database where all mutating readiness flows are
+operator-approved, use the named wrapper so every required opt-in stays visible
+and versioned together:
+
+```bash
+E2E_LOCAL_READINESS_SEED=1 pnpm run seed:e2e-readiness
+set -a; source .env.e2e-readiness; set +a
+pnpm run verify:e2e-mutating-readiness:local
+```
+
+`verify:e2e-mutating-readiness:local` sets
+`REQUIRE_E2E_MUTATING_READINESS=1`,
+`E2E_ALLOW_REGISTRATION_CREATE=1`, `E2E_ALLOW_ACCOUNT_CREATE=1`,
+`E2E_ALLOW_NO_TENANT_ACCOUNT_CREATE=1`,
+`E2E_ALLOW_PLUGIN_LIFECYCLE_MUTATION=1`, and
+`E2E_ALLOW_PLUGIN_SETTINGS_WRITE=1`. Do not use it against shared staging or
+production databases.
+Registration provisioning has two explicit branches: keep
+`E2E_REGISTRATION_EXPECTS_EMAIL_VERIFICATION=0` when the lane must prove that a
+new user reaches a tenant dashboard and can log back in; set it to `1` only for
+a disposable verification-required lane that proves signup stops at the email
+verification confirmation instead of entering the app shell.
+When `E2E_ALLOW_NO_TENANT_ACCOUNT_CREATE=1` is set, the wrapper also fails
+before Playwright starts unless `E2E_NO_TENANT_EMAIL` and
+`E2E_NO_TENANT_PASSWORD` are present. The local readiness seed writes those
+values, so the named wrapper should be run after sourcing `.env.e2e-readiness`.
+
+When the local general worker is also running and subscribed to the workflow
+outbox path, use the worker-drain wrapper instead:
+
+```bash
+E2E_LOCAL_READINESS_SEED=1 pnpm run seed:e2e-readiness
+set -a; source .env.e2e-readiness; set +a
+pnpm run verify:e2e-mutating-readiness:worker-local
+```
+
+`verify:e2e-mutating-readiness:worker-local` delegates to the disposable local
+mutating readiness lane and additionally sets
+`E2E_REQUIRE_WORKFLOW_OUTBOX_DRAIN=1`. That makes the browser workflow run poll
+the resulting `WorkflowExecution.status` until it reaches
+`E2E_WORKFLOW_OUTBOX_DRAIN_EXPECTED_STATUS` (`success` by default), so the lane
+proves the worker bridge drained the workflow instead of only proving that the
+web runtime accepted a manual run request. Do not use it against shared staging
+or production databases.
+
+Plugin lifecycle persistence checks also stay opt-in. Default plugin settings
+readiness verifies mobile/a11y, consent, revoke forms, lifecycle forms,
+manifest disclosure, and settings iframe contracts without changing the seeded
+plugin state. Set `E2E_ALLOW_PLUGIN_LIFECYCLE_MUTATION=1` only for disposable
+local/readiness databases where the suite may persist capability grant/revoke
+and plugin enable/disable state, then leave it unset for non-mutating staging
+smoke runs.
+
+`REQUIRE_E2E_READINESS=1` intentionally requires the mobile browser matrix, strict a11y mode, real log-only workflow-run opt-in, and every seeded path or credential consumed by the non-mutating strict specs: dashboard, account/profile settings, governance/policies, subscription, API overview/keys/docs/logs, portal analytics/settings, admin email sequences, plugin settings, read-only plugin credentials, workflow engine index, workflow builder/API-run/manual-run, webhook builder/manual-run, and the local plugin binary base path. That keeps staging/readiness runs from silently exercising only the desktop Chromium lane, skipping account/profile continuity, API docs/logs, portal management coverage, plugin permission coverage, workflow engine coverage, email-sequence dashboard coverage, or governance coverage, or ignoring moderate accessibility regressions. It does not require `E2E_ALLOW_WORKFLOW_WEBHOOK_RUN=1`; real webhook delivery stays operator-owned.
+
+Subscription checkout success remains a separate strict opt-in because it
+requires an externally acquired Stripe checkout session return path. Normal
+readiness runs stay non-mutating and do not require
+`E2E_TENANT_SUBSCRIPTION_SUCCESS_PATH`. In an operator-approved lane that has a
+valid success return URL, enable the manual `subscription_success` dispatch
+input and provide `subscription_success_path`; the workflow exports
+`E2E_REQUIRE_SUBSCRIPTION_SUCCESS=1` together with
+`E2E_TENANT_SUBSCRIPTION_SUCCESS_PATH`. The wrapper fails before Playwright
+starts if that path is missing, and the subscription success smoke fails
+instead of silently skipping if the configured path is unavailable or resolves
+to an invalid checkout session. The same `E2E_REQUIRE_SUBSCRIPTION_SUCCESS`
+lane now also verifies reconciliation followthrough: after the checkout return
+renders, it follows the dashboard link, opens the subscription settings page,
+and asserts the reconciled active-subscription state survives both a fresh
+page reload and a full logout plus re-login (a fresh appData load). Because
+reconciliation is loader-side (pull-based), the reconciled state is read back
+through the real subscription settings loader rather than the success-page
+render alone, and the flow only navigates app routes (dashboard, settings,
+login/logout) so it never triggers a real Stripe network call. This stays a
+strict opt-in for the same external-blocker reason: it needs a real Stripe
+checkout return path, so the lane skips when `E2E_REQUIRE_SUBSCRIPTION_SUCCESS`
+is off and fails loudly when it is required but `E2E_TENANT_SUBSCRIPTION_SUCCESS_PATH`
+or tenant credentials are missing.
+
+The static a11y coverage gate also enforces named browser readiness surfaces,
+not only raw file counts. `pnpm run verify:a11y-coverage` must keep explicit
+coverage for registration-to-dashboard, plugin settings, tenant API docs/logs,
+tenant API keys, audit logs, audit export, privacy self-service, tenant members,
+tenant notification settings, tenant chatbot settings, AI usage, workflow
+builder/run, portal management, admin email sequences, subscription smoke, authenticated mobile shell,
+authenticated mobile locale persistence, public auth, and auth-locale flows. The required
+files must contain the expected guard calls, not only exist: mobile/readiness
+flows need `expectNoHorizontalOverflow`, and critical user surfaces also need
+`expectNoCriticalA11yViolations`. The authenticated mobile locale flow must
+switch locales, reload the dashboard, navigate through the mobile shell to
+profile settings, and restore the original locale in `finally`.
+Registration/account
+readiness also keeps a non-mutating mobile login redirect replay for
+`/new-account`, verifies the account creation form contract is present, and
+returns to the seeded dashboard without creating another tenant. The public
+mobile surface is part of that named coverage: landing, legal, error,
+feature-disabled, and footer selector flows must cover narrow and 320px compact
+viewports, no horizontal overflow, and critical axe checks.
+
+The scheduled readiness workflows intentionally stay separate from main CI. They
+install Chromium and WebKit, enable the mobile browser matrix, require staging
+credentials via `E2E_TENANT_EMAIL` / `E2E_TENANT_PASSWORD`, require a read-only
+plugin user via `E2E_PLUGIN_READONLY_EMAIL` /
+`E2E_PLUGIN_READONLY_PASSWORD`, and run
+`pnpm run verify:e2e-a11y` with the non-mutating readiness `REQUIRE_*` flags set. Missing credentials,
+missing seeded paths, missing browsers, unreachable staging, or moderate
+accessibility issues fail the lane instead of producing a SKIP.
+Do not set `SKIP_E2E_A11Y=1` in required readiness, auth, mutating readiness,
+or subscription-success lanes; the wrapper treats that combination as a
+configuration failure.

hivegrid-cloud/docs/testing/e2e-skipped-tests-loop-handoff.md

@@ -0,0 +1,586 @@
+# E2E Skipped-Tests Loop — Handoff for the next agent
+
+## ✅ GREEN — 4th agent (2026-06-02 ~20:15 UTC)
+
+**Full "everything-on" chromium run against boiler (web image `944ef476`, freshly
+reseeded): `203 passed / 0 failed / 5 skipped` (9.3m).** The 5 skips are correctly
+operator-gated and justifiably impossible here: real-Stripe `subscription-success`
+×3 (need an externally-acquired Stripe session), the registration email-verification
+inverse pair (env runs with `E2E_REGISTRATION_EXPECTS_EMAIL_VERIFICATION=0`), and the
+`E2E_REQUIRE_PLUGIN_HOOK_AUDIT` hook-audit lane (operator-gated). A second clean
+full run was used to confirm stability.
+
+**The 12 remaining failures from the previous handoff, all fixed this session:**
+1. **plugin cluster (49/120/173/201)** — cross-spec pollution. `plugin-hook-lifecycle`
+   now targets the dedicated, fully-provisioned `com.toolctrl.e2e-lifecycle` plugin
+   instead of mutating the shared `e2e-readiness` plugin's audit-missing baseline.
+2. **plugin-settings 244** — the grant/revoke + restore reloaded the page before the
+   in-flight single-fetch POST landed (stale read). Now waits for RR7 revalidation.
+3. **plugin-settings 587** — TWO bugs. (a) PRODUCT: the settings-iframe write posted to
+   `fetch(".")` (resolves to the parent plugins-list route) and expected raw JSON;
+   RR7 returns a turbo-stream document for non-`.data` fetches, so every write failed.
+   Fixed to post the detail route's `.data` endpoint and gate on the HTTP status.
+   (b) TEST: the post-reload iframe `get` is non-deterministic (the host frame
+   quarantines its window + rotates the channel on a late `load`), so persistence is
+   now verified by reading the settings loader's `.data` payload after reload.
+4. **admin-email-sequences ×2** — logged in as the shared tenant user (not an admin).
+   Seed now provisions a dedicated `e2e.admin@example.test` (AdminUser + global
+   SuperAdmin, no TOTP) and emits `E2E_ADMIN_EMAIL/PASSWORD`; spec logs in as it.
+5. **mobile-shell 570 (run/manual SSR 500)** — `MonacoEditor` mounted its browser-only
+   `./MonacoEditor.client` during SSR, so the streaming renderer set HTTP 500 on EVERY
+   editor page (run/manual, webhook-run, **api-docs too** — the prior handoff was wrong
+   that api-docs was fine; only this test asserts `status < 500`). Fixed by gating the
+   editor behind a client mount (server + first client render emit an identical
+   placeholder → no hydration mismatch).
+6. **portal-management** — operator chose to SHIP portals. Un-ignored the portal routes
+   (`app.$tenant/__portals/**`, `admin/portals.tsx`), gated the global enable behind
+   `PORTALS_ENABLED` (boilerplate default stays off; **boiler Coolify env set to
+   `true`**), seeded a minimal published Portal fixture (subdomain `e2e-readiness`),
+   and fixed 2 pre-existing type-drift errors the route-tree typecheck surfaced.
+7. **subscription:394, workflow-builder-run 528/816** — were timing flakes; green on
+   clean reseeded runs (CI `retries:1` absorbs them).
+
+**Commits (all on `main`, deployed):** `44e30fb3` (plugin fixture sep + races),
+`147d92e4` (plugin settings .data fix), `011c6958` (admin user), `13edf803`+`944ef476`
+(portals + type fixes), the Monaco fix, and `a9556041` (587 loader read-back, test-only).
+**Coolify (boiler web):** added `PORTALS_ENABLED=true` (single row — deduped).
+
+**Gotchas observed:** (a) a local-machine `net::ERR_INTERNET_DISCONNECTED` mid-run
+cascades ~70 false failures from one point onward — re-run, it's not real. (b) loader
+`.data` payloads embed the iframe HTML (~25KB) — search the FULL body, not a prefix.
+(c) the plugin settings iframe quarantines its window on late `load`s; drive it via the
+loader, not a post-reload `get`.
+
+---
+
+**Written:** 2026-06-02 00:10 UTC · **Repo HEAD at handoff:** `494f52de` (branch `main`,
+also deployed to boiler staging) · **Companion docs:** `docs/testing/e2e-readiness-handoff.md`
+(full bug-fix history), agent memory `boiler-e2e-remaining-tail.md`, `boiler-deploy-seed.md`,
+`e2e-singlefetch-interceptor-contract.md`.
+
+---
+
+## 🚀 START HERE — NEXT AGENT (handoff from the 3rd agent, 2026-06-02 ~16:35 UTC)
+
+**Current state:** `main` HEAD `d8aaeacb`, deployed to boiler (web image `3cbb12d3`; the two
+commits after it are test-only/docs and don't change the runtime image). Full chromium run, freshly
+reseeded: **192 pass / 11 fail / 5 skip**. The original 26 failures are mostly fixed; the 11 are the
+hard tail below. The 5 skips are correctly-gated (real-Stripe success + email-verification inverse).
+
+**What's already done (don't redo):** subscription/api-docs/golden-path/registration/workflow-builder-actions/
+mobile-shell-drawer race+selector fixes; seed prunes leaked accounts; seed fully-provisions the
+LIFECYCLE plugin; seed DELETES any operator-admin grant from the tenant user; host-aware admin path
+`/email-sequences`. **Coolify env on boiler (set this session, persist):**
+`PLUGIN_RUNTIME_PRODUCTION_ENABLED=true` (web + worker-general + worker-plugin) and
+`PLUGIN_HOOK_FANOUT_REQUIRED=true` (worker-general + worker-plugin). Coolify API:
+`https://saas-ctrl.com/api/v1`, token `~/.cli-tokens/coolify-token`, web uuid
+`mrf8lj70kmy9hqprx9dd6w7q` (see `boiler-deploy-seed.md`).
+
+**How to run (everything-on):** `bash /tmp/e2e-cr.sh <specs?>` — it sources `.env.e2e-readiness.boiler`
+and sets every opt-in + `REQUIRE_E2E_READINESS=1` (recreate it if /tmp was wiped; see §2/§4). **RESEED
+before every run** (after every deploy too — /tmp plugin bundles are ephemeral):
+`WEB=$(ssh boilerplate-staging "docker ps --format '{{.Names}}' | grep mrf8lj | head -1"); ssh boilerplate-staging "docker exec -w /tmp -e E2E_SEED_ALLOW_PRODUCTION=1 -e E2E_LOCAL_READINESS_SEED=1 -e PLUGIN_BINARY_BASE_PATH=/tmp/e2e-plugins $WEB node /myapp/packages/db/dist/e2e-readiness-seed.server.js"`.
+Seed source is `scripts/seed-e2e-readiness.ts`, built to that dist on deploy → **seed changes need a
+deploy** (push main → ~20min; watch the web image tag flip, NOT the commit-status API). DB inspection
+is now permitted (recipe in §4); you can also just psql the readiness plugin/grant state.
+
+**The 11 remaining, in priority order (full root-cause + fixes in the "third agent" ITERATION LOG below):**
+
+1. **[BIGGEST WIN — ~6 tests] Plugin cluster needs FIXTURE SEPARATION.** plugin-settings is **14/0 in
+   isolation** but fails ~5 in the serial run as CROSS-SPEC POLLUTION: `plugin-hook-lifecycle.*` sorts
+   first, mutates the SHARED `com.toolctrl.e2e-readiness` plugin (grants audit / deactivates), and the
+   "enabled + audit-missing" baseline is UI-unrestorable (revoke auto-disables; `plugins.tsx:415`
+   hides the enable form for disabled+missing-cap). **Fix:** seed a SECOND, fully-provisioned readiness
+   plugin WITH a settings UI (clone `buildE2eReadinessPluginManifest` → new id e.g.
+   `com.toolctrl.e2e-hooklifecycle`, grant ALL caps incl `hook:workflow.executed`), and repoint
+   `plugin-hook-lifecycle.mobile-a11y.spec.ts` (146/201) + the lifecycle/iframe tests at it; keep the
+   audit-missing `com.toolctrl.e2e-readiness` ONLY for the review/grant/deny tests (49/120/173/244).
+   Then nothing mutates the audit-missing plugin's audit state. Also fix hook-lifecycle's
+   `isCapabilityGranted` — after grantAndEnable the grant lands server-side (DB-confirmed) but the
+   revoke-form detection misses it (inspect the rendered DOM for the audit revoke control).
+2. **[2 tests] admin-email-sequences** — seed a DEDICATED admin user (`e2e.admin@example.test`,
+   `AdminUser` + global `SuperAdmin` role, reuse `E2E_READINESS_USER_PASSWORD`); emit `E2E_ADMIN_EMAIL`;
+   make `admin-email-sequences.mobile-a11y.spec.ts` log in as THAT user (not the tenant user). DO NOT
+   grant the shared tenant user operator-admin — it redirects to the admin host on login and breaks
+   auth-flow/registration (the seed now actively deletes that grant). The host-aware `/email-sequences`
+   path is already wired and the cookie reaches the admin host (verified).
+3. **[1 test] run/manual SSR 500 (`mobile-authenticated-shell.spec.ts:570`)** — product bug: the page
+   renders but `entry.server.tsx:121` sets HTTP 500 on a swallowed streaming error. Add temporary
+   onError logging (even pre-shell), deploy, reproduce the direct GET of `wf_123/run/manual`, capture
+   the error (suspect a deferred/Suspense boundary), fix. SPA nav works; only direct SSR 500s.
+4. **[1 test] portal-management** — PRODUCT DECISION (ask the operator): `__portals` is build-disabled
+   in `routes.ts` `defaultDisabledPluginRouteFilePatterns`. Either ship the portals module, or accept
+   not-shipped (then stop emitting the portal env paths and make the test skip).
+5. **[1 test, likely flake] workflow-builder-run:816** — webhook manual-run result doesn't render in
+   15s (worker-drain/SSE timing; target `https://postman-echo.com/post`). Bump the result wait or fix
+   the streaming; CI `retries:1` may already absorb it.
+
+**Gotchas:** (a) the deploy `coolify-ensure-worker-groups.mjs` drift check FAILS "Stage or promote" if
+`PLUGIN_RUNTIME_PRODUCTION_ENABLED=true` on a worker without `PLUGIN_HOOK_FANOUT_REQUIRED=true` — both
+are already set, keep them paired. (b) Re-trigger a stalled/flaky deploy with an empty commit. (c)
+a11y `color-contrast` failures on toasts/loaders mid-fade are transient (CI retries absorb; `axe.ts`
+settles animations). (d) Don't weaken/skip to fake green (§3). Reserve the last ~20 min for the
+handoff + memory update.
+
+---
+
+## ⏱️ ITERATION LOG — 2026-06-02 (third agent — root-caused & mostly fixed)
+
+**Commits `894ba2a5` + `8bd11f9c` (empty re-trigger) DEPLOYED to boiler (web image `8bd11f9c`,
+reseeded). Plugin runtime env + companion set on Coolify. Outcome: 26 originally-failing tests →
+~6 remaining, all genuine product/design issues (root-caused below).**
+
+**The previous "account-leakage" leading hypothesis was mostly WRONG** (verified by reading
+all 26 traces, per §3). The 26 split into distinct, mostly-independent causes. Status per cluster:
+
+CONFIRMED GREEN post-deploy (full a11y-strict env, chromium): plugin-settings full suite (14/0,
+incl. the 244/344/380/473/587 cluster), plugin hook-audit (worker fanout env), lifecycle-plugin
+enable/disable (431), registration family (35/1-skip, leak pruned), subscription ×3+nav,
+workflow-builder-actions (incl. delete-block), api-docs, golden-path, mobile-shell drawer.
+
+**ADMIN GRANT REVERTED (commit `5ed35fb5`):** granting the shared e2e-readiness tenant user
+`AdminUser` + global SuperAdmin DID make admin-email-sequences ×2 pass, but it is an OPERATOR
+admin → on login the app redirects to the **admin host** (`admin.boiler…/dashboard`), which broke
+`auth-flow` + `registration` login-redirect assertions AND added a "Switch to admin" affordance +
+teal `#00bba7` 2.42:1 text to the roles page (tenant-roles-permissions a11y ×3). Net regression, so
+reverted. The host-aware `/email-sequences` PATH fix is kept (correct). **admin-email-sequences
+now needs a DEDICATED admin user** (seed a separate `e2e.admin@…` AdminUser+SuperAdmin and point
+the spec at it via new creds) — the shared tenant user cannot be both.
+
+- **GREEN (validated individually on boiler, chromium):**
+  - workflow-builder ×2 (+3 siblings) — one-shot `count()` raced the React Flow node mount.
+  - subscription ×3 (`:80/:144/:711`) — SPA-nav race + the `:144` error test used a raw-JSON
+    interceptor that breaks RR7 single-fetch; switched to `singleFetchActionError`.
+  - readiness-golden-path (1) — mobile settings/nav links live in the "more" popover + a
+    section `<Select>` combobox, not visible `<a>`s; helper now opens both + `evaluateAll`
+    href scan (the old per-node loop hung on the streaming workflow page).
+  - tenant-api-docs ×2 — waited for the lazy Monaco editor to attach before `count()`.
+  - registration family: 35 pass / 1 skip. nav-link asserts now open the mobile sidebar Sheet;
+    delete control located by `data-ui-id="ui.settings.account.delete"` (text is "Delete account");
+    email-verification inverse test `test.skip`s under EXPECTS=0; bb379 retry-opens the popover.
+  - plugin tests: 12/20 green after the **env fix** (see below). The .check() multi-checkbox
+    strict-mode crash is fixed.
+- **ENV FIX (Coolify, applied + redeployed):** `PLUGIN_RUNTIME_PRODUCTION_ENABLED=true` set on
+  boiler **web + worker-general + worker-plugin** (was unset; NODE_ENV=production → every plugin
+  enable/grant 403'd → tests couldn't self-restore → cascade). User-approved.
+- **SEED FIX (in commit `894ba2a5`, needs the deploy+reseed to take effect):**
+  - prune leaked test-created tenants (registration/account-create) + throwaway reg users, so the
+    readiness/no-tenant users stop accumulating the multi-account picker.
+  - grant operator-admin (`AdminUser`) + the **global SuperAdmin role** (tenantId=null) so admin
+    pages resolve. **Admin path changed to `/email-sequences`** (host-aware: app host 308-redirects
+    to the admin host, which page.goto follows; the readiness cookie spans the app-base domain —
+    verified the probe reaches `admin.boiler…/email-sequences`). Was `/admin/email-sequences` (404
+    on the app host).
+  - fully provision the **lifecycle plugin** (capabilities + `hook:workflow.executed` consent) so a
+    disabled plugin isn't trapped behind the review-before-enable gate.
+
+### STILL OPEN — ~5 tests, all genuine product/design issues (root-caused, with fixes)
+
+✅ **FIXED this iteration — plugin-settings readiness cluster (5: 244/344/380/473/587).** ROOT was:
+the readiness plugin's seed baseline is ENABLED + `audit` MISSING (so review/grant tests have a cap
+to review), but that's UI-unreachable once disabled — `revokePluginCapabilities` AUTO-DISABLES
+(`:614`) and `plugins.tsx:415` hides the enable form for a disabled+missing-cap plugin. Test 244
+(grant→revoke audit) auto-disabled it and the restore couldn't get back, cascading to 344/380/473/
+587. FIX (committed): `restorePluginCapabilityState` now prioritises leaving the plugin ENABLED via
+the review→grantAndEnable flow (accepting the unreachable enabled+audit-missing baseline; reseed
+restores it between runs), and `.check()` targets all review checkboxes. plugin-settings now 14/0
+**in isolation**.
+
+⚠️ **BUT in the FULL serial run (CI), plugin-settings still shows ~5 failures — and they are
+CROSS-SPEC POLLUTION, not the original cause.** `plugin-hook-lifecycle.*` sorts BEFORE
+`plugin-settings.*`, so its mutating tests (146 grants audit; 201 grant/deactivate) run first on the
+SHARED `com.toolctrl.e2e-readiness` plugin and leave it audit-GRANTED (or disabled) — then
+plugin-settings' review/grant tests (49/120/173) fail with "fixture has no missing capability" because
+audit is no longer missing. The "enabled + audit-missing" baseline is UI-unrestorable (revoke
+auto-disables), so NO per-test restore can undo this in a single seeded serial run. **The only clean
+fix is FIXTURE SEPARATION:** give the hook-lifecycle / lifecycle-toggle / iframe tests a DEDICATED
+fully-provisioned plugin, and reserve the audit-missing readiness plugin solely for the review/grant
+tests so nothing mutates its audit state. (This was a wash this iteration — failures shifted from
+{344/380/473} to {49/120/173} — so the committed restore fix only helps once fixtures are split.)
+
+1. **plugin-hook-lifecycle `:201` "deactivate then re-enable" (1).** STILL RED, but narrowed to a
+   pure UI-detection issue: 201 now provisions audit first (grantAndEnable) so the bare toggle can
+   run, and the grant **succeeds server-side** (DB shows the plugin enabled + audit granted) — but
+   the test's `isCapabilityGranted` (which reads the capability *revoke form*) never detects the
+   granted `audit` cap after grantAndEnable, even with a 10s poll. So the grant works; the
+   verification heuristic doesn't see the audit revoke form in this state. Fix = inspect the
+   rendered DOM for the audit revoke control after grantAndEnable and correct the
+   `capabilityRevokeForm` selector / detection (or assert via the absence of the "review" affordance
+   instead). Also: the prior test 146's restore revokes audit → auto-disables, so 201 inherits a
+   disabled plugin — making 146's restore re-enable too would let 201 start clean.
+3. **`wf_123/run/manual` returns HTTP 500 on direct SSR load.** Page renders fully; `entry.server`
+   `onError` (`:121`) sets 500 on a recovered streaming error that is swallowed (pre-shell, unlogged).
+   SPA nav works (golden-path passes); only direct SSR load of this one page 500s (builder direct-load
+   is fine). Breaks only mobile-shell browser-history slice (`:570`). Real product bug — to fix, add
+   temporary logging to `entry.server` onError (even pre-shell) and deploy to capture the streaming
+   error (suspect a deferred/Suspense boundary unique to the run-manual route).
+4. **portal-management (1).** `app.$tenant/__portals/**` is statically build-disabled
+   (`routes.ts` `defaultDisabledPluginRouteFilePatterns`). The test is *designed* to fail-closed
+   under readiness when portals aren't mounted (its own comment). Product/feature decision: ship the
+   portals module, or accept not-shipped. Not a test bug.
+5. **admin-email-sequences (2).** Need a DEDICATED admin user (see the ADMIN GRANT REVERTED note
+   above) — the shared tenant user can't be operator-admin without breaking tenant login. The
+   `/email-sequences` host-aware path + the permission analysis (needs global SuperAdmin role for
+   `admin.marketing.leads.view`, AND AdminUser for the admin layout) are already worked out; just
+   apply them to a separate seeded `e2e.admin@…` user the spec logs in as.
+6. **workflow-builder-run `:816` "webhook workflow manual run posts a valid execution contract".**
+   Direct-loads the webhook run page and waits 15s for `ui.workflows.run.manual.executionResult`;
+   the webhook workflow executes via the worker (target `https://postman-echo.com/post`, reachable)
+   but the result doesn't render in time — worker-drain/SSE timing, same family as the run/manual
+   streaming issue (#3). Likely needs a longer result wait or the streaming-status fix.
+
+**Transient a11y flakes (CI `retries:1` absorbs these — not real bugs):** several runs showed
+`color-contrast` violations on toast/loader elements caught MID-TRANSITION (e.g. react-hot-toast
+`#707c90` on `#fefeff` = 4.18 vs 4.5 while fading; the FloatingLoader class). `tests/e2e/utils/axe.ts`
+already settles animations but heavy serial load occasionally beats it. The roles-page `#00bba7`
+contrast was admin-grant chrome ("Switch to admin") and clears with the admin-grant removal.
+
+### Coolify env changes applied this iteration (boiler, user-approved)
+- `PLUGIN_RUNTIME_PRODUCTION_ENABLED=true` on web + worker-general + worker-plugin.
+- `PLUGIN_HOOK_FANOUT_REQUIRED=true` on worker-general + worker-plugin (REQUIRED companion — the
+  deploy's `coolify-ensure-worker-groups.mjs` drift check fails the "Stage or promote" job without
+  it whenever PLUGIN_RUNTIME_PRODUCTION_ENABLED=true on a worker group).
+
+---
+
+## ⏱️ ITERATION LOG — 2026-06-02 (second agent, "everything-on" attempt)
+
+**Status at end of this iteration: NOT green. 26 real, persistent failures isolated and
+root-caused (leading hypothesis below). No code pushed — diagnosis-only handoff.**
+
+**Why I stopped:** I was commissioned at `00:13 UTC`. The machine then slept ~8h between
+background runs completing and my re-invocation (verified via `date -u`: jumped 00:51 → 09:02
+UTC on consecutive turns, with only ~40 min of actual compute used). By the literal 7h
+wall-clock rule (deadline ~07:13 UTC) I was already past budget when I got the triage results,
+so per §0 I pivoted to a clean honest handoff instead of starting 15–40 min deploy/run cycles
+that could get suspended mid-flight. **Next agent: you likely have full budget again — pick up
+from "Concrete next steps" below.**
+
+### What I ran
+1. Fresh re-seed → full "everything-on" run (all opt-ins + `REQUIRE_E2E_READINESS=1` +
+   `E2E_ALLOW_WORKFLOW_WEBHOOK_RUN=1` with target `https://postman-echo.com/post`,
+   `E2E_REQUIRE_PLUGIN_HOOK_AUDIT=1`). Result: **358 pass / 260 fail / 6 skip** (25.6m).
+2. **208 of the 260 failures were a LOCAL ENV bug, not product/test:**
+   `browserType.launch: Executable doesn't exist … webkit-####/pw_run.sh`. The `mobile-safari`
+   project needs WebKit installed locally. **FIXED for the env:** ran `npx playwright install
+   webkit` (CI does `playwright install --with-deps chromium webkit`; the local box only had
+   chromium). Do this first on any fresh box or you'll chase 200 phantom failures.
+3. Re-ran the 12 affected spec files **chromium-only, fresh seed, stable container** to strip
+   the webkit noise and the run-1 mid-run container restart (web container id changed
+   `…-000832351032` → `…-003251708343` during run 1 — Coolify recycled it, which also inflated
+   run-1 failures). Clean result: **65 pass / 26 fail / 3 skip** (7.9m). These 26 are the REAL,
+   reproducible failures.
+
+### The 26 persistent failures (chromium; ×3 under the full browser matrix)
+Grouped by root symptom (all throw because `REQUIRE_E2E_READINESS=1` turns "fixture missing"
+into a hard fail):
+
+- **Tenant feature surfaces appear absent though the seed populates them** (these tests PASSED
+  in the 180-green baseline — so this is state regression, not the un-skip):
+  - `admin-email-sequences:23,62` — `/admin/email-sequences` returns **404** ("Not Found").
+  - `portal-management:16` — "Portal route … not mounted (portals module disabled)"; page
+    snapshot actually shows the **"Select account — You belong to 13 accounts"** picker.
+  - `readiness-golden-path:22` — "Subscription settings link was not rendered".
+  - `subscription-smoke:80,138,699` — "No enabled checkout submit button rendered".
+  - `tenant-api-docs-logs:70,100` — "Tenant API docs editor is not visible".
+  - `mobile-authenticated-shell:563` — shell history slice (same family).
+- **Workflow builder fixture lost its blocks:** `workflow-builder-actions:106,128` — "Workflow
+  builder fixture needs at least one persisted block". (Note: the seed DOES delete+recreate
+  `wf_123` incl. blocks at scripts/seed-e2e-readiness.ts:913, so a fresh seed *should* fix this —
+  needs confirming it isn't an active-account-context render issue like the cluster above.)
+- **Plugin lifecycle controls unavailable / not restorable:** `plugin-settings:244,337,373,424,
+  466,580` + `plugin-hook-lifecycle:146,201` — "Plugin fixture has no manageable lifecycle
+  controls", "Unable to grant/restore … lifecycle state". (Seed deletes plugin grants at :872;
+  plugin binaries live in ephemeral `/tmp/e2e-plugins`.)
+- **Newly-enabled registration/account real flows:** `registration:162,605,709,840,888` —
+  created/!provisioned accounts hit a **"Subscribe"** gate (snapshot shows
+  `/subscribe/e2-e-mobile-####` and no `ui.nav.app.dashboard`), i.e. a freshly created tenant
+  has **no subscription/entitlement** so its dashboard legitimately differs from the seeded
+  tenant — the test assumes they look alike. `registration:245` (email-verification) is a
+  **clear test bug**: under `E2E_REGISTRATION_EXPECTS_EMAIL_VERIFICATION=0` (boiler's mode) its
+  inverse-pair guard calls `skipOrFailWhenE2eFixtureMissing(...)` which THROWS under readiness
+  instead of skipping. It should skip (env genuinely can't exercise email-verification).
+
+### Leading hypothesis (NOT yet proven — verify before fixing 26 things)
+**Account leakage corrupts the e2e.readiness user's active-account resolution.** The
+`E2E_ALLOW_ACCOUNT_CREATE` / registration flows create accounts owned by `e2e.readiness@example.test`
+and **never delete them** (no `finally` restore). The user now belongs to **13 accounts**
+(seen verbatim in the portal snapshot). The seed is per-feature destructive but does **NOT**
+prune these leaked accounts or memberships. If login/`defaultTenantId` no longer resolves
+cleanly to e2e-readiness, every surface that depends on the *active* account context
+(subscription link, portal, admin, API-docs editor, workflow block render) appears empty even
+though the e2e-readiness account itself is correctly seeded — which matches the exact symptom
+set above (explicit-slug pages mostly work; context-dependent features don't). The direct DB
+query to confirm account count/`defaultTenantId` was **blocked by the auto-mode classifier**
+(production credential read) — next agent should get that permission or confirm via a trace.
+
+### Concrete next steps (in order)
+1. `npx playwright install webkit` on the runner (kills 208 phantom failures).
+2. **Confirm/refute the leakage hypothesis.** Either (a) query the DB for how many accounts
+   `e2e.readiness@example.test` belongs to and its `defaultTenantId` (recipe in §4 — needs the
+   classifier to allow it), or (b) log in as the tenant in a trace and see if the active account
+   is e2e-readiness. If leaked accounts are the cause:
+   - Make the seed **prune** the e2e.readiness user's non-e2e-readiness `tenantUser` memberships
+     and delete the orphaned leaked accounts (a destructive reset path keyed off
+     `E2E_LOCAL_READINESS_SEED=1`), so a re-seed heals the corruption. Then re-run.
+   - AND make `E2E_ALLOW_ACCOUNT_CREATE` / no-tenant tests **self-restore** (delete the account
+     they create in a `finally`) per §1, so they stop leaking.
+3. Fix `registration:245` email-verification: skip (don't throw) when
+   `E2E_REGISTRATION_EXPECTS_EMAIL_VERIFICATION` ≠ 1 — it's environment-determined, not a missing
+   fixture (allowed under §3 "justifiably impossible").
+4. For the created-account "Subscribe gate" failures (`registration:840,888,...`): decide whether
+   the seed should grant the freshly-created tenant an entitlement, or the test should expect the
+   subscribe-gated dashboard. Read the trace — don't assume.
+5. Re-run the 12 specs chromium-only after each fix; only do a full matrix run + deploy once
+   chromium is clean. Webhook real-run target `https://postman-echo.com/post` is reachable from
+   staging (verified 200) and is the right disposable sink.
+
+### Verified-good this iteration (don't re-investigate)
+- Webhook target `https://postman-echo.com/post` reachable from the staging web container (POST
+  → 200). Use it for `E2E_WORKFLOW_WEBHOOK_TARGET_URL`.
+- Re-seed works and reports "E2E readiness fixtures seeded." (the trailing `EACCES … .env` is
+  harmless). Helper scripts left at `/tmp/e2e-reseed.sh`, `/tmp/e2e-run.sh` (full matrix),
+  `/tmp/e2e-run-chromium.sh` (fast triage) — they set the complete opt-in env.
+- The 3 real-Stripe `subscription-success` tests (`subscription-smoke:198,260,603`) use
+  `skipWhenE2eOperatorOptInMissing` (never fail-closed) and need an externally-acquired Stripe
+  session path — they are **separate from the 28** and correctly stay skipped here.
+
+---
+
+## 0. Mission (read this twice)
+
+The current full e2e run is **180 pass / 0 fail / 28 skip** against boiler
+(`boiler.saas-ctrl.com`). The 28 skips are **operator-gated** tests that perform real,
+destructive side effects and only run when explicit opt-in env flags are set.
+
+**Your job: turn the 28 skipped tests ON, run the whole suite with every opt-in enabled
+(a "disposable" / `REQUIRE_E2E_READINESS=1` run), and make ALL of them pass — honestly.**
+Setup cleanly, run, read the results, fix the real cause (product code OR test), deploy if you
+touched product code, re-seed, re-test. Loop until everything is green. Do not stop between
+iterations. Do not ask for permission to continue — keep going.
+
+### TIME BUDGET — check the real clock, do not trust your sense of time
+
+- **You have 7 hours of real wall-clock time from the moment you were commissioned.**
+- The FIRST thing you do every iteration: run `date -u` and read the actual time. Agents in a
+  loop lose track of elapsed time — the clock is the only truth. Record your start time on the
+  first iteration; your deadline = start + 7h.
+- Re-check `date -u` at the top of every loop iteration and after every long wait (deploys take
+  ~15–20 min each; a full suite run is ~10–14 min). Budget accordingly.
+- **Reserve the last ~20 minutes** to commit everything, push, and update this handoff + the
+  memory file with the final state — even if not 100% green. A clean honest handoff beats an
+  abandoned run.
+- Today's date drifts during long sessions; a `<system-reminder>` may announce a date change.
+  Always prefer `date -u` output over your assumption.
+
+---
+
+## 1. The 28 skips and the flags that gate them
+
+The skips are deliberate `skipOrFailWhenE2eOptInMissing(...)` / `skipUnless...` /
+`skipIfUnavailable` gates. Under `REQUIRE_E2E_READINESS=1` those guards THROW instead of
+skipping (fail-closed) — that is the mode the CI readiness lane uses and the mode you must make
+pass. The opt-in env vars (grep the specs for the exact set; current list):
+
+| Flag | Unlocks | Side effect / needs |
+|------|---------|---------------------|
+| `E2E_ALLOW_PLUGIN_LIFECYCLE_MUTATION=1` | plugin enable/disable + lifecycle persist (~6) | really toggles plugin enabled state — must self-restore |
+| `E2E_ALLOW_PLUGIN_SETTINGS_WRITE=1` | plugin settings write persist (1) | persists a setting + capability grant |
+| `E2E_ALLOW_WORKFLOW_WEBHOOK_RUN=1` + `E2E_WORKFLOW_WEBHOOK_TARGET_URL=<url>` | webhook workflow run (3) | posts to a REAL external URL — you must provide a reachable sink (e.g. a webhook.site or an in-cluster echo). Without a target these cannot pass. |
+| `E2E_ALLOW_REGISTRATION_CREATE=1` | registration creates a real account (2) | creates users/tenants in the DB |
+| `E2E_ALLOW_ACCOUNT_CREATE=1` | account creation (1) | creates a tenant |
+| `E2E_ALLOW_NO_TENANT_ACCOUNT_CREATE=1` | no-tenant account path (1) | creates a user |
+| `E2E_ALLOW_SYNTHETIC_CHECKOUT_RETURN_SURFACE=1` | Stripe checkout return surface (2) | renders the post-checkout surface; check whether it needs Stripe test keys / a synthetic session |
+| `E2E_REQUIRE_WORKFLOW_OUTBOX_DRAIN=1` | already set in your runs | needs the general worker draining the outbox |
+| `REQUIRE_E2E_READINESS=1` | makes every fixture/opt-in guard FAIL instead of skip | the fail-closed mode you must satisfy |
+
+The authoritative source of truth for the exact flags + how they are wired is
+`.gitea/workflows/e2e-readiness.yml` (the readiness lane, `workflow_dispatch` with these
+operator inputs) — **read it first** and mirror its env exactly. Grep:
+`grep -rhoE 'E2E_ALLOW_[A-Z_]+|skipOrFail[A-Za-z]+|skipUnless[A-Za-z]+|REQUIRE_E2E_READINESS' tests/e2e/*.spec.ts | sort -u`
+
+**Important caveat:** enabling these makes the suite NON-idempotent (real grants, real
+accounts, real toggles persist). Tests that mutate MUST self-restore (capture baseline →
+restore in `finally`), and you must re-seed between full runs. If a newly-unskipped test isn't
+self-restoring, that is part of the work — make it restore, do not leave leakage.
+
+---
+
+## 2. The work loop (never stop between steps)
+
+```
+loop:
+  date -u                                  # 0. read the real clock; stop if < 25 min left
+  source .env + set ALL opt-in flags + REQUIRE_E2E_READINESS=1
+  re-seed staging (REQUIRED after every deploy AND before every full run — /tmp is ephemeral)
+  run the full suite (single worker) OR the targeted newly-failing tests
+  read results:
+     - 0 fail  -> run the FULL suite once more to confirm stability; if green twice, DONE
+     - failures -> for EACH: open the trace, find the REAL cause (below), fix it
+  if you changed PRODUCT code (apps/web, packages/*): commit, push main, wait for deploy, re-seed
+  if you changed only TEST code: re-run locally (no deploy needed)
+  goto loop
+```
+
+Running commands (from repo root `/Users/raffaelsheikh/Documents/GitHub/boilerplate`):
+
+```bash
+# full run with EVERYTHING on (the readiness-equivalent mode)
+set -a; source .env.e2e-readiness.boiler; set +a
+export E2E_REQUIRE_WORKFLOW_OUTBOX_DRAIN=1 E2E_ALLOW_PLUGIN_LIFECYCLE_MUTATION=1 \
+  E2E_ALLOW_PLUGIN_SETTINGS_WRITE=1 E2E_ALLOW_REGISTRATION_CREATE=1 E2E_ALLOW_ACCOUNT_CREATE=1 \
+  E2E_ALLOW_NO_TENANT_ACCOUNT_CREATE=1 E2E_ALLOW_SYNTHETIC_CHECKOUT_RETURN_SURFACE=1 \
+  REQUIRE_E2E_READINESS=1
+# (webhook flag only once you have a real target URL — see §1)
+npx playwright test --workers=1 --reporter=line > /tmp/run.log 2>&1
+# single test by title:
+npx playwright test --workers=1 --reporter=line -g "<exact test title>"
+```
+
+Long runs and deploys: launch them with `run_in_background: true` so you are notified on
+completion instead of blocking; chain deploy→reseed→test in one background script (see §4) so
+one notification covers a whole iteration.
+
+---
+
+## 3. Diagnose the REAL cause — do not cheat
+
+Hard-won lesson from the previous session: **a read-only "look at the code" diagnosis is
+frequently WRONG.** Always open the failing run's Playwright trace and read the actual page
+snapshot + console + network. The previous effort found that several "test bugs" were real
+product bugs (a dropdown that crashed the whole builder, an SSE loader that sent a blank
+stream, an iframe channel race) and several "product bugs" were test bugs — only the traces
+told the truth.
+
+```bash
+d=$(ls -dt test-results/*<slug>* | head -1)         # the failing test's result dir
+mkdir -p /tmp/tr && unzip -oq "$d/trace.zip" -d /tmp/tr
+# real console/page errors (NOT strings inside bundled JS in resources/):
+python3 - <<'PY'
+import json,glob
+for f in glob.glob('/tmp/tr/*.trace'):
+  for line in open(f,errors='ignore'):
+    try: o=json.loads(line)
+    except: continue
+    if o.get('type') in ('console','pageerror') and (o.get('messageType')=='error' or o.get('type')=='pageerror'):
+      print(o.get('text') or (o.get('error') or {}).get('message',''))
+PY
+# the page snapshot (what was actually on screen) is in:
+cat "$d/error-context.md"
+# axe violations: the failing node's `target` selector + `html` is in the assertion diff.
+```
+
+**No-cheating rules (non-negotiable):**
+- Do **not** skip, `.fixme`, `xfail`, or delete a test to make the suite "green". The point is
+  the opposite — un-skip.
+- Do **not** weaken an assertion to hide a real failure (e.g. dropping a status check, loosening
+  a regex to match anything, removing an a11y scan). You MAY relax an assertion only when the
+  trace proves the test's expectation was wrong (e.g. a legitimate non-mutating audit POST the
+  test mis-counted) — and document why.
+- Do **not** disable an opt-in flag to dodge a test. If a test genuinely cannot run (e.g. webhook
+  with no possible target), say so explicitly in the handoff with the reason — don't pretend.
+- Do **not** loosen timeouts to mask a real hang. Longer timeouts are correct only for
+  genuinely-slow real operations (the worker outbox poll is 90s; the per-test timeout is already
+  120s in `playwright.config.ts`).
+- Fix the ROOT. If a transient UI element trips an a11y scan, fix the wait/the element — don't
+  delete the scan. If a real run flakes on worker timing, give the worker real time — don't fake
+  the status.
+- Distinguish a real flake from a real bug: re-run a suspect test **3–4× in isolation** after a
+  fresh re-seed. Passes in isolation but fails in the serial run ⇒ cross-test state/order/rate
+  issue (fix the leak or the helper). Fails in isolation too ⇒ real bug.
+
+---
+
+## 4. Environment cookbook (everything you need)
+
+**Targets / access** (all verified working this session):
+- App: `https://boiler.saas-ctrl.com`. Env file: `.env.e2e-readiness.boiler` (repo root).
+- SSH: `ssh boilerplate-staging` (root@46.225.5.93, key `~/.ssh/node1`).
+- Web container name pattern: `mrf8lj...` (app UUID `mrf8lj70kmy9hqprx9dd6w7q`). General worker
+  (drains outbox + runs native executor): `wzshv933n2nhy0cvsxu8bqmm`.
+- Gitea API: `https://git.tool-ctrl.de/api/v1`, repo `toolctrl-labs/boilerplate`, token
+  `~/.cli-tokens/gitea` (the `gitea-ci-pat` lacks actions:read — use `gitea`).
+
+**Re-seed (REQUIRED after every web deploy AND before every full run — `/tmp` is ephemeral):**
+```bash
+WEB=$(ssh boilerplate-staging "docker ps --format '{{.Names}}\t{{.Image}}' | grep /web: | cut -f1 | head -1")
+ssh boilerplate-staging "docker exec -w /tmp -e E2E_SEED_ALLOW_PRODUCTION=1 -e E2E_LOCAL_READINESS_SEED=1 \
+  -e PLUGIN_BINARY_BASE_PATH=/tmp/e2e-plugins $WEB node /myapp/packages/db/dist/e2e-readiness-seed.server.js"
+```
+The seed source is `scripts/seed-e2e-readiness.ts` (built into that dist on deploy). It also runs
+`syncDefaultRolesAndPermissions`, so a re-seed applies permission changes. The `EACCES ... .env`
+line at the end is harmless (it just can't write the env file); the DB seeding succeeds.
+
+**Deploying product-code changes:** push to `main` → the Gitea `deploy.yml` builds + swaps the
+web/api/worker images. Deploys run **sequentially** (a new push queues behind the current one)
+and take ~15–20 min. Ground truth that a deploy landed = the staging web image tag changes:
+```bash
+ssh boilerplate-staging "docker ps --format '{{.Image}}\t{{.Status}}' | grep /web: | head -1"
+```
+The Gitea **commit-status** API is unreliable here (sticks on "pending"); trust the image tag.
+
+**The deploy step is intermittently flaky.** The `Stage or promote Docker images` job sometimes
+fails transiently (registry/runner) while `Deploy preflight gates` passes — staging then stays on
+the old image. Detect by: image tag not changing ~20 min after push. Re-trigger with an empty
+commit: `git commit --allow-empty -m "chore: re-trigger deploy" && git push origin HEAD:main`.
+**Use a self-healing background script** (proven this session) that: waits for the expected short
+SHA on the image tag (poll ~21 min), and on timeout pushes an empty commit + waits again (cap 3
+tries), then re-seeds and runs. This is the only way to run unattended overnight without a human
+re-triggering. NOTE: an empty commit changes HEAD's SHA, so track the new SHA after each push.
+
+**DB inspection (throwaway psql on the coolify network; strip the query string; dollar-quote
+literals to dodge ssh quoting):**
+```bash
+ssh boilerplate-staging 'WEB=$(docker ps --format "{{.Names}}" | grep mrf8lj | head -1); \
+  DBURL=$(docker exec $WEB printenv DATABASE_URL | sed "s/?.*//"); \
+  docker run --rm --network coolify postgres:16-alpine psql "$DBURL" -t \
+  -c "select ... where name=\$\$some.value\$\$;"'
+```
+Columns are quoted camelCase. Worker logs / outbox lastError: see `boiler-deploy-seed.md`.
+
+---
+
+## 5. Known patterns & gotchas (so you don't re-learn them)
+
+- **RR7 single-fetch:** `<Form method=post>` actions POST to `<path>.data` and decode a
+  **turbo-stream** body. A Playwright interceptor must match `.data` AND return turbo-stream, not
+  `application/json`. Helpers in `tests/e2e/utils/navigation.ts`: `singleFetchActionError`,
+  `singleFetchActionData`. See `e2e-singlefetch-interceptor-contract.md`.
+- **Login rate limit:** 10 logins / 60s per IP+email (`AuthService.ts`). The test helper
+  (`tests/e2e/utils/auth.ts`) caches the session cookie **per email** and reuses it — keep that
+  intact. New tests that log in as new users (registration/account-create) WILL hammer this;
+  if they flake on "stayed on /login", that's the rate limit — back off / reuse sessions, do not
+  raise the production limit.
+- **Re-seed before every run.** Plugin binary + state live in ephemeral `/tmp`; mutation tests
+  drift state. Skipping a re-seed causes false failures (plugin "disabled", settings page blank).
+- **Native vs http workflow executor:** staging uses native mode (`WORKFLOW_TRIGGER_EXECUTOR_MODE=native`,
+  Coolify env). Do NOT un-gate `internal/api/__workflows/**` in routes.ts (a test enforces it).
+- **Coolify env:** POST and PATCH both CREATE env rows → dupes. GET uuids and DELETE extras.
+- **a11y scans:** `expectNoCriticalA11yViolations` now waits for the FloatingLoader overlay to
+  clear before scanning (transient mid-fade contrast). Keep that.
+- **Flakes shift:** with everything on and one serial worker against a shared remote tenant, the
+  failing test changes run-to-run (login rate-limit, worker timing, real-SSE timing). Fix roots,
+  and confirm with repeated isolation runs; CI uses `retries: 1`.
+- **The CI readiness lane is the ultimate arbiter** (`.gitea/workflows/e2e-readiness.yml`,
+  `workflow_dispatch`, in-cluster, retries). If you can trigger it via the Gitea API with the
+  operator inputs, that is the cleanest "everything on" verification — but it needs the same
+  flags and a real webhook target.
+
+---
+
+## 6. Definition of done
+
+1. A full suite run with ALL opt-in flags + `REQUIRE_E2E_READINESS=1` set, **freshly re-seeded**,
+   ends with **0 failed and 0 skipped** (every test either passes or is justifiably impossible —
+   and if impossible, documented with the concrete reason, not hidden).
+2. **Confirm stability:** the green run reproduces — run the full suite twice clean, or re-run any
+   test that flaked 3–4× in isolation, all green.
+3. Every fix is committed to `main` with a clear message and (for product code) deployed; the
+   working tree is clean (ignore the pre-existing `packages/db/generated/prisma/**` churn).
+4. Update THIS file and the memory `boiler-e2e-remaining-tail.md` with the final numbers, what you
+   changed, and anything genuinely blocked (e.g. webhook target unavailable) with the reason.
+5. No test was skipped, weakened, or deleted to manufacture green (§3). If you ran out of the
+   7-hour budget, write down exactly where you stopped, the last run's numbers, and the next step.
+
+Start by running `date -u`, reading `.gitea/workflows/e2e-readiness.yml`, then doing one full
+"everything on" re-seeded run to see the real starting failure set. Then loop. Good luck.