HiveGrid-Wireframes/hivegrid-cloud/docs/testing/e2e-readiness.md

E2E Readiness Fixtures

Use the local readiness seed when a fresh database should exercise the authenticated Playwright suite instead of silently skipping plugin, workflow, billing, and dashboard paths. The same fail-closed contract is wired into .github/workflows/e2e-readiness.yml and .gitea/workflows/e2e-readiness.yml for scheduled or manually dispatched staging/readiness runs.

This is a local test-readiness runbook. It does not replace the manual, redacted staging evidence contract in Staging, Live, and UAT Readiness, and it does not mark the plugin runtime production-ready before the external security sign-off described in Plugins. These fixtures prove local authenticated browser/a11y coverage for the named UI paths; they do not prove production readiness for plugin execution, workflow workers, provider delivery, staging UAT, or operator-owned infrastructure. Run the browser command below only as an operator-approved local/readiness browser run; it is intentionally not part of automatic docs/static readiness verification. If E2E_BASE_URL is unset, empty, or only whitespace, the Playwright wrapper probes the local default http://localhost:3000.

E2E_LOCAL_READINESS_SEED=1 pnpm run seed:e2e-readiness
set -a; source .env.e2e-readiness; set +a
pnpm run verify:a11y-coverage
REQUIRE_E2E_A11Y=1 REQUIRE_AUTH_E2E=1 REQUIRE_E2E_READINESS=1 E2E_A11Y_STRICT=1 E2E_BROWSER_MATRIX=1 E2E_ALLOW_WORKFLOW_RUN=1 pnpm run verify:e2e-a11y

The seed is idempotent, refuses NODE_ENV=production, and writes .env.e2e-readiness with deterministic credentials, E2E_NO_TENANT_EMAIL / E2E_NO_TENANT_PASSWORD for the disposable no-tenant account-creation flow, E2E_PLUGIN_READONLY_EMAIL / E2E_PLUGIN_READONLY_PASSWORD, account slug/name, tenant dashboard/settings/profile paths, governance/policies path, API overview/keys/docs/logs paths, admin email sequence path, plugin settings path, workflow builder/API-run/manual-run paths, a separate webhook workflow builder/manual-run path, and a waiting-workflow execution path. The seeded workflow ids are fixed (wf_123, wf_webhook_123, and wf_waiting_123) so the local env file and scheduled .github/.gitea readiness workflows target the same staging paths. The seed also creates one successful manual execution (wfe_123) with a successful log block run (wfbe_log_123) so fresh strict-readiness databases can render workflow execution list/detail mobile and a11y checks before any mutating run drains through a worker. It also creates a paused wait-for-input execution (wfe_waiting_123) so waiting-state execution details and continuation form contracts can be inspected deterministically without claiming that the worker completed the continuation. The no-tenant fixture user is reset with no default tenant, no memberships, and no roles so E2E_ALLOW_NO_TENANT_ACCOUNT_CREATE=1 exercises the real account onboarding path instead of reusing the seeded tenant. The read-only plugin user joins the seeded tenant, has that tenant as its default tenant, and is reset to only the default User app role, which includes app.plugins.use but not app.plugins.manage. It also writes E2E_ALLOW_WORKFLOW_WEBHOOK_RUN=0 plus an empty E2E_WORKFLOW_WEBHOOK_TARGET_URL so a sourced readiness env never accidentally calls an external endpoint. It also writes E2E_REQUIRE_WORKFLOW_OUTBOX_DRAIN=0, E2E_WORKFLOW_OUTBOX_DRAIN_EXPECTED_STATUS=success, E2E_WORKFLOW_OUTBOX_DRAIN_ASSERTION=operator-gated, and an operator-gated readiness note because successful workflow execution requires the general worker/outbox path to be running. It also seeds one active owned subscription plan for billing/cancellation checks and a second public unowned upgrade plan so checkout-catalog smokes always have an enabled submit button without mutating the active subscription. It also seeds deterministic admin email sequences, one non-secret API key, and one /v1/e2e-readiness API log row so authenticated API overview, docs, logs, governance, email-sequence dashboard, and key-management mobile/a11y checks can run without production telemetry. The default manual workflow is log-only and enables E2E_ALLOW_WORKFLOW_RUN=1; the API-run page stays form-contract tested without invoking the backend API directly, and the webhook workflow stays contract-tested by default so it can validate HTTP/Webhook payload wiring without external secrets. The plugin fixture intentionally has one granted capability and one missing capability so consent, revoke, manifest, and settings iframe checks have meaningful state.

Workflow worker/outbox-drain readiness is now available as an explicit operator-gated browser probe. Keep E2E_REQUIRE_WORKFLOW_OUTBOX_DRAIN unset or 0 for normal local, staging, and scheduled readiness runs unless the general worker is running and subscribed to the workflow/outbox path. When an operator sets E2E_REQUIRE_WORKFLOW_OUTBOX_DRAIN=1, the workflow mobile/a11y spec requires seeded tenant credentials plus E2E_WORKFLOW_RUN_MANUAL_PATH, submits the log-only manual workflow, follows the returned execution details link, and polls until the execution reaches E2E_WORKFLOW_OUTBOX_DRAIN_EXPECTED_STATUS (success by default). This probe intentionally waits for the domain WorkflowExecution, not merely for the OutboxEvent row to become DONE: outbox DONE only proves handoff to BullMQ, while outbox.workflow job state and WorkflowExecution.status prove whether the worker bridge actually ran the workflow. A drained execution that ends in error is a failed readiness signal unless an operator explicitly sets E2E_WORKFLOW_OUTBOX_DRAIN_EXPECTED_STATUS=error for a negative-path probe. Use E2E_WORKFLOW_OUTBOX_DRAIN_TIMEOUT_MS to tune the default 90-second poll window for slower readiness environments.

Plugin workflow.executed hook readiness is a separate operator-gated browser probe layered on top of the workflow outbox drain. Keep E2E_REQUIRE_PLUGIN_HOOK_AUDIT unset or 0 for normal local, staging, and scheduled readiness runs; the plugin hook lifecycle spec then only exercises the non-mutating audit-trail view and the mutation-gated deactivate→re-enable lifecycle flow. Setting E2E_REQUIRE_PLUGIN_HOOK_AUDIT=1 requires a running general worker, the plugin runtime, and outbox drain, so it reuses the same workflow outbox-drain readiness gate (it internally calls requireWorkflowOutboxDrainReadiness, which still needs E2E_REQUIRE_WORKFLOW_OUTBOX_DRAIN=1). When enabled, the spec requires E2E_TENANT_EMAIL, E2E_TENANT_PASSWORD, E2E_PLUGIN_SETTINGS_PATH, and E2E_WORKFLOW_RUN_MANUAL_PATH; any missing value fails with a clear configuration error instead of skipping. The probe grants the seeded readiness plugin's audit capability (mutation-gated additionally behind E2E_ALLOW_PLUGIN_LIFECYCLE_MUTATION=1), runs the seeded log-only manual workflow to fire workflow.executed, waits for the worker to drain the outbox and the execution to terminate, then polls the plugin capability-use audit trail until a success row appears for the hook-driven audit/log capability. It restores the seeded baseline (revoking the audit grant; re-running the readiness seed also resets capability state) so reruns stay idempotent. Use E2E_PLUGIN_HOOK_AUDIT_TIMEOUT_MS to tune the default 90-second audit poll window for slower readiness environments. This operator opt-in is owned by the readiness lane, not the seed, so the seed's buildE2eReadinessEnv does not emit E2E_REQUIRE_PLUGIN_HOOK_AUDIT or E2E_PLUGIN_HOOK_AUDIT_TIMEOUT_MS.

Real webhook/HTTP execution is a separate opt-in. Keep E2E_ALLOW_WORKFLOW_WEBHOOK_RUN unset or 0 for normal local, staging, and scheduled readiness runs; the Playwright spec intercepts the form POST and verifies the payload contract only. For a disposable operator-approved run that may call an HTTP endpoint, set both E2E_ALLOW_WORKFLOW_WEBHOOK_RUN=1 and E2E_WORKFLOW_WEBHOOK_TARGET_URL=<operator-owned-url> after sourcing .env.e2e-readiness. When that opt-in is enabled, the wrapper and spec require E2E_TENANT_EMAIL, E2E_TENANT_PASSWORD, E2E_WORKFLOW_WEBHOOK_BUILDER_PATH, E2E_WORKFLOW_WEBHOOK_RUN_MANUAL_PATH, and E2E_WORKFLOW_WEBHOOK_TARGET_URL; any missing value fails with a clear configuration error instead of skipping.

The scheduled GitHub/Gitea readiness workflows keep those operator-owned probes off by default. Manual workflow_dispatch exposes explicit inputs for plugin lifecycle mutation, plugin settings writes, workflow outbox drain, and real webhook execution; the wrapper fails before Playwright starts when a selected input is missing its required path, credential, expected status, or target URL contract.

Mutating provisioning checks stay opt-in and are separate from normal strict readiness. Set REQUIRE_E2E_MUTATING_READINESS=1 together with the relevant E2E_ALLOW_* opt-ins only for disposable local/readiness databases where the suite may create a new user, tenant, no-tenant account, or mutate plugin state. Keep those values unset for non-mutating staging smoke runs.

For a disposable local database where all mutating readiness flows are operator-approved, use the named wrapper so every required opt-in stays visible and versioned together:

E2E_LOCAL_READINESS_SEED=1 pnpm run seed:e2e-readiness
set -a; source .env.e2e-readiness; set +a
pnpm run verify:e2e-mutating-readiness:local

verify:e2e-mutating-readiness:local sets REQUIRE_E2E_MUTATING_READINESS=1, E2E_ALLOW_REGISTRATION_CREATE=1, E2E_ALLOW_ACCOUNT_CREATE=1, E2E_ALLOW_NO_TENANT_ACCOUNT_CREATE=1, E2E_ALLOW_PLUGIN_LIFECYCLE_MUTATION=1, and E2E_ALLOW_PLUGIN_SETTINGS_WRITE=1. Do not use it against shared staging or production databases. Registration provisioning has two explicit branches: keep E2E_REGISTRATION_EXPECTS_EMAIL_VERIFICATION=0 when the lane must prove that a new user reaches a tenant dashboard and can log back in; set it to 1 only for a disposable verification-required lane that proves signup stops at the email verification confirmation instead of entering the app shell. When E2E_ALLOW_NO_TENANT_ACCOUNT_CREATE=1 is set, the wrapper also fails before Playwright starts unless E2E_NO_TENANT_EMAIL and E2E_NO_TENANT_PASSWORD are present. The local readiness seed writes those values, so the named wrapper should be run after sourcing .env.e2e-readiness.

When the local general worker is also running and subscribed to the workflow outbox path, use the worker-drain wrapper instead:

E2E_LOCAL_READINESS_SEED=1 pnpm run seed:e2e-readiness
set -a; source .env.e2e-readiness; set +a
pnpm run verify:e2e-mutating-readiness:worker-local

verify:e2e-mutating-readiness:worker-local delegates to the disposable local mutating readiness lane and additionally sets E2E_REQUIRE_WORKFLOW_OUTBOX_DRAIN=1. That makes the browser workflow run poll the resulting WorkflowExecution.status until it reaches E2E_WORKFLOW_OUTBOX_DRAIN_EXPECTED_STATUS (success by default), so the lane proves the worker bridge drained the workflow instead of only proving that the web runtime accepted a manual run request. Do not use it against shared staging or production databases.

Plugin lifecycle persistence checks also stay opt-in. Default plugin settings readiness verifies mobile/a11y, consent, revoke forms, lifecycle forms, manifest disclosure, and settings iframe contracts without changing the seeded plugin state. Set E2E_ALLOW_PLUGIN_LIFECYCLE_MUTATION=1 only for disposable local/readiness databases where the suite may persist capability grant/revoke and plugin enable/disable state, then leave it unset for non-mutating staging smoke runs.

REQUIRE_E2E_READINESS=1 intentionally requires the mobile browser matrix, strict a11y mode, real log-only workflow-run opt-in, and every seeded path or credential consumed by the non-mutating strict specs: dashboard, account/profile settings, governance/policies, subscription, API overview/keys/docs/logs, portal analytics/settings, admin email sequences, plugin settings, read-only plugin credentials, workflow engine index, workflow builder/API-run/manual-run, webhook builder/manual-run, and the local plugin binary base path. That keeps staging/readiness runs from silently exercising only the desktop Chromium lane, skipping account/profile continuity, API docs/logs, portal management coverage, plugin permission coverage, workflow engine coverage, email-sequence dashboard coverage, or governance coverage, or ignoring moderate accessibility regressions. It does not require E2E_ALLOW_WORKFLOW_WEBHOOK_RUN=1; real webhook delivery stays operator-owned.

Subscription checkout success remains a separate strict opt-in because it requires an externally acquired Stripe checkout session return path. Normal readiness runs stay non-mutating and do not require E2E_TENANT_SUBSCRIPTION_SUCCESS_PATH. In an operator-approved lane that has a valid success return URL, enable the manual subscription_success dispatch input and provide subscription_success_path; the workflow exports E2E_REQUIRE_SUBSCRIPTION_SUCCESS=1 together with E2E_TENANT_SUBSCRIPTION_SUCCESS_PATH. The wrapper fails before Playwright starts if that path is missing, and the subscription success smoke fails instead of silently skipping if the configured path is unavailable or resolves to an invalid checkout session. The same E2E_REQUIRE_SUBSCRIPTION_SUCCESS lane now also verifies reconciliation followthrough: after the checkout return renders, it follows the dashboard link, opens the subscription settings page, and asserts the reconciled active-subscription state survives both a fresh page reload and a full logout plus re-login (a fresh appData load). Because reconciliation is loader-side (pull-based), the reconciled state is read back through the real subscription settings loader rather than the success-page render alone, and the flow only navigates app routes (dashboard, settings, login/logout) so it never triggers a real Stripe network call. This stays a strict opt-in for the same external-blocker reason: it needs a real Stripe checkout return path, so the lane skips when E2E_REQUIRE_SUBSCRIPTION_SUCCESS is off and fails loudly when it is required but E2E_TENANT_SUBSCRIPTION_SUCCESS_PATH or tenant credentials are missing.

The static a11y coverage gate also enforces named browser readiness surfaces, not only raw file counts. pnpm run verify:a11y-coverage must keep explicit coverage for registration-to-dashboard, plugin settings, tenant API docs/logs, tenant API keys, audit logs, audit export, privacy self-service, tenant members, tenant notification settings, tenant chatbot settings, AI usage, workflow builder/run, portal management, admin email sequences, subscription smoke, authenticated mobile shell, authenticated mobile locale persistence, public auth, and auth-locale flows. The required files must contain the expected guard calls, not only exist: mobile/readiness flows need expectNoHorizontalOverflow, and critical user surfaces also need expectNoCriticalA11yViolations. The authenticated mobile locale flow must switch locales, reload the dashboard, navigate through the mobile shell to profile settings, and restore the original locale in finally. Registration/account readiness also keeps a non-mutating mobile login redirect replay for /new-account, verifies the account creation form contract is present, and returns to the seeded dashboard without creating another tenant. The public mobile surface is part of that named coverage: landing, legal, error, feature-disabled, and footer selector flows must cover narrow and 320px compact viewports, no horizontal overflow, and critical axe checks.

The scheduled readiness workflows intentionally stay separate from main CI. They install Chromium and WebKit, enable the mobile browser matrix, require staging credentials via E2E_TENANT_EMAIL / E2E_TENANT_PASSWORD, require a read-only plugin user via E2E_PLUGIN_READONLY_EMAIL / E2E_PLUGIN_READONLY_PASSWORD, and run pnpm run verify:e2e-a11y with the non-mutating readiness REQUIRE_* flags set. Missing credentials, missing seeded paths, missing browsers, unreachable staging, or moderate accessibility issues fail the lane instead of producing a SKIP. Do not set SKIP_E2E_A11Y=1 in required readiness, auth, mutating readiness, or subscription-success lanes; the wrapper treats that combination as a configuration failure.