HiveGrid-Wireframes/hivegrid-cloud/docs/production-checklist.md

Production Checklist

Production readiness

Use this checklist before an operator records a boilerplate release as production-ready. It connects release evidence from docs/release-checklist.md, staging and UAT readiness from docs/staging-live-uat.md, downstream baseline evidence from docs/boilerplate-version-tracking.md, deploy boundaries from docs/deployment.md, database operations from docs/database.md, environment ownership from docs/env-vars.md, security handling from docs/security.md, observability guidance from docs/observability.md, worker operations from docs/worker.md, mail operations from docs/mail.md, upgrade planning from docs/upgrade.md, and release-entry rules from docs/releases/README.md.

Production readiness is a documented decision, not proof that live infrastructure was exercised. A complete record names the release or source version label, changelog alignment, release artifact label, deployment target labels, environment readiness status, rollback class, redaction review, and any manual runtime records that a human explicitly approved and performed.

Scope

This checklist covers release-operator readiness for the web, API, worker, database, mail provider, deployment image set, documentation roots, and incident handoff. It may be used for a production release, a release candidate, or a downstream boilerplate adoption record when the record needs production-facing evidence boundaries.

Out of scope for local documentation-readiness verification are ad hoc live Docker builds, registry pushes, Coolify execution, provider-console inspection, browser journeys, Playwright review, Cypress review, and production traffic checks. The configured Gitea deploy workflow may perform Docker, registry, Coolify, migration, and health steps automatically; production records should cite that workflow evidence by safe job labels, target labels, immutable tags, status labels, and redacted correlation identifiers.

Environment and configuration readiness

Review docs/env-vars.md before marking configuration ready.

• Required key names for the selected runtime surfaces are listed by name only.
• DATABASE_URL, DIRECT_URL, SESSION_SECRET, CRYPTO_SECRET, Redis keys, Mailjet keys, browser runtime keys, registry variables, and Coolify slots are referenced only as key names or placeholder labels.
• Missing, present, enabled, disabled, parsed, or invalid statuses are recorded without raw values.
• The release record avoids environment dumps, connection strings, endpoint values, customer identifiers, tenant identifiers, cookies, bearer tokens, and credential fragments.
• Optional integrations state whether they are disabled, deferred, or manually verified through an approved runtime path.

Database and migration readiness

Review docs/database.md and docs/upgrade.md for migration-sensitive releases.

• The release record states whether database changes are absent, documentation-only, migration-sensitive, backup-dependent, or irreversible.
• Migration evidence names command labels, migration registry identifiers, phase labels, exit status labels, and redacted diagnostic codes only.
• Backup or restore-point readiness is recorded as a manual operator status when production data can be affected.
• Local-only Prisma authoring commands are not presented as production deployment commands.
• Rollback notes explain whether a previous image redeploy is enough or whether backup restore, queue control, or forward-fix ownership is required.

Deployment and image readiness

Review docs/deployment.md for image target and deploy workflow boundaries.

• The deployable image targets are identified with safe labels web, api, and worker; deployed worker resources are named separately as worker-general, worker-mail, worker-import, worker-webhook, and worker-plugin per docs/ops/worker-queues.md.
• Artifact evidence uses immutable release tag, commit SHA label, image target label, digest label, or build job label.
• Image references stay placeholder-shaped, such as {registry}/{namespace}/{web|api|worker}:{tag}.
• Local automatic documentation verification is limited to static checks such as pnpm run verify:deploy-workflow and relevant focused non-E2E checks that already exist in package.json.
• Gitea deploy workflow evidence for Docker, registry, Coolify, migration, and health steps is recorded only with safe labels and redacted correlation identifiers.
• Ad hoc live Docker, registry, Coolify, deployment-provider, and production traffic checks outside the configured deploy workflow are recorded only as manual activities requiring explicit human approval.

Health and readiness checks

Review docs/observability.md before copying health or readiness evidence.

• Web health evidence is limited to route label, status label, target label, timestamp label, and redacted dependency code when applicable.
• API health evidence is limited to endpoint label, status, x-request-id, JSON requestId, service label, and stable error code.
• Readiness failures name dependency category and redacted diagnostic code rather than connection details.
• Health checks do not include cookies, authorization headers, customer pages, browser screenshots, full response bodies with user data, or tenant hostnames.
• Runtime health inspection against deployed targets is a manual record unless a human approved the exact focused run.

Worker and queue readiness

Review docs/worker.md for worker enablement and queue diagnostics.

• Worker readiness states whether queue processors are disabled, enabled for a target, paused, draining, retrying, or healthy.
• Persistent worker evidence is recorded separately for each required group:

Worker app	Required group	Queue list evidence	Revision evidence	Execute-health evidence
worker-general	WORKER_QUEUE_GROUP=general	redacted label	redacted label	redacted label
worker-mail	WORKER_QUEUE_GROUP=mail	redacted label	redacted label	redacted label
worker-import	WORKER_QUEUE_GROUP=import	redacted label	redacted label	redacted label
worker-webhook	WORKER_QUEUE_GROUP=webhook	redacted label	redacted label	redacted label
worker-plugin	WORKER_QUEUE_GROUP=plugin	redacted label	redacted label	redacted label

• Queue evidence uses queue names, job type labels, job IDs, status labels, retry counts, failed-job counts, timestamps, and redacted dependency codes.
• Redis evidence names selected key labels such as BULLMQ_REDIS_URL or fallback REDIS_URL, plus parse/readiness status only.
• Import submit and export artifact retention checks record final enqueue status, queue name, job ID, and worker pickup status before staging is called complete.
• Replay, drain, pause, dead-letter inspection, or provider-console actions are manual operational records.
• Raw job payloads, Redis URLs, database URLs, provider payloads, screenshots, tenant identifiers, and customer data are excluded.

Mail and provider readiness

Review docs/mail.md and docs/security.md before recording mail provider status.

• Outbound mail readiness names provider category, sender-domain status, delivery status class, queue/job label, and request correlation label.
• Webhook readiness names signing verification status, header class labels, replay-window status, and redacted authentication result.
• Mailjet keys are referenced only as MAILJET_API_KEY, MAILJET_FROM_EMAIL, MAILJET_FROM_NAME, and MAILJET_WEBHOOK_SECRET.
• Provider-console inspection, test sends, webhook tests, and delivery dashboard checks are manual records.
• Provider payloads, email addresses from customers, message bodies, signatures, API keys, signing secrets, and dashboard screenshots with personal data are excluded.

Security, CORS, and headers readiness

Review docs/security.md for the security posture boundary.

• Header evidence names the header family, expected policy label, response status, route class, and request ID when available.
• CORS evidence records allowed or denied origin labels, header presence, status, and request ID without raw customer domains.
• Authentication incidents record method, endpoint label, status, stable error code, and request ID without credentials.
• Redaction review confirms that no tickets, release notes, logs, screenshots, or examples include secrets, provider payloads, tenant identifiers, customer data, registry credentials, webhook values, bearer tokens, cookies, or secret-shaped strings.
• Security exceptions are linked to follow-up ownership rather than silently weakening runtime policy.

Observability and incident triage

Use docs/observability.md as the incident handoff source.

• The handoff identifies the affected surface: web, API, worker, database, Redis, Mailjet, browser runtime, deployment, or documentation verifier.
• Safe correlation values are included when available: request ID, job ID, queue name, target label, timestamp label, release label, or redacted diagnostic code.
• Incident status is summarized with pass, fail, degraded, retrying, paused, blocked, resolved, or owner-needed labels.
• Raw logs are summarized instead of pasted when they can contain secrets or customer data.
• Any accidental exposure is recorded with containment and rotation status, not with the exposed value.

Manual runtime checks

Manual runtime checks are allowed only as human-approved evidence records. Keep them separate from automatic command examples.

Manual-only activities include:

• Browser journey review.
• Playwright or Cypress review.
• Live Docker build or smoke review.
• Registry push confirmation.
• Coolify execution or deployment-provider inspection.
• Provider-console inspection for mail, Redis, database, or runtime providers.
• Production traffic or customer-impacting request review.

Each manual record should contain owner label, approval label, date label, target label, release or artifact label, status, safe request ID or job ID when available, and redacted outcome. Do not turn these manual activities into automatic checklist commands.

Deployment boundary

Automatic readiness proof for this documentation surface is local and static. It may verify Markdown roots, links, package scripts, deploy workflow shape, and focused non-E2E tests already present in package.json. It must not open browsers, start Playwright, start Cypress, contact providers, push images, invoke Docker, run Coolify, or inspect production traffic.

Deployment evidence should name safe labels only: release version, source version, immutable tag label, artifact label, build job label, image target label, health status label, migration registry ID, request ID, queue name, and redacted diagnostic code.

Rollback plan

Every production readiness record must choose a rollback class before release:

• Redeploy-only when a prior immutable artifact or tag can be restored without data changes.
• Migration-sensitive when database, queue, configuration, provider, or security changes require operator sequencing.
• Backup-dependent when rollback requires restore-point or backup evidence.
• Forward-fix when rollback is unsupported after irreversible migration or external state changes.

Rollback notes should link to the relevant release entry, docs/upgrade.md, and any migration registry evidence identified by docs/database.md. Production checklist completion is not rollback proof by itself; rollback proof requires immutable artifact, tag, backup, queue-control, or operator evidence.

Redacted release evidence

Safe production evidence includes command names, pass or fail labels, exit code labels, release-entry paths, changelog labels, artifact labels, tag labels, image target labels, migration registry IDs, health status labels, request IDs, job IDs, queue names, and redacted diagnostic codes.

Unsafe evidence must be removed before the record is committed or shared: secrets, credential-bearing URLs, UUID-like deploy endpoints, tenant identifiers, customer identifiers, provider payloads, raw environment values, tokens, passwords, cookies, database connection strings, Redis connection strings, registry credentials, webhook values, screenshots containing sensitive data, and full logs that can contain private data.

Verification

Supported automatic verification for this production checklist and the S04 release-documentation roots is local, static, and non-E2E:

pnpm run verify:release-docs
pnpm run verify:staging-live-uat-readiness
pnpm run verify:docs-links
pnpm run verify:deploy-workflow

Relevant focused non-E2E checks that already exist in package.json may also be recorded when they directly cover the changed surface. Record command name, exit code, checked surface, and a short pass or fail summary only.

The release-documentation verifier rejects missing required sections, unresolved local links, forbidden placeholder markers, automatic browser or E2E command recommendations, and secret-shaped example values. Manual runtime evidence is reviewed separately and must use redacted labels rather than raw operational details.