191 lines
8.4 KiB
Markdown
191 lines
8.4 KiB
Markdown
# Lighthouse-CI audit and Web-Vitals reporting
|
|
|
|
This document describes how the Lighthouse-CI audit is wired into the
|
|
Toolctrl Boilerplate, how to run it locally when debugging a regression,
|
|
and how the Web-Vitals → Sentry bridge surfaces real-user metrics.
|
|
|
|
Spec: `docs/improvement-plan/10-compliance-polish/lighthouse.md`
|
|
(Task #70). Companion artefacts: `.lighthouserc.staging.json`,
|
|
`.lighthouserc.json`, `scripts/lighthouse-puppeteer-auth.cjs`,
|
|
`scripts/rotate-lighthouse-session-cookie.mjs`,
|
|
`.github/workflows/lighthouse.yml`, `.gitea/workflows/lighthouse.yml`,
|
|
`apps/web/app/lib/observability/web-vitals.ts`,
|
|
`apps/web/app/lib/pwa/registerServiceWorker.ts`,
|
|
`apps/web/public/manifest.webmanifest`, `apps/web/public/sw.js`.
|
|
|
|
## Where the audit runs
|
|
|
|
A meaningful Lighthouse audit needs a live server, a database and seeded
|
|
content. Reproducing that whole stack inside a CI job is brittle, so the
|
|
CI workflow audits the **deployed staging site**
|
|
(`https://boiler.saas-ctrl.com`) rather than a CI-local server.
|
|
|
|
That makes it a **post-deploy / monitoring audit, not a per-commit merge
|
|
gate** — staging always lags the commit under review. The workflow
|
|
(`.gitea/workflows/lighthouse.yml`, mirrored in `.github/`) runs:
|
|
|
|
- **nightly** at 03:00 UTC (`schedule`), and
|
|
- **on demand** (`workflow_dispatch`).
|
|
|
|
A red run means the deployed staging site fell below a threshold — treat
|
|
it as a production-quality alarm, not a blocked PR.
|
|
|
|
The Gitea runner image (`node:20-bookworm`) ships no browser, so the
|
|
workflow installs `chromium` via `apt-get` and points LHCI at it with
|
|
`CHROME_PATH=/usr/bin/chromium`. The GitHub runner (`ubuntu-latest`)
|
|
already bundles Chrome.
|
|
|
|
## Audit targets
|
|
|
|
`.lighthouserc.staging.json` audits the public staging routes — the
|
|
surface that renders without authentication:
|
|
|
|
| URL |
|
|
| -------------------------------------------- |
|
|
| `https://boiler.saas-ctrl.com/login` |
|
|
| `https://boiler.saas-ctrl.com/privacy-policy` |
|
|
| `https://boiler.saas-ctrl.com/terms-and-conditions` |
|
|
|
|
Only **Performance** is a hard gate (`error`). The product is a B2B
|
|
SaaS, not a search-discoverable marketing site, so the SEO category is
|
|
not asserted at all. Accessibility and Best-Practices stay visible as
|
|
`warn` — accessibility is already hard-gated separately by `a11y.yml`
|
|
(axe-core plus the `jsx-a11y` ESLint baseline, task #73), so a second
|
|
blocking gate here would be redundant.
|
|
|
|
| Category | Threshold | Severity |
|
|
| --------------- | --------- | ------------------------- |
|
|
| Performance | 0.95 | error |
|
|
| Accessibility | 0.95 | warn |
|
|
| Best Practices | 0.95 | warn |
|
|
| PWA | 0.90 | warn |
|
|
| SEO | — | not asserted (irrelevant) |
|
|
|
|
Numeric audits (FCP / LCP / TBT / CLS) stay at `warn` severity so trend
|
|
regressions remain visible without failing the run.
|
|
|
|
## Session cookie rotation
|
|
|
|
The workflow mints a fresh staging session before every LHCI run via
|
|
`scripts/rotate-lighthouse-session-cookie.mjs`. The script logs in as the
|
|
dedicated `lighthouse-bot` account, extracts the CSRF token from
|
|
`/login`, posts the login form with the CSRF header, then appends only
|
|
the resulting session cookie to the Actions environment. It never prints
|
|
the cookie or password.
|
|
|
|
Required workflow secrets:
|
|
|
|
| Secret | Purpose |
|
|
| ------ | ------- |
|
|
| `LIGHTHOUSE_LOGIN_EMAIL` | Email of the staging Lighthouse bot user. |
|
|
| `LIGHTHOUSE_LOGIN_PASSWORD` | Password of the staging Lighthouse bot user. |
|
|
|
|
The session cookie name is `RJ_session`; workflows also pass
|
|
`LIGHTHOUSE_SESSION_COOKIE_NAME=RJ_session` and
|
|
`LIGHTHOUSE_COOKIE_DOMAIN=boiler.saas-ctrl.com`.
|
|
|
|
During rollout the workflow still forwards the older
|
|
`LIGHTHOUSE_SESSION_COOKIE` secret as a fallback. If the login email or
|
|
password secret is absent, the script reuses that pre-provisioned cookie
|
|
without printing it. Permanent operation should use the login secrets so
|
|
each run mints a fresh cookie.
|
|
|
|
## Authenticated routes
|
|
|
|
Auditing `/app/$tenant/*` and `/admin/*` needs that freshly minted
|
|
`RJ_session` cookie. `scripts/lighthouse-puppeteer-auth.cjs` injects the
|
|
cookie for protected URLs and is already wired into
|
|
`.lighthouserc.staging.json`. The current staging target list still
|
|
audits only public routes; add protected URLs once the exact tenant/admin
|
|
coverage list is chosen. Without a protected URL in the config, the
|
|
cookie step is a readiness/rotation guard rather than an auth coverage
|
|
expansion.
|
|
|
|
## Running Lighthouse locally
|
|
|
|
`.lighthouserc.json` is the local-iteration config (URLs on
|
|
`http://localhost:3000`); `pnpm verify:lighthouse` runs it via
|
|
`scripts/run-lighthouse.mjs`. Bring the app up first:
|
|
|
|
```sh
|
|
# Build apps/web (writes hashed assets the runtime SW expects to cache)
|
|
pnpm --filter @toolctrl/web build
|
|
|
|
# Serve the build at http://localhost:3000.
|
|
pnpm --filter @toolctrl/web start &
|
|
|
|
# Run the full LHCI flow against the local server
|
|
pnpm verify:lighthouse
|
|
```
|
|
|
|
To reproduce a CI staging finding locally, point LHCI at the staging
|
|
config instead — no local server needed, just a Chrome install:
|
|
|
|
```sh
|
|
pnpm dlx @lhci/cli@0.13.x autorun --config=./.lighthouserc.staging.json
|
|
```
|
|
|
|
The run prints the temporary-public-storage report URLs in its output;
|
|
share those when debugging a regression.
|
|
|
|
## PWA manifest + service-worker
|
|
|
|
`apps/web/public/manifest.webmanifest` describes the installable app.
|
|
`apps/web/public/sw.js` is a minimal cache-first / network-first service
|
|
worker registered from `root.tsx` via `registerServiceWorker`.
|
|
|
|
Cache strategy:
|
|
|
|
- `/assets/...` (hashed by React-Router build) → **cache-first**
|
|
- `/api/...`, `/app/...`, `/admin/...` → **network-first** with cache fallback
|
|
- everything else (marketing routes, brand assets) → **network-first**
|
|
|
|
The `CACHE_VERSION` constant inside `sw.js` is the invalidation lever —
|
|
bump it on any deploy that changes the asset graph in a backward-
|
|
incompatible way. Old caches are dropped automatically in the activate
|
|
handler.
|
|
|
|
To debug a stuck SW, open DevTools → Application → Service Workers and
|
|
click "Unregister". The next page load picks up the new `sw.js`.
|
|
|
|
## Web-Vitals → Sentry
|
|
|
|
`apps/web/app/lib/observability/web-vitals.ts` registers callbacks for
|
|
CLS, LCP, INP, TTFB, and FCP via the `web-vitals` library. Each metric is
|
|
captured as a Sentry message named `web-vitals.<metric>` with tags:
|
|
|
|
```text
|
|
web.vital.name = CLS | LCP | INP | TTFB | FCP
|
|
web.vital.rating = good | needs-improvement | poor
|
|
```
|
|
|
|
In the Sentry dashboard:
|
|
|
|
1. Filter to `message:"web-vitals.LCP"` to view a metric in isolation.
|
|
2. Group by `tags.web.vital.rating` to see the share of "poor" loads.
|
|
3. Group by `transaction` (Sentry React-Router integration) to compare
|
|
metrics across routes.
|
|
|
|
The helper is SSR-safe (bails on `typeof window === "undefined"`) and
|
|
silently no-ops when Sentry isn't initialised, so the same code runs in
|
|
dev, preview, staging, and production without extra gating.
|
|
|
|
## Troubleshooting CI runs
|
|
|
|
| Symptom | Likely cause / fix |
|
|
| -------------------------------------------------- | ----------------------------------------------------------------------------- |
|
|
| `Chrome installation not found` / `Healthcheck failed!` | The runner has no browser. The Gitea workflow installs `chromium`; confirm the install step ran and `CHROME_PATH` is set. |
|
|
| Cookie rotation step fails with missing env vars | Add `LIGHTHOUSE_LOGIN_EMAIL` and `LIGHTHOUSE_LOGIN_PASSWORD` as repo Actions secrets. |
|
|
| Audit reports the `/login` page for a protected URL | Cookie injection failed or the bot no longer belongs to the tenant/admin surface being audited. |
|
|
| Performance score < 0.95 only on first run | Cold cache; LHCI reruns automatically (`numberOfRuns=3`). |
|
|
| Audit times out | Lower `numberOfRuns` to 1 temporarily while iterating; restore afterwards. |
|
|
| All URLs fail to load | Staging is down or redeploying — check the `web` container on staging. |
|
|
|
|
## Updating thresholds
|
|
|
|
Permanent relaxation requires a `.lighthouserc.staging.json` commit plus
|
|
a paragraph in this document explaining the trade-off. Prefer switching
|
|
a category from `error` to `warn` over dropping it, so regressions stay
|
|
visible — SEO is the one deliberate exception (not relevant to a B2B
|
|
SaaS, so not asserted at all).
|