246 lines
10 KiB
Markdown
246 lines
10 KiB
Markdown
# Storage AV Scanning
|
|
|
|
Defense-in-depth model for user-uploaded files. Two layers:
|
|
|
|
- **Layer 1 — Magic-Bytes Pre-Upload Scan**. Synchronous, runs in the upload
|
|
request handler **before** the bytes are committed to the storage backend
|
|
(S3 / local). Validates the actual content type via signature detection and
|
|
rejects mismatches or disallowed types.
|
|
- **Layer 2 — ClamAV Post-Upload Scan**. Asynchronous BullMQ job that runs
|
|
after the object is in the bucket. Hits trigger server-side quarantine
|
|
(CopyObject + DeleteObject into a `quarantine/` prefix).
|
|
|
|
## How it works
|
|
|
|
The `MagicBytesScanAdapter` in `@toolctrl/storage` reads the first ~4100 bytes
|
|
of an upload buffer and resolves its MIME type via the `file-type` library's
|
|
signature database. It then validates the detected type against:
|
|
|
|
1. The client-declared `Content-Type` (rejects mismatches — common attack
|
|
vector: an `.exe` smuggled as `image/png`).
|
|
2. A per-context allow-list (rejects e.g. a PDF uploaded into the avatar
|
|
context — see allow-list table below).
|
|
3. The filename — control characters (`\x00-\x1f`, `\x7f`) and path traversal
|
|
sequences (`../`, `..\\`) are rejected; filenames over 255 chars are
|
|
rejected.
|
|
|
|
## Allow-list defaults
|
|
|
|
See `packages/storage/src/scan/allow-list.ts` for the canonical list.
|
|
|
|
| Context | Allowed MIME types |
|
|
|---|---|
|
|
| `avatar` | image/png, image/jpeg, image/webp |
|
|
| `mediaLibrary` | + image/gif, image/svg+xml, image/x-icon, image/vnd.microsoft.icon |
|
|
| `emailAttachment` | image/{png,jpeg,webp,gif} + application/pdf, text/plain, text/csv, application/zip, Office .docx/.xlsx/.pptx |
|
|
| `formUpload` | image/{png,jpeg,webp} + application/pdf |
|
|
| `kbAsset` | image/{png,jpeg,webp} + application/pdf |
|
|
|
|
Per-context override via env: `STORAGE_SCAN_ALLOWLIST_<CONTEXT>` accepts a
|
|
comma-separated list of MIME types and **replaces** the default for that
|
|
context (not merged). Contexts in env-var form: `AVATAR`, `MEDIA_LIBRARY`,
|
|
`EMAIL_ATTACHMENT`, `FORM_UPLOAD`, `KB_ASSET`.
|
|
|
|
Example: allow only PNG and JPEG for tenant avatars in production:
|
|
|
|
```
|
|
STORAGE_SCAN_ALLOWLIST_AVATAR=image/png,image/jpeg
|
|
```
|
|
|
|
## Notfall-Bypass
|
|
|
|
Set `STORAGE_SCAN_LAYER1_ENABLED=false` to disable scanning entirely (the
|
|
adapter returns `status: "skipped"` for every call). Intended for emergency
|
|
incident response only — do not run with scanning disabled in production
|
|
for any extended period.
|
|
|
|
## How callers wire it in
|
|
|
|
`storeFile` from `apps/web/app/utils/integrations/storageService.ts` accepts
|
|
an optional `scan` parameter:
|
|
|
|
```ts
|
|
import { storeFile } from "~/utils/integrations/storageService";
|
|
import { auditRejectedUpload } from "~/utils/integrations/storageAudit.server";
|
|
|
|
await storeFile({
|
|
bucket: "users-icons",
|
|
content: avatarDataUrl,
|
|
id: userId,
|
|
scan: {
|
|
context: "avatar",
|
|
filename: `${userId}.avatar`,
|
|
onReject: auditRejectedUpload(request, "tenant.profile.avatar"),
|
|
},
|
|
});
|
|
```
|
|
|
|
A rejected scan throws `StorageScanRejectedError` from `storageService.ts`.
|
|
The error carries the bucket, id, filename and full `PreUploadScanResult`
|
|
(status, reason, detail, detectedMimeType). Callers can catch it and map
|
|
to a 415 response with a user-friendly message.
|
|
|
|
The `onReject` callback fires before the error throws and is awaited, so the
|
|
audit record is persisted even when the route swallows the error.
|
|
|
|
## Audit events
|
|
|
|
`auditRejectedUpload(request, context)` writes a `createAdminLog`-style row
|
|
with action `upload.rejected.scan`. The serialised detail field contains:
|
|
|
|
```json
|
|
{
|
|
"context": "tenant.profile.avatar",
|
|
"reason": "mime_mismatch",
|
|
"detail": "declared=\"image/png\", detected=\"application/x-msdownload\"",
|
|
"detectedMimeType": "application/x-msdownload"
|
|
}
|
|
```
|
|
|
|
The buffer content is never written to the audit log. Only metadata
|
|
(bounded reason enum + adapter-level diagnostic string + detected MIME).
|
|
|
|
## Currently-scanned upload paths
|
|
|
|
| Route | Context | Buckets |
|
|
|---|---|---|
|
|
| `apps/web/app/routes/admin/settings/general.tsx` | `mediaLibrary` | branding (logo, dark logo, icon, dark icon, favicon) |
|
|
| `apps/web/app/routes/admin/settings/profile.tsx` | `avatar` | users-icons |
|
|
| `apps/web/app/routes/app.$tenant/settings/profile.tsx` | `avatar` | users-icons |
|
|
| `apps/web/app/routes/app.$tenant/settings/account.tsx` | `avatar` | accounts-icons |
|
|
| `apps/web/app/routes/app.$tenant/__portals/portals.$portal/settings.tsx` | `mediaLibrary` | branding (5), seo (2) |
|
|
| `apps/web/app/routes/app.$tenant/__widgets/widgets/$id.tsx` | `mediaLibrary` | widgets |
|
|
|
|
Any new upload route that takes user-provided file bytes **must** opt into
|
|
the scan by passing a `scan` parameter. Server-internal uploads (e.g. seed
|
|
data, automated reports) may skip it because the bytes never came from a
|
|
user — but this is the only legitimate reason to omit `scan`.
|
|
|
|
## Limits & known gaps
|
|
|
|
- **Polyglot files** (a valid JPEG with an embedded JS payload exploiting an
|
|
ImageMagick CVE) pass Layer 1 because the magic-bytes signature is still
|
|
a valid JPEG. Layer 2 (`#17`) closes this with content scanning.
|
|
- **SVG XSS**: SVG is permitted in the `mediaLibrary` context. Files
|
|
containing `<script>` tags can execute when rendered in a permissive
|
|
context. CSP headers + DOMPurify on the render path are the right
|
|
mitigation; sanitisation is out of scope for this task — track separately.
|
|
- **Text/plain and CSV** cannot be detected from magic bytes (no signature).
|
|
The scanner accepts them when the declared MIME matches `text/plain` or
|
|
`text/csv` AND the allow-list permits — this is a known weak point.
|
|
- **Rate limiting** on upload endpoints is the responsibility of the route
|
|
itself. The scanner does not gate request rates.
|
|
|
|
---
|
|
|
|
# Layer 2 — ClamAV Post-Upload Scan
|
|
|
|
## Components
|
|
|
|
- **ClamAV container**: `clamav/clamav:stable` in `docker-compose.yml` with a
|
|
named volume for the signature DB. First boot pulls ~250 MB of signatures
|
|
via freshclam — the health check's `start_period` is set to 1200 s to
|
|
cover that. Subsequent boots pass in under 2 s.
|
|
- **`createClamAVStorageScanner(...)`** in `@toolctrl/storage` —
|
|
StorageScanner implementation. Talks to clamd over TCP via the `clamscan`
|
|
npm package. Pings before every scan; throws `ClamAVUnreachableError`
|
|
when clamd is unreachable.
|
|
- **`createS3StorageQuarantineAdapter(...)`** — StorageQuarantineAdapter.
|
|
Server-side `CopyObjectCommand` into `quarantine/<key>` (or a dedicated
|
|
`STORAGE_QUARANTINE_BUCKET`), then `DeleteObjectCommand` on the source.
|
|
Writes `scan-signature`, `scan-quarantined-at`, `scan-source-bucket` and
|
|
`scan-source-key` as S3 metadata so an admin restore action has a full
|
|
back-pointer.
|
|
- **`createWorkerScanService(env)`** in `apps/worker/src/scanService.ts` —
|
|
wires the scanner + quarantine adapter together via
|
|
`createStorageScanService` from `@toolctrl/storage`. Returns `undefined`
|
|
when STORAGE credentials are missing; the worker handler then short-
|
|
circuits the scan-file job rather than running blind.
|
|
- **`enqueueFileScan(...)`** in `apps/web/app/utils/integrations/
|
|
fileScanQueue.server.ts` — producer-side helper that opens a
|
|
`TypedQueue<typeof queueNames.fileScan>`, enqueues a `scan-file` job
|
|
built from `buildFileScanJob`, and closes the queue.
|
|
|
|
## Enabling Layer 2
|
|
|
|
Layer 2 is **opt-in** via two env knobs:
|
|
|
|
- `SCAN_QUEUE_ENABLED=true` on both the web and worker processes. On the
|
|
web side the producer helper short-circuits without it; on the worker
|
|
side `getWorkerQueueNames` appends `toolctrl.file-scan` only when set.
|
|
- `CLAMAV_HOST` (default `clamav`), `CLAMAV_PORT` (default `3310`),
|
|
`CLAMAV_TIMEOUT_MS` (default `30000`). Defaults match the compose
|
|
service. Override for staging/prod when ClamAV lives on a separate host.
|
|
- `STORAGE_QUARANTINE_BUCKET` (optional) — dedicated bucket for infected
|
|
objects. Defaults to the source bucket with the `quarantine/` prefix.
|
|
|
|
Existing storage env (`STORAGE_REGION`, `STORAGE_ACCESS_KEY_ID`,
|
|
`STORAGE_SECRET_ACCESS_KEY`, etc.) is reused for both the scanner's GET
|
|
path and the quarantine adapter.
|
|
|
|
## How callers wire it in
|
|
|
|
`storeFile`'s `scan` option already learnt about Layer 1; for Layer 2 set
|
|
`enqueuePostScan: true`:
|
|
|
|
```ts
|
|
await storeFile({
|
|
bucket: "users-icons",
|
|
content: avatarDataUrl,
|
|
id: userId,
|
|
scan: {
|
|
context: "avatar",
|
|
filename: `${userId}.avatar`,
|
|
onReject: auditRejectedUpload(request, "tenant.profile.avatar"),
|
|
enqueuePostScan: true,
|
|
tenantId,
|
|
},
|
|
});
|
|
```
|
|
|
|
After a successful PUT, `storeFile` dynamically imports
|
|
`fileScanQueue.server` and enqueues. Failures to enqueue are swallowed (a
|
|
queue write hiccup must not block the synchronous user-visible upload);
|
|
they should be picked up by Sentry once #29 lands.
|
|
|
|
## When ClamAV is down
|
|
|
|
The worker's scan-file handler routes through `createStorageScanService`
|
|
which wraps the scanner's `ClamAVUnreachableError` into a `StorageScanResult`
|
|
with `verdict: "error"`. The job is retried per BullMQ's default policy; if
|
|
the outage persists, the job ends up in the failed-jobs DB row written by
|
|
the existing `recordQueueFailedJob` path. The synchronous upload remains
|
|
unaffected — fail-open by design.
|
|
|
|
When clamd is reachable but the signature DB is stale (freshclam hasn't
|
|
run for >24 h), the operator should restart the container or trigger
|
|
freshclam manually. The next task (`#29` Sentry self-host) gives us
|
|
alerting on signature staleness.
|
|
|
|
## EICAR smoke test
|
|
|
|
The standard EICAR test file (`X5O!P%@AP[4\PZX54(P^)7CC)7}$EICAR-...`) is
|
|
recognised by every AV engine including ClamAV. Use it for staging
|
|
verification:
|
|
|
|
1. Upload an `eicar.txt` (renamed to `.png` and re-typed to `image/png`
|
|
so it passes layer 1) into the media library.
|
|
2. Within ~5 s the worker should run the scan-file job, get an "infected"
|
|
verdict with signature `Win.Test.EICAR_HDB-1`, and quarantine the
|
|
object.
|
|
3. Verify: `s3://<source-bucket>/<key>` no longer exists; the corresponding
|
|
`quarantine/<key>` does, with the `scan-signature` metadata header set.
|
|
|
|
## Known limits
|
|
|
|
- **DB tracking is deferred.** The `media.scanStatus` / `media.scanResult`
|
|
columns from the original detail-MD are not added yet. Today the only
|
|
durable signal that a file was quarantined is the absence of the
|
|
source-key + the presence of the quarantine copy + the S3 metadata. A
|
|
follow-up commit will add the columns + the UI badge.
|
|
- **User notification on quarantine is deferred.** No email is sent today.
|
|
Operator can watch the worker logs / S3 access logs.
|
|
- **Concurrency**: BullMQ defaults to a per-worker concurrency of 1.
|
|
Bursty uploads will queue. Raise via the standard worker config knobs
|
|
once production traffic justifies it.
|