Refresh HiveGrid Cloud reference snapshot
This commit is contained in:
1 parent
5e99e14772
commit
7f88026f79
5955 files changed
+1177325
-188362
No files matched your search
@@ -6,9 +6,10 @@ Defense-in-depth model for user-uploaded files. Two layers:
|
||||
request handler **before** the bytes are committed to the storage backend
|
||||
(S3 / local). Validates the actual content type via signature detection and
|
||||
rejects mismatches or disallowed types.
|
||||
- **Layer 2 — ClamAV Post-Upload Scan**. Asynchronous BullMQ job that runs
|
||||
after the object is in the bucket. Hits trigger server-side quarantine
|
||||
(CopyObject + DeleteObject into a `quarantine/` prefix).
|
||||
- **Layer 2 — ClamAV malware scan**. Directly served assets such as avatars
|
||||
and branding are scanned synchronously before the storage PUT. File-module
|
||||
uploads remain private and non-deliverable while an asynchronous BullMQ job
|
||||
scans them; hits trigger server-side quarantine.
|
||||
|
||||
## How it works
|
||||
|
||||
@@ -70,6 +71,7 @@ await storeFile({
|
||||
scan: {
|
||||
context: "avatar",
|
||||
filename: `${userId}.avatar`,
|
||||
requireMalwareScan: true,
|
||||
onReject: auditRejectedUpload(request, "tenant.profile.avatar"),
|
||||
},
|
||||
});
|
||||
@@ -133,11 +135,11 @@ user — but this is the only legitimate reason to omit `scan`.
|
||||
|
||||
---
|
||||
|
||||
# Layer 2 — ClamAV Post-Upload Scan
|
||||
# Layer 2 — ClamAV Malware Scan
|
||||
|
||||
## Components
|
||||
|
||||
- **ClamAV container**: `clamav/clamav:stable` in `docker-compose.yml` with a
|
||||
- **ClamAV container**: `clamav/clamav:stable@sha256:…` in `docker-compose.yml` with a
|
||||
named volume for the signature DB. First boot pulls ~250 MB of signatures
|
||||
via freshclam — the health check's `start_period` is set to 1200 s to
|
||||
cover that. Subsequent boots pass in under 2 s.
|
||||
@@ -168,9 +170,10 @@ Layer 2 is **opt-in** via two env knobs:
|
||||
- `SCAN_QUEUE_ENABLED=true` on both the web and worker processes. On the
|
||||
web side the producer helper short-circuits without it; on the worker
|
||||
side `getWorkerQueueNames` appends `hivegrid.file-scan` only when set.
|
||||
- `CLAMAV_HOST` (default `clamav`), `CLAMAV_PORT` (default `3310`),
|
||||
`CLAMAV_TIMEOUT_MS` (default `30000`). Defaults match the compose
|
||||
service. Override for staging/prod when ClamAV lives on a separate host.
|
||||
- `CLAMD_HOST`, `CLAMD_PORT` (default `3310`) and `CLAMD_TIMEOUT_MS`
|
||||
(default `30000`) configure synchronous pre-PUT scans. The legacy
|
||||
`CLAMAV_*` names remain accepted for the worker/compose path. Production
|
||||
uses the mesh-only central ClamAV endpoint from `infra/STANDARDS.md`.
|
||||
- `STORAGE_QUARANTINE_BUCKET` (optional) — dedicated bucket for infected
|
||||
objects. Defaults to the source bucket with the `quarantine/` prefix.
|
||||
|
||||
@@ -180,8 +183,8 @@ path and the quarantine adapter.
|
||||
|
||||
## How callers wire it in
|
||||
|
||||
`storeFile`'s `scan` option already learnt about Layer 1; for Layer 2 set
|
||||
`enqueuePostScan: true`:
|
||||
`storeFile`'s `scan` option already handles Layer 1. Directly served uploads
|
||||
must additionally set `requireMalwareScan: true`:
|
||||
|
||||
```ts
|
||||
await storeFile({
|
||||
@@ -192,25 +195,25 @@ await storeFile({
|
||||
context: "avatar",
|
||||
filename: `${userId}.avatar`,
|
||||
onReject: auditRejectedUpload(request, "tenant.profile.avatar"),
|
||||
enqueuePostScan: true,
|
||||
tenantId,
|
||||
requireMalwareScan: true,
|
||||
},
|
||||
});
|
||||
```
|
||||
|
||||
After a successful PUT, `storeFile` dynamically imports
|
||||
`fileScanQueue.server` and enqueues. Failures to enqueue are swallowed (a
|
||||
queue write hiccup must not block the synchronous user-visible upload);
|
||||
they should be picked up by Sentry once #29 lands.
|
||||
`storeFile` sends the in-memory buffer to clamd before the storage PUT. An
|
||||
infected, unavailable, malformed or indeterminate scanner response rejects
|
||||
the upload. File-module objects use the asynchronous queue instead and stay
|
||||
at `scanStatus=pending`; media delivery and bulk ZIPs accept only `clean`.
|
||||
|
||||
## When ClamAV is down
|
||||
|
||||
The worker's scan-file handler routes through `createStorageScanService`
|
||||
Synchronous direct-asset uploads fail closed before storage. The worker's
|
||||
scan-file handler routes through `createStorageScanService`
|
||||
which wraps the scanner's `ClamAVUnreachableError` into a `StorageScanResult`
|
||||
with `verdict: "error"`. The job is retried per BullMQ's default policy; if
|
||||
the outage persists, the job ends up in the failed-jobs DB row written by
|
||||
the existing `recordQueueFailedJob` path. The synchronous upload remains
|
||||
unaffected — fail-open by design.
|
||||
the existing `recordQueueFailedJob` path. The object remains pending and
|
||||
cannot be downloaded while the scanner is unavailable.
|
||||
|
||||
When clamd is reachable but the signature DB is stale (freshclam hasn't
|
||||
run for >24 h), the operator should restart the container or trigger
|
||||
|
||||
Reference in new issue
Block a user