130 lines
10 KiB
Markdown
130 lines
10 KiB
Markdown
# Plugin system (Phase 06 #21 foundation)
|
||
|
||
> **Runtime gate:** This is staging/foundation runtime documentation, not
|
||
> production authorization. Production third-party plugin execution remains
|
||
> blocked until the external WASM sandbox review is signed off in
|
||
> [Plugin Runtime Signoff](../security/plugin-runtime-signoff.md).
|
||
|
||
This document describes the WASM-sandbox foundation shipped under Phase 06 Task #21. It captures the runtime choice, manifest schema, signature trust model, capability gating, rate-limit, audit handling, and the work that is intentionally deferred to follow-up tasks.
|
||
|
||
## Scope
|
||
|
||
What the foundation ships:
|
||
|
||
- **`@toolctrl/plugin-runtime`** — host-side runtime:
|
||
- Manifest schema (Zod) with closed-enum capabilities + reverse-DNS plugin IDs + SemVer versions + hex-encoded Ed25519 signature fields.
|
||
- Ed25519 signature verifier against the operator-controlled trusted-keys allowlist (`PLUGIN_TRUSTED_PUBLIC_KEYS` CSV).
|
||
- WebAssembly sandbox wrapper (`createPluginSandbox`) with deterministic memory cap + host-fn fuel budget, invoked through a killable worker-thread boundary with a wall-clock budget.
|
||
- Capability gateway (`createCapabilityGateway`) that wraps host-fn maps and rejects undeclared capability calls.
|
||
- **`@toolctrl/plugin-sdk`** — TS types plugin authors build against (`HookHandler`, `HostApi`, `PluginContext`, `defineHook`) and a `hello-world` example.
|
||
- **`Plugin` Prisma model + migration** — one row per installed plugin per tenant.
|
||
- **`plugin` worker queue group** with the `toolctrl.plugin-invoke` BullMQ queue.
|
||
- **`handlePluginInvokeJob`** worker handler (`apps/worker/src/jobs/plugin-invoke.ts`) that wires Plugin lookup → binary load → signature verify → rate-limit → sandbox invoke → audit.
|
||
|
||
What is **deferred** (tracked in `docs/improvement-plan/06-plugin-system/wasm-sandbox.md` under `## Deviations from Plan` once that file is updated by the executing agent):
|
||
|
||
- Hot-reload — operators restart the `worker-plugin` pod to pick up new binaries.
|
||
- Marketplace integration — backlog (#24).
|
||
- CLI build/sign/publish is shipped; see [Plugin CLI Reference](../plugins/cli-reference.md).
|
||
- Full guided hook-lifecycle UI (#22) — install/guided consent dialogs remain
|
||
deferred; tenant settings now supports list-level approval for missing
|
||
manifest-declared capability grants with explicit review confirmation and
|
||
stale-manifest digest checks.
|
||
- Audit + docs deep-dive (#23) — author docs and the capability matrix have
|
||
landed; remaining gaps are external audit sign-off, richer toolchain notes,
|
||
and high-risk host bindings beyond the current fail-closed surface.
|
||
- E2E testcontainer harness — staging deployment + manual smoke tests in the meantime.
|
||
- Worker-thread pool with epoch interruption — the current worker bootstrap uses one killable worker thread per invocation; a pooled runtime and true per-instruction interruption remain deferred.
|
||
|
||
Operational lifecycle status, worker topology, and current production gaps are maintained in [Plugin Lifecycle Operations](../operations/plugin-lifecycle.md). Product-facing status and user/admin behavior are maintained in [Plugins](../features/plugins.md).
|
||
|
||
## Runtime choice
|
||
|
||
**Default: Node 22's built-in `WebAssembly` (V8) with host-fn-driven fuel charging.**
|
||
|
||
Rationale: native Wasmtime bindings for Node (`@bytecodealliance/jco`, `@bytecodealliance/preview2-shim`) are still pre-1.0 and add a ~40 MB build asset. For the foundation we ship a portable wrapper around Node's built-in `WebAssembly` runtime — sufficient for hard memory limits via `WebAssembly.Memory { maximum }`, a host-fn fuel budget on capability calls, and a worker-thread wall-clock kill for non-yielding code.
|
||
|
||
**Upgrade path:** `PluginSandbox` is an abstract surface — `invoke`, `close`, `fuelRemaining`. A Wasmer or Wasmtime-jco backend can be dropped in behind the same contract once either offers a stable Component-Model story on Node. This is the documented escape hatch when richer per-instruction fuel becomes a requirement.
|
||
|
||
## Manifest schema
|
||
|
||
A plugin ships a `plugin.toml` (or `plugin.json`) alongside the `.wasm` binary. The runtime parses it via `parsePluginManifest`, which returns a discriminated `Result`:
|
||
|
||
```ts
|
||
{
|
||
schemaVersion: 1,
|
||
id: "de.toolctrl.hello", // reverse-DNS, lowercase
|
||
version: "1.0.0", // SemVer
|
||
name: "Hello",
|
||
entry: "plugin.wasm",
|
||
wasmSha256: "<64 hex chars>", // SHA-256 of the .wasm at signing
|
||
signature: "<128 hex chars>", // Ed25519 sig over the .wasm bytes
|
||
signedBy: "<64 hex chars>", // Ed25519 public key (32 bytes)
|
||
allowedHosts: ["api.example.com"], // outbound HTTP allowlist
|
||
capabilities: [
|
||
{ name: "log" },
|
||
{ name: "http.outbound", scope: { allowedHosts: ["api.example.com"] } },
|
||
],
|
||
hooks: [{ name: "onHello", export: "handleHello" }],
|
||
}
|
||
```
|
||
|
||
The schema is `strict()` — unknown top-level fields are rejected. Unknown capability names are rejected against the closed enum in `packages/plugin-runtime/src/capabilities.ts`.
|
||
|
||
## Signature trust model
|
||
|
||
- Ed25519 chosen for deterministic signatures, audited Node `libcrypto` implementation, and short 64-byte signatures.
|
||
- Operators seed `PLUGIN_TRUSTED_PUBLIC_KEYS` (CSV of hex-encoded 32-byte public keys). Plugins signed by any key outside this list are rejected with `public_key_not_trusted`.
|
||
- `verifyPluginSignature`:
|
||
1. SHA-256-hashes the `.wasm` bytes and compares against `manifest.wasmSha256` — fail-closed on mismatch.
|
||
2. Confirms `manifest.signedBy` is in the trusted-keys allowlist — fail-closed otherwise.
|
||
3. Verifies `manifest.signature` over the `.wasm` bytes using `signedBy` — fail-closed otherwise.
|
||
- All three checks must pass; the function returns a discriminated reason on failure and never throws.
|
||
|
||
## Capability gating
|
||
|
||
The capability gateway wraps a host-fn record so that calls to capabilities the plugin did not declare in its manifest, or that the tenant did not approve for this invocation, throw `CapabilityDeniedError`. The default fuel cost is 100 units per host-fn call; configurable per gateway. Deny paths (`not_declared_in_manifest`, `not_granted_for_tenant`, `no_host_implementation`) fire the `onDenied` callback so the worker can audit them. The plugin worker currently exposes safe synchronous `env.toolctrl_log(i32)` and `env.toolctrl_audit_record(i32)` imports through `invokeSync(...)`; higher-risk host functions remain absent and fail closed.
|
||
|
||
## Rate limit
|
||
|
||
The worker handler consumes a per-tenant + per-plugin sliding-window rate limit before invoking the sandbox. The limit's parameters (window size, max calls) are owned by the worker bootstrap and read from env. The handler's `consumeRateLimit` seam returns `{ allowed, remaining }`; the handler short-circuits to `rate_limit_exceeded` on a deny.
|
||
|
||
## Audit
|
||
|
||
`handlePluginInvokeJob` always writes an audit row, even on failure. Invocation audit persistence is fail-closed: if the audit write fails, the handler logs `plugin.invoke.audit_failed` and throws so BullMQ can retry/dead-letter instead of acknowledging a plugin invocation with no forensic trail. The audit row carries the tuple `(tenantId, pluginId, version, hookName, outcome)`.
|
||
|
||
Capability-use telemetry is batched through `PluginCapabilityUseRecorder`. The production plugin worker enables the recorder's fail-closed mode; if the outbox sink is unavailable, the sandbox invocation fails and the plugin invocation is recorded as failed. Local/dev callers can keep the default soft mode when they need best-effort telemetry only.
|
||
|
||
## Deferred env vars
|
||
|
||
The full env-var matrix in `docs/improvement-plan/06-plugin-system/wasm-sandbox.md` is broader than the current worker bootstrap. The foundation currently wires:
|
||
|
||
| Env var | Default | Used by |
|
||
|---|---|---|
|
||
| `PLUGIN_TRUSTED_PUBLIC_KEYS` | (empty → fail-closed) | `parseTrustedPublicKeys` |
|
||
| `PLUGIN_MAX_FUEL` | 1000000 | sandbox fuel limit |
|
||
| `PLUGIN_MAX_MEMORY_MB` | 64 | sandbox memory limit |
|
||
| `PLUGIN_MAX_WALL_CLOCK_MS` | 5000 | worker-thread sandbox wall-clock kill |
|
||
| `PLUGIN_OUTPUT_SANITIZATION_STRICTNESS` | moderate | plugin output redaction strictness |
|
||
| `PLUGIN_BINARY_BASE_PATH` | `/app/var/plugins` | plugin binary lookup root for the worker bootstrap |
|
||
| `PLUGIN_RATE_LIMIT_PER_MINUTE` | 100 | worker sliding-window rate limit |
|
||
| `PLUGIN_RATE_LIMIT_WINDOW_SECONDS` | 60 | worker sliding-window rate limit |
|
||
| `PLUGIN_AUTO_DISABLE_THRESHOLD` | 10 | per-tenant/plugin crash auto-disable threshold |
|
||
| `PLUGIN_AUTO_DISABLE_WINDOW_MS` | 600000 | auto-disable sliding window |
|
||
| `PLUGIN_AUDIT_RETENTION_DAYS` | 90 | `PluginCapabilityUse` retention window |
|
||
| `PLUGIN_AUDIT_RETENTION_BATCH_SIZE` | 1000 | audit-retention cleanup batch size |
|
||
| `PLUGIN_AUDIT_RETENTION_MAX_BATCHES` | 100 | audit-retention cleanup max batches per run |
|
||
| `PLUGIN_AUDIT_RETENTION_CRON` | `0 3 * * *` | audit-retention scheduler pattern |
|
||
| `PLUGIN_INSTALL_RATE_LIMIT_PER_MIN` | 5 | tenant plugin-install route rate limit |
|
||
| `PLUGIN_SETTINGS_WRITE_RATE_LIMIT_PER_MIN` | 60 | per-user plugin settings write route rate limit |
|
||
| `PLUGIN_SETTINGS_FRAME_VIOLATION_RATE_LIMIT_PER_MIN` | 120 | client-reported plugin settings iframe violation route rate limit |
|
||
| `PLUGIN_MAX_INSTALLS_PER_TENANT` | 50 | tenant plugin-install cap |
|
||
| `WORKER_CONCURRENCY_PLUGIN` | 5 | `group-runtime.ts` |
|
||
| `WORKER_QUEUE_GROUP=plugin` | — | worker pod entry-point |
|
||
|
||
Richer host-binding controls remain runtime follow-up work. Do not treat undocumented knobs as production-enforced until the worker bootstrap reads them; the operator-facing status is tracked in [Plugin Lifecycle Operations](../operations/plugin-lifecycle.md).
|
||
|
||
## Why this is "foundation"
|
||
|
||
The full spec describes a 4–5 week effort covering 17 implementation steps and dual-track SDKs (AssemblyScript + Rust), per-instruction fuel via Wasmtime, multi-worker-thread pools, hot-reload, marketplace, signing-key rotation. The foundation shipped here is the **structural skeleton plus minimum-viable WASM execution** — every later step is the same `PluginSandbox` / `CapabilityGateway` / `handlePluginInvokeJob` contract with a richer implementation behind it.
|