Track HiveGrid Cloud planning reference

This commit is contained in:
david committed 2026-06-26 20:48:01 +02:00
1 parent b3d5ea6764
commit e341c5938a
5784 files changed
+1104929 -3

No files matched your search

@@ -0,0 +1,49 @@
# async-task
The long-running-task pattern. Plugin handlers should never do heavy
work inline — the sandbox has a fuel ceiling and the hook fan-out is
held while the handler runs. This example records an audit event
representing the work-request and returns immediately; a host-side
worker consumes the event and runs the actual work outside the
sandbox.
## Why not just block in the handler
- Sandbox fuel ceiling: handlers that exceed the budget trap and the
invocation is logged as a failure.
- Hook fan-out: each plugin subscriber serialises the host event loop.
A slow plugin slows every later subscriber.
- Audit clarity: a fast "work-requested" row in the trail is more
useful than a 30-second-blocking row that may or may not have
completed by the time the page loads.
## Pattern
```
Host emits entity.updated
│
▼
Plugin handler runs in sandbox
│
├── ctx.audit.record({ action: "async-task.work-requested", ... })
│ (writes one PluginCapabilityUse audit row + an audit event)
│
└── returns { enqueued: true } immediately
│
▼
Outbox dispatcher reads audit event
│
▼
worker subscribes to action="async-task.work-requested",
runs the long work outside the sandbox, emits a new hook
event on completion if a follow-up is needed.
```
## Caveats
- The audit body is not a payload queue — it has a small size cap and
the runtime sanitises it. Pass IDs through it, fetch the full record
from the worker side via `db.read` if needed.
- The audit row arrives *eventually* (transactional outbox), not
synchronously. If you need synchronous behaviour, plugins are the
wrong tool — call out to a regular host RPC instead.
@@ -0,0 +1,44 @@
/**
* Async-task example — Phase 06 #23.
*
* Long-running side effects should not run inside the plugin invocation
* (the sandbox has a fuel ceiling and the hook fan-out is blocked
* while a handler runs). Instead, the plugin records an audit event
* and returns immediately; a host worker picks up the event from the
* outbox and does the heavy lifting outside the sandbox.
*
* The plugin never blocks waiting on the worker — that is the whole
* point of the pattern. If the host wants a callback, it emits a new
* hook event when the worker is done; the plugin can subscribe to
* that too.
*/
import { defineHook, type PluginHookEntityPayload } from "../../src/index.js";
export const handleEntityUpdated = defineHook<PluginHookEntityPayload, { enqueued: boolean }>(
async (input, ctx) => {
ctx.log.info(
{
tenantId: input.tenantId,
entity: input.entityType,
changedFieldCount: input.changedFields?.length ?? 0,
},
"async-task.handleEntityUpdated received update"
);
// Record one structured audit event — the host worker subscribes
// to plugin-emitted audit events and runs the actual work. Body
// is bounded so we don't accidentally turn the audit log into a
// queue (the audit sink rejects payloads above its cap).
await ctx.audit.record({
action: "async-task.work-requested",
target: `${input.entityType}:${input.entityId}`,
meta: {
tenantId: input.tenantId,
changedFieldCount: input.changedFields?.length ?? 0,
},
});
ctx.log.info({}, "async-task enqueued work-request, returning fast");
return { enqueued: true };
}
);
@@ -0,0 +1,32 @@
# Phase 06 #23 example — async-task plugin.
#
# Demonstrates the long-running-task pattern: instead of doing the work
# inline (which blocks the hook fan-out and risks fuel exhaustion), the
# plugin enqueues the work through `audit` (the capability for emitting
# host events) and returns immediately. The host outbox picks up the
# event and a worker handles the actual work outside the sandbox.
#
# The benefit: the plugin invocation stays fast and traceable while the
# heavy lifting happens in regular BullMQ. The plugin still has clear
# audit rows for the trigger; the worker handles its own observability.
schemaVersion = 1
id = "de.toolctrl.examples.async-task"
version = "0.1.0"
name = "Async Task Demo"
description = "Enqueues a long-running task via the audit capability, returns immediately."
entry = "plugin.wasm"
wasmSha256 = "0000000000000000000000000000000000000000000000000000000000000000"
signature = "00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000"
signedBy = "0000000000000000000000000000000000000000000000000000000000000000"
allowedHosts = []
[[capabilities]]
name = "log"
[[capabilities]]
name = "audit"
[[hooks]]
name = "entity.updated"
export = "handleEntityUpdated"
@@ -0,0 +1,35 @@
# capability-use
Three host capabilities, one handler. Designed as the reference for
the audit-trail page: every invocation produces a non-trivial set of
`PluginCapabilityUse` rows you can browse from
`/app/:tenant/settings/plugins/:id/audit`.
## Capabilities used
| Capability | Used for |
| --------------- | ----------------------------------------------- |
| `log` | Structured logging through `ctx.log` |
| `storage.read` | Look up the tenant's Slack webhook URL |
| `http.outbound` | POST a notification to `hooks.slack.com` |
The manifest restricts `http.outbound` via `allowedHosts =
["hooks.slack.com"]` — any other host triggers a denied row instead of
egress.
## Why each call shows up in the audit trail
The capability gateway emits an `onCall` event per host-fn call. The
runtime wires that to `PluginCapabilityUseRecorder`, which buffers
entries per invocation then flushes one outbox batch at sandbox close.
The worker `plugin-capability-use` job persists each row. The audit
page filters by capability + UTC date and pages by keyset cursor.
## Tenant-admin pre-requisites
Before this plugin can post:
1. Install + enable the plugin via the plugins settings page.
2. Write the Slack webhook URL under storage key
`slack:<tenantId>:webhook-url` (operator action — see
`docs/plugins/security-best-practices.md`).
@@ -0,0 +1,64 @@
/**
* Capability-use example — Phase 06 #23.
*
* One handler that touches every "regular" capability so the audit-
* trail page shows a non-trivial set of rows per invocation:
* 1. storage.read to fetch the tenant's Slack webhook URL
* 2. http.outbound to POST a notification to Slack
* 3. log everywhere for structured tracing
*
* When run with the host audit recorder wired in, each capability call
* emits one PluginCapabilityUse row with outcome=success / error and a
* sha256(payloadDescriptor). Tenant admins see exactly which channel
* received which event without the message body ever being persisted.
*/
import { defineHook } from "../../src/index.js";
export interface EntityCreatedInput {
readonly tenantId: string;
readonly entityName: string;
readonly recordId: string;
}
export const handleEntityCreated = defineHook<EntityCreatedInput, { posted: boolean }>(
async (input, ctx) => {
ctx.log.info(
{ tenantId: input.tenantId, entity: input.entityName },
"capability-use.handleEntityCreated invoked"
);
if (!ctx.storage) {
ctx.log.warn({}, "storage capability not bound — manifest must declare storage.read");
return { posted: false };
}
const webhookUrl = await ctx.storage.get(`slack:${input.tenantId}:webhook-url`);
if (!webhookUrl) {
ctx.log.info({ tenantId: input.tenantId }, "no slack webhook configured — skipping");
return { posted: false };
}
if (!ctx.http) {
ctx.log.warn({}, "http capability not bound — manifest must declare http.outbound");
return { posted: false };
}
const body = JSON.stringify({
text: `New ${input.entityName} created (id ${input.recordId})`,
});
const response = await ctx.http.fetch({
url: webhookUrl,
method: "POST",
headers: { "Content-Type": "application/json" },
body,
});
if (response.status >= 200 && response.status < 300) {
ctx.log.info({ status: response.status }, "slack webhook posted");
return { posted: true };
}
ctx.log.error(
{ status: response.status, body: response.body.slice(0, 256) },
"slack webhook returned non-2xx"
);
return { posted: false };
}
);
@@ -0,0 +1,35 @@
# Phase 06 #23 example — capability-use plugin.
#
# Demonstrates multiple capabilities working together inside a single
# hook handler:
# - http.outbound for a webhook call (restricted to hooks.slack.com)
# - storage.read to look up the tenant's per-user channel mapping
# - log for structured tracing throughout
# Each host-fn call gets one row in PluginCapabilityUse, so the tenant
# admin's audit-trail page surfaces exactly what this plugin did per
# invocation.
schemaVersion = 1
id = "de.toolctrl.examples.capability-use"
version = "0.1.0"
name = "Capability-Use Demo"
description = "Calls http.outbound + storage.read + log on every invocation. Exercises the audit trail."
entry = "plugin.wasm"
wasmSha256 = "0000000000000000000000000000000000000000000000000000000000000000"
signature = "00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000"
signedBy = "0000000000000000000000000000000000000000000000000000000000000000"
allowedHosts = ["hooks.slack.com"]
[[capabilities]]
name = "log"
[[capabilities]]
name = "storage.read"
[[capabilities]]
name = "http.outbound"
scope = { allowedHosts = ["hooks.slack.com"] }
[[hooks]]
name = "entity.created"
export = "handleEntityCreated"
@@ -0,0 +1,37 @@
# Hello-World example plugin
Minimum-viable plugin demonstrating the `@toolctrl/plugin-sdk` shape.
## Files
- `plugin.toml` — the manifest the host parses + verifies. The `wasmSha256`, `signature`, and `signedBy` fields are placeholders; the operator overwrites them after signing the compiled `.wasm` against a trusted Ed25519 keypair.
- `handler.ts` — the plugin source. Exports `handleHello`, the hook the manifest references via `[[hooks]] export = "handleHello"`.
## Build
The plugin build CLI is **deferred** to Phase 06 Task #23 (see `docs/improvement-plan/06-plugin-system/wasm-sandbox.md`). Until then, plugin authors compile to WASM with their preferred toolchain. Two practical paths today:
1. **AssemblyScript** — write the handler in AssemblyScript syntax and run `asc handler.ts -o plugin.wasm --optimize`.
2. **Rust** — port the handler to a `#[no_mangle]` extern function and build with `wasm-pack build --target nodejs`.
Both produce a `.wasm` binary the runtime can instantiate.
## Sign + install
Once the binary is built:
```bash
# Compute the hash and update the manifest in-place.
sha256sum plugin.wasm
# Sign the .wasm with the publisher's Ed25519 private key.
# (Sign helper for foundation: use Node's crypto.sign with no algorithm.)
node -e "const c=require('crypto'); const fs=require('fs'); \
const key=c.createPrivateKey({key: fs.readFileSync('publisher.priv'), format:'pem'}); \
process.stdout.write(c.sign(null, fs.readFileSync('plugin.wasm'), key).toString('hex'))"
# Drop the signed binary + manifest into the host's plugin volume.
cp plugin.wasm plugin.toml <plugin-volume>/<plugin-id>/1.0.0/
```
The operator then ensures the Ed25519 public key matching the signature is in `PLUGIN_TRUSTED_PUBLIC_KEYS` and inserts a row into the `Plugin` table.
@@ -0,0 +1,23 @@
/**
* Hello-World example plugin source.
*
* Plugin authors write TypeScript / AssemblyScript / Rust against the
* `@toolctrl/plugin-sdk` types, then compile to a single `.wasm` binary
* with their toolchain of choice (wasm-pack, AssemblyScript's `asc`,
* etc.). The host runtime treats the .wasm as opaque — only its
* exports (`handleHello` here) and the `plugin.toml` manifest matter.
*
* Building this file to WASM is intentionally outside the scope of
* the foundation: the `toolctrl plugin build` CLI is deferred (see
* `docs/improvement-plan/06-plugin-system/wasm-sandbox.md` Deviations).
* Operators wanting a working hello-world today can hand-compile via
* AssemblyScript or use any other WASM toolchain that produces a flat
* function export — the runtime contract is just `(arg) -> result`.
*/
import { defineHook } from "../../src/index.js";
export const handleHello = defineHook<{ name: string }, { greeting: string }>((input, ctx) => {
ctx.log.info({ name: input.name }, "hello-plugin.invoked");
return { greeting: `Hello, ${input.name}!` };
});
@@ -0,0 +1,28 @@
# Phase 06 #21 — Hello-World example plugin manifest.
#
# A plugin author ships exactly this file alongside the .wasm binary.
# The runtime parses it via @toolctrl/plugin-runtime's PluginManifestSchema
# (note: foundation supports both TOML and JSON shape — the schema works
# off the parsed object so authors may choose the format their toolchain
# emits). The values below are illustrative; the wasmSha256 / signature /
# signedBy fields are placeholders that an operator replaces after
# signing the actual binary with `toolctrl plugin sign` (deferred CLI,
# see Deviations in wasm-sandbox.md).
schemaVersion = 1
id = "de.toolctrl.hello"
version = "1.0.0"
name = "Hello World"
description = "Returns a greeting. Demonstrates the minimum viable plugin."
entry = "plugin.wasm"
wasmSha256 = "0000000000000000000000000000000000000000000000000000000000000000"
signature = "00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000"
signedBy = "0000000000000000000000000000000000000000000000000000000000000000"
allowedHosts = []
[[capabilities]]
name = "log"
[[hooks]]
name = "onHello"
export = "handleHello"
@@ -0,0 +1,25 @@
# simple-hook-listener
The minimum-viable plugin. Subscribes to one hook, logs when invoked,
returns. No external IO.
## What it demonstrates
- Manifest with exactly one capability (`log`)
- One hook subscription (`tenant.created`) and one export
- `defineHook` for compile-time-typed handler arguments
## How to run
1. Compile `handler.ts` to wasm with your toolchain of choice (see
`docs/plugins/cli-reference.md#build`).
2. `toolctrl-cli plugins sign --wasm plugin.wasm --manifest plugin.json --key <ed25519-priv>`.
3. `toolctrl-cli plugins publish --wasm plugin.wasm --manifest plugin.json` → upload the resulting `.tar.gz`.
4. Install through the tenant-admin plugins page.
5. Create a new tenant; observe the host log line.
## Pattern
Hook-only plugins are the safest plugin shape — they cannot leak data
or make outbound requests because they declared no such capabilities.
Start here when you only need to *react* to host events.
@@ -0,0 +1,25 @@
/**
* Simple hook listener — Phase 06 #23 example.
*
* One handler, one capability (`log`), no external IO. The host emits
* `tenant.created` whenever a new workspace is provisioned; this
* plugin records the event through the host logger. Useful for
* smoke-testing the full invoke pipeline without committing to any
* outbound capability.
*/
import { defineHook, type PluginHookTenantCreatedPayload } from "../../src/index.js";
export const handleTenantCreated = defineHook<
PluginHookTenantCreatedPayload,
{ logged: true }
>((input, ctx) => {
ctx.log.info(
{
tenantId: input.tenantId,
tenantSlug: input.slug,
plugin: "simple-hook-listener",
},
"simple-hook-listener received tenant.created"
);
return { logged: true };
});
@@ -0,0 +1,25 @@
# Phase 06 #23 example — simple hook listener.
#
# Demonstrates the minimum-viable hook subscription: declare ONE hook
# in the manifest, register ONE export, log when invoked. Requires only
# the `log` capability — nothing leaves the host. A new plugin author
# can copy this directory, rename, and have a working invocation
# pipeline in minutes.
schemaVersion = 1
id = "de.toolctrl.examples.simple-hook-listener"
version = "0.1.0"
name = "Simple Hook Listener"
description = "Logs tenant.created events. Demonstrates the smallest viable plugin."
entry = "plugin.wasm"
wasmSha256 = "0000000000000000000000000000000000000000000000000000000000000000"
signature = "00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000"
signedBy = "0000000000000000000000000000000000000000000000000000000000000000"
allowedHosts = []
[[capabilities]]
name = "log"
[[hooks]]
name = "tenant.created"
export = "handleTenantCreated"