# Chatbot MVP — Master Switch + 3 Base Capabilities

**Status:** Shipped (Phase 07, Task #13).

This document describes how the in-app chatbot is gated, which capabilities it
exposes, how to add new capabilities, and how to troubleshoot the most common
failure modes.

## Architecture overview

The chatbot is a streaming AI assistant exposed in the in-app shell. In MVP it
ships behind a **master feature flag** (`chatbot`) and exposes three read-only,
tenant-scoped capabilities. Future phases (V1, V2) extend the capability set
with per-user allow-lists, RAG, and write-tools.

```
+------------------+    flag off    +------------+
| chatbot-stream   |  ───────────►  | 403 (json) |
| (server)         |                +------------+
|                  |    flag on, key missing
|                  |  ───────────►  +------------+
|                  |                | 503 NO_AI_ |
|                  |                | PROVIDER_  |
|                  |                | KEY        |
|                  |                +------------+
|                  |    flag on, key present
|                  |  ───────────►  capability resolver
|                  |                  │
|                  |                  ▼
|                  |                tool registry (only active caps)
|                  |                  │
|                  |                  ▼
|                  |                streamText  → SSE
+------------------+
```

Source-of-truth modules:

| Concern                  | File                                                                            |
| ------------------------ | ------------------------------------------------------------------------------- |
| Capability enum + metadata | `apps/web/app/modules/chatbot/lib/capabilities/capability-types.ts`             |
| Capability resolver      | `apps/web/app/modules/chatbot/lib/capabilities/resolver.server.ts`              |
| System prompt builder    | `apps/web/app/modules/chatbot/lib/system-prompt.ts`                             |
| `qa_help` tool           | `apps/web/app/modules/chatbot/lib/capabilities/qa-help.server.ts`               |
| `kb_search` tool         | `apps/web/app/modules/chatbot/lib/capabilities/kb-search.server.ts`             |
| `account_info` tool      | `apps/web/app/modules/chatbot/lib/capabilities/account-info.server.ts`          |
| Tool registry            | `apps/web/app/modules/chatbot/lib/capabilities/tool-registry.server.ts`         |
| Stream entry point       | `apps/web/app/modules/chatbot/lib/chatbot-stream.server.ts`                     |
| Provider-key status      | `apps/web/app/modules/chatbot/lib/provider-status.server.ts`                    |
| Tenant settings page     | `apps/web/app/routes/app.$tenant/settings/chatbot.tsx`                          |
| CLI commands             | `packages/cli/src/commands/chatbot/`                                            |

## How the master flag works

A single global feature flag named `chatbot` controls availability across the
platform. The flag is evaluated per request via
`evaluateFlagForContext(CHATBOT_MASTER_FLAG_NAME, …)` using the Phase-05
feature-flag engine (`@toolctrl/feature-flags`). Result is cached at the
request level.

- **Flag off / not found** → tenant chats return
  `403 { error: "chatbot-disabled", message: … }`.
- **Flag on + no AI-provider env key** → return
  `503 { error: "ai-provider-not-configured", code: "NO_AI_PROVIDER_KEY", docs: … }`.
- **Flag on + key present** → resolver returns
  `['qa_help', 'kb_search', 'account_info']`, the tool registry registers the
  three corresponding tools, and `streamText` runs.

The flag is consulted only for **tenant** chats. Admin chats use a separate
static gate (`getChatbotContextConfig('admin').enabled`) enforced upstream.

### Toggling the flag

Three equivalent ways:

1. **Tenant Settings UI**: `/app/$tenant/settings/chatbot` → "Enable chatbot".
2. **Admin Feature Flags UI**: `/admin/feature-flags` → flip `chatbot` row.
3. **CLI**:
   ```bash
   toolctrl-cli chatbot enable
   toolctrl-cli chatbot disable
   toolctrl-cli chatbot status
   ```

Every flip publishes on the `feature-flags:invalidate` channel — the web
replica's eval cache drops automatically.

## The 3 MVP capabilities

| ID             | Tool name              | Description                                              | Risk |
| -------------- | ---------------------- | -------------------------------------------------------- | ---- |
| `qa_help`      | `search_help`          | Ranked snippets from bundled help docs (no DB, no net).  | low  |
| `kb_search`    | `search_knowledge_base`| Tenant-scoped knowledge-base search.                     | low  |
| `account_info` | `get_account_info`     | Read-only plan / credits / limits / members projection.  | low  |

All three are **read-only**, **tenant-scoped via closure** (tenantId is
injected into the tool factory; the tool schema deliberately does not accept a
`tenantId` field), and never expose sensitive billing identifiers.

### Indirect-prompt-injection mitigation

`kb_search` wraps every snippet in explicit
`---KB ARTICLE START---` / `---KB ARTICLE END---` markers. The system prompt
contains an explicit rule that content between these markers is USER-DATA, not
instructions. This is a partial mitigation — V1 will add per-tool audit logs
and V2 will introduce a separated untrusted-context channel.

## How to add a new capability

1. Add the ID to the `CHATBOT_CAPABILITIES` union and metadata table in
   `capability-types.ts`. Set `mvp: false`.
2. Implement the tool factory in `capabilities/<your-cap>.server.ts` exporting
   a `build<YourCap>Tool(deps)` function. Use closure-bound tenantId; never
   accept tenantId as a tool arg.
3. Add a case in `buildToolRegistry` mapping the capability to the new tool.
4. (V1+) Update the resolver so the new capability is included only for the
   right users / plans.
5. Tests: unit-test the tool's projection, and add a registry case that
   asserts the tool is exposed when the capability is active and NOT when it
   is inactive.

## Provider-key onboarding

`provider-status.server.ts` checks for any of:

- `MISTRAL_API_KEY`
- `SCALEWAY_API_KEY`
- `ALEPH_ALPHA_API_KEY`
- `OPENAI_API_KEY`
- `ANTHROPIC_API_KEY`

If none are set but the chatbot flag is ON, the stream returns
`NO_AI_PROVIDER_KEY` (503). The settings UI surfaces a banner. The CLI
`chatbot status` command warns operators about this state.

## Troubleshooting

| Symptom                                                  | Diagnosis                                                                 |
| -------------------------------------------------------- | ------------------------------------------------------------------------- |
| `403 chatbot-disabled` in browser network tab            | Feature flag `chatbot` is off. Toggle in settings UI or via CLI.          |
| `503 NO_AI_PROVIDER_KEY` after enabling                   | No AI-provider key in env. Set one of the 5 supported `*_API_KEY` vars.   |
| Chatbot answers without using a tool                     | Question may not require a tool. Inspect Sentry tags / steps in DB.       |
| KB search returns no results for known-good query        | Verify tenant-id scoping. Cross-tenant negative test in `kb-search.server.test.ts`. |
| Stream succeeds but no cost event in DB                  | Check `AI_COST_TRACKING_ENABLED` env. MVP wires cost-tracking via the existing `streamText` path; full `executeLLMCall` integration arrives in V2. |

## Known MVP limitations (resolved in V1 / V2)

- Single global flag — no per-user allow-list (V1).
- Capability set is hard-wired to the three MVP ids (V1 will add the remaining
  five: `data_read`, `data_write`, `workflow_trigger`, `admin_actions`,
  `external_tools`).
- No RAG over tenant docs (V2).
- No per-capability rate limiting (V1 adds this on top of the global per-user
  limiter).
- No audit-log entry per tool call (V1 adds this for high-risk tools).

## References

- Spec: `docs/improvement-plan/07-chatbot/chatbot-mvp.md`
- V1: `docs/improvement-plan/07-chatbot/chatbot-v1.md`
- V2: `docs/improvement-plan/07-chatbot/chatbot-v2.md`
- Feature flags engine: `docs/features/feature-flags.md`
- AI cost engine: `docs/features/ai-cost-tracking.md`