407 lines
20 KiB
Markdown
407 lines
20 KiB
Markdown
# AI Chatbot Architecture
|
|
|
|
Status: historical plugin-candidate notes. The default boilerplate keeps chatbot UI disabled. Do not add web API routes or marketing chatbot surfaces from this document; new runtime links must use host-aware helpers and new service APIs belong in `apps/api/src/app.ts` under `/v1`.
|
|
|
|
This document explains a possible AI chatbot feature in ToolCTRL Boilerplate. The chatbot uses Vercel AI SDK for streaming responses and provides context-specific tools across tenant and admin contexts.
|
|
|
|
---
|
|
|
|
## Overview
|
|
|
|
```
|
|
┌─────────────────────────────────────────────────────────────────────────────────┐
|
|
│ CLIENT ACCESS POINTS │
|
|
│ │
|
|
│ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ ┌───────────┐ │
|
|
│ │ Nav Bar Icon │ │ Tenant Chat │ │ Admin Chat │ │ Marketing │ │
|
|
│ │ (Dialog Mode) │ │ appPath(...) │ │ adminPath(...) │ │ disabled │ │
|
|
│ └─────────────────┘ └─────────────────┘ └─────────────────┘ └───────────┘ │
|
|
│ │ │ │ │ │
|
|
└──────────│────────────────────│────────────────────│───────────────────│────────┘
|
|
│ │ │ │
|
|
▼ ▼ ▼ ▼
|
|
┌─────────────────────────────────────────────────────────────────────────────────┐
|
|
│ CHATBOT WIDGET (React) │
|
|
│ │
|
|
│ ┌─────────────────────────────────────────────────────────────────────────┐ │
|
|
│ │ AI Elements Components (11 reusable) │ │
|
|
│ │ conversation | message | response | reasoning | tool | sources │ │
|
|
│ │ code-block | loader | actions | prompt-input | suggestion │ │
|
|
│ └─────────────────────────────────────────────────────────────────────────┘ │
|
|
│ │
|
|
└─────────────────────────────────────────────────────────────────────────────────┘
|
|
│
|
|
│ POST with context
|
|
▼
|
|
┌─────────────────────────────────────────────────────────────────────────────────┐
|
|
│ API ROUTES │
|
|
│ │
|
|
│ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────────────────────┐ │
|
|
│ │ apps/api /v1 │ │ apps/api /v1 │ │ No default marketing route │ │
|
|
│ │ tenant context │ │ admin context │ │ │ │
|
|
│ │ (Auth required) │ │ (Admin only) │ │ │ │
|
|
│ └─────────────────┘ └─────────────────┘ └─────────────────────────────────┘ │
|
|
│ │
|
|
└─────────────────────────────────────────────────────────────────────────────────┘
|
|
│
|
|
▼
|
|
┌─────────────────────────────────────────────────────────────────────────────────┐
|
|
│ STREAMING SERVICE │
|
|
│ (chatbot-stream.server.ts) │
|
|
│ │
|
|
│ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────────────────────┐ │
|
|
│ │ Tenant Tools │ │ Admin Tools │ │ Plugin Tools │ │
|
|
│ │ - session │ │ - accounts_list │ │ - features_list │ │
|
|
│ │ - info │ │ - users_list │ │ - pricing_info │ │
|
|
│ │ - navigate │ │ - users_find │ │ │ │
|
|
│ │ - logs_recent │ │ - stats │ │ │ │
|
|
│ │ + MCP tools │ │ - navigate │ │ │ │
|
|
│ └─────────────────┘ └─────────────────┘ └─────────────────────────────────┘ │
|
|
│ │
|
|
└─────────────────────────────────────────────────────────────────────────────────┘
|
|
│
|
|
┌────────────┴────────────┐
|
|
▼ ▼
|
|
┌────────────────────────────────┐ ┌─────────────────────────────────────────────┐
|
|
│ OPENAI API │ │ DATABASE (Prisma) │
|
|
│ │ │ │
|
|
│ - GPT-4o (default) │ │ Chat ──┬── ChatMessage ── ChatMessagePart │
|
|
│ - GPT-4o-mini │ │ │ │
|
|
│ - o1, o1-mini, o1-pro │ │ └── GeneratedObject (tracking) │
|
|
│ - o3-mini │ │ │
|
|
└────────────────────────────────┘ └─────────────────────────────────────────────┘
|
|
```
|
|
|
|
---
|
|
|
|
## Module Structure
|
|
|
|
```
|
|
apps/web/app/modules/chatbot/
|
|
├── chatbot-config.ts # Centralized configuration
|
|
├── components/
|
|
│ ├── chatbot-widget.tsx # Main chat UI component
|
|
│ └── chat-list.tsx # Chat history list
|
|
└── lib/
|
|
├── chatbot.db.server.ts # Database operations
|
|
├── chatbot-stream.server.ts # Shared streaming service
|
|
├── chatbot-dtos.ts # Prisma-based DTOs
|
|
├── chatbot-context-dtos.ts # Stream config types
|
|
├── system-prompt.ts # Context system prompts
|
|
├── capabilities/ # Capability resolver + tool registry
|
|
├── tools-admin/ # Admin prompt + tools
|
|
└── tools-tenant/ # Tenant prompt + tools
|
|
|
|
apps/web/app/modules/ai-gen/
|
|
├── db/
|
|
│ └── ai-generation.db.server.ts # Generation tracking
|
|
├── lib/
|
|
│ └── ai-models.ts # Available AI models
|
|
├── dtos/
|
|
│ └── ai-generation-dto.ts # Generation DTOs
|
|
└── services/
|
|
└── ai-tools.server.ts # AI tool utilities
|
|
|
|
apps/web/app/components/ai-elements/ # 11 reusable components
|
|
├── conversation.tsx # Chat container with scroll
|
|
├── message.tsx # Individual message wrapper
|
|
├── prompt-input.tsx # Input with model selector
|
|
├── response.tsx # AI response with markdown
|
|
├── reasoning.tsx # Collapsible reasoning display
|
|
├── tool.tsx # Tool call visualization
|
|
├── sources.tsx # Citation display
|
|
├── code-block.tsx # Syntax highlighting
|
|
├── loader.tsx # Loading indicators
|
|
├── actions.tsx # Message action buttons
|
|
└── suggestion.tsx # Quick suggestion buttons
|
|
```
|
|
|
|
---
|
|
|
|
## Database Schema
|
|
|
|
```prisma
|
|
model Chat {
|
|
id String @id @default(cuid())
|
|
createdAt DateTime @default(now())
|
|
updatedAt DateTime @updatedAt
|
|
type String // "tenant" | "admin" | "marketing"
|
|
tenantId String? // For tenant chats
|
|
userId String? // For authenticated users
|
|
sessionId String? // For anonymous marketing chats
|
|
title String
|
|
messages ChatMessage[]
|
|
|
|
tenant Tenant? @relation(fields: [tenantId], references: [id], onDelete: Cascade)
|
|
user User? @relation(fields: [userId], references: [id], onDelete: Cascade)
|
|
}
|
|
|
|
model ChatMessage {
|
|
id String @id @default(cuid())
|
|
createdAt DateTime @default(now())
|
|
updatedAt DateTime @updatedAt
|
|
chatId String
|
|
role String // "user" | "assistant"
|
|
parts ChatMessagePart[]
|
|
|
|
chat Chat @relation(fields: [chatId], references: [id], onDelete: Cascade)
|
|
}
|
|
|
|
model ChatMessagePart {
|
|
id String @id @default(cuid())
|
|
createdAt DateTime @default(now())
|
|
messageId String
|
|
order Int
|
|
type String // "text" | "reasoning" | "tool-*" | "source-url"
|
|
content String // JSON stringified
|
|
|
|
message ChatMessage @relation(fields: [messageId], references: [id], onDelete: Cascade)
|
|
}
|
|
|
|
model GeneratedObject {
|
|
id String @id @default(cuid())
|
|
createdAt DateTime @default(now())
|
|
userId String
|
|
chatId String? // Nullable, SetNull on delete
|
|
object String // "ChatMessage"
|
|
type String // "chat"
|
|
model String
|
|
status String // "pending" | "success" | "error"
|
|
inputTokens Int @default(0)
|
|
outputTokens Int @default(0)
|
|
costInCents Float @default(0)
|
|
timeMs Int @default(0)
|
|
|
|
user User @relation(fields: [userId], references: [id], onDelete: Cascade)
|
|
chat Chat? @relation(fields: [chatId], references: [id], onDelete: SetNull)
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
## Three Chat Contexts
|
|
|
|
| Context | Route | Auth | Tools | Persistence |
|
|
| ------- | --------------------------------------- | ---------- | ---------------------------------- | ---------------- |
|
|
| Tenant | `appPath({ tenantSlug, path: "chat" })` | Required | session, info, navigate, logs, MCP | User + Tenant ID |
|
|
| Admin | `adminPath("chat")` | Admin only | accounts, users, stats, navigate | User ID |
|
|
|
|
### Tenant Tools
|
|
|
|
| Tool | Description |
|
|
| -------------------- | ------------------------------------- |
|
|
| `system_session` | Get current account and user info |
|
|
| `system_info` | Platform features and capabilities |
|
|
| `system_navigate` | Navigate to platform sections |
|
|
| `tenant_logs_recent` | View recent application logs |
|
|
| + MCP tools | Extended tools via MCP server at /mcp |
|
|
|
|
### Admin Tools
|
|
|
|
| Tool | Description |
|
|
| ---------------------- | -------------------------- |
|
|
| `system_session` | Get current user profile |
|
|
| `admin_accounts_list` | List all tenant accounts |
|
|
| `admin_users_list` | List platform users |
|
|
| `admin_users_find` | Search users by email/name |
|
|
| `admin_stats_platform` | Platform statistics |
|
|
| `admin_navigate` | Navigate to admin sections |
|
|
|
|
### Plugin Tools
|
|
|
|
No marketing chatbot tools are part of the default boilerplate. A later plugin must own its route, API, permissions, and manifest updates.
|
|
|
|
---
|
|
|
|
## API Routes
|
|
|
|
| Endpoint | Method | Context | Auth Required |
|
|
| ----------------------------- | ------ | ------- | ------------- |
|
|
| `apps/api /v1/ai/chat/tenant` | POST | Tenant | Yes |
|
|
| `apps/api /v1/ai/chat/admin` | POST | Admin | Yes (admin) |
|
|
|
|
### Request Format
|
|
|
|
```typescript
|
|
{
|
|
chatId?: string; // Existing chat ID or undefined for new
|
|
messages: Message[]; // Conversation history
|
|
model?: string; // Model override (optional)
|
|
}
|
|
```
|
|
|
|
### Response Format
|
|
|
|
Server-Sent Events (SSE) stream with text deltas, tool calls, and message parts.
|
|
|
|
---
|
|
|
|
## Configuration
|
|
|
|
### Chatbot Config (`chatbot-config.ts`)
|
|
|
|
```typescript
|
|
type ChatbotContextConfig = {
|
|
enabled: boolean; // Enable/disable this context
|
|
name: string; // Assistant name (e.g., "Sales Assistant")
|
|
defaultModel: LanguageModel; // Default AI model
|
|
maxSteps: number; // Max tool execution steps
|
|
showModelSelector: boolean; // Show model dropdown in UI
|
|
showAttachments: boolean; // Allow file attachments
|
|
showTools: boolean; // Show tool invocations in chat
|
|
suggestions: string[]; // Quick action suggestions
|
|
welcomeMessages: string[]; // Welcome messages when chat is empty
|
|
pages?: { path: string; exact: boolean }[]; // Marketing pages whitelist
|
|
};
|
|
```
|
|
|
|
### Default Settings by Context
|
|
|
|
| Setting | Marketing | Admin | Tenant |
|
|
| ------------------- | --------------- | --------------- | -------------- |
|
|
| `enabled` | true | true | true |
|
|
| `name` | Sales Assistant | Admin Assistant | User Assistant |
|
|
| `maxSteps` | 2 | 5 | 5 |
|
|
| `showModelSelector` | false | false | false |
|
|
| `showAttachments` | false | true | true |
|
|
| `showTools` | false | true | true |
|
|
|
|
### Environment Variables
|
|
|
|
```bash
|
|
OPENAI_API_KEY=sk-... # Required for OpenAI models
|
|
ANTHROPIC_API_KEY=... # Required for Claude models (future)
|
|
```
|
|
|
|
### Available Models
|
|
|
|
Configured in `apps/web/app/modules/ai-gen/lib/ai-models.ts`:
|
|
|
|
| Model | Provider | Cost (per 1M input tokens) |
|
|
| ----------- | -------- | -------------------------- |
|
|
| gpt-4o | OpenAI | $0.25 |
|
|
| gpt-4o-mini | OpenAI | $0.015 |
|
|
| o1 | OpenAI | $1.50 |
|
|
| o1-mini | OpenAI | $0.30 |
|
|
| o1-pro | OpenAI | $15.00 |
|
|
| o3-mini | OpenAI | $1.10 |
|
|
|
|
---
|
|
|
|
## Adding Custom Tools
|
|
|
|
Current capability-driven tool wiring is documented in
|
|
`docs/features/chatbot-mvp.md`. Context-specific legacy tools are grouped by
|
|
context in:
|
|
|
|
```
|
|
apps/web/app/modules/chatbot/lib/
|
|
├── capabilities/
|
|
├── tools-admin/tools-admin.server.ts
|
|
└── tools-tenant/tools-tenant.server.ts
|
|
```
|
|
|
|
Current capability files export focused tool builders and the registry wires
|
|
them only when the resolver enables the matching capability. Context tool files
|
|
export:
|
|
|
|
- context prompt helpers
|
|
- context tool definitions using Zod schemas
|
|
- context TypeScript types
|
|
|
|
### Tool Definition Example
|
|
|
|
```typescript
|
|
import { z } from "zod";
|
|
import { tool } from "ai";
|
|
|
|
my_custom_tool: tool({
|
|
description: "What this tool does",
|
|
inputSchema: z.object({
|
|
param: z.string().describe("Parameter description"),
|
|
}),
|
|
execute: async ({ param }) => {
|
|
// Tool logic here
|
|
return { result: "data" };
|
|
},
|
|
}),
|
|
```
|
|
|
|
---
|
|
|
|
## Admin Monitoring
|
|
|
|
### Routes
|
|
|
|
| Route | Purpose |
|
|
| ----------------------------- | ------------------------ |
|
|
| `adminPath("ai")` | Overview + Stats |
|
|
| `adminPath("ai/generations")` | All AI Generations table |
|
|
| `adminPath("ai/chats")` | All Chats table |
|
|
| `adminPath("ai/chats/:id")` | Chat Detail (Read-Only) |
|
|
|
|
### Stats Dashboard
|
|
|
|
- Total Generations
|
|
- Total Cost
|
|
- Average Cost per Object
|
|
- Average Time per Object
|
|
- Breakdown by Object Type
|
|
- Breakdown by Model
|
|
|
|
---
|
|
|
|
## Request/Response Flow
|
|
|
|
```
|
|
1. User types message in ChatbotWidget
|
|
2. Widget calls API route with chatId + messages
|
|
3. API route calls createChatStreamResponse()
|
|
4. StreamService:
|
|
- Ensures chat exists in DB
|
|
- Saves user message
|
|
- Loads conversation history
|
|
- Creates GeneratedObject (pending)
|
|
- Calls streamText() with context tools
|
|
5. Tool Execution Loop:
|
|
- AI requests tool call
|
|
- Execute tool
|
|
- Return result to AI
|
|
6. AI streams response chunks
|
|
7. StreamService sends SSE to client
|
|
8. Widget displays streaming text
|
|
9. StreamService saves assistant message
|
|
10. Updates GeneratedObject (success + metrics)
|
|
```
|
|
|
|
---
|
|
|
|
## Quick Reference
|
|
|
|
### Adding Chatbot to a Page
|
|
|
|
```tsx
|
|
import { ChatbotWidget } from "~/modules/chatbot/components/chatbot-widget";
|
|
|
|
<ChatbotWidget
|
|
context="tenant" // "tenant" | "admin" | "marketing"
|
|
chatId={chatId} // Optional: existing chat
|
|
initialMessages={[]} // Optional: preload messages
|
|
readOnly={false} // Optional: disable input
|
|
/>;
|
|
```
|
|
|
|
### Creating a New Context
|
|
|
|
1. Create `tools-{context}.server.ts` in `apps/web/app/modules/chatbot/lib/`
|
|
2. Export `CONTEXT_SYSTEM_PROMPT`, `getContextChatTools`, `ContextChatContext`
|
|
3. Register the service API in `apps/api/src/app.ts` under `/v1`
|
|
4. Add context config to `chatbot-config.ts`
|
|
5. Create chat routes at `apps/web/app/routes/{path}/chat/`
|
|
|
|
### Enabling MCP Integration
|
|
|
|
The tenant chat automatically connects to the MCP server at `/mcp` when available. MCP tools extend the chatbot's capabilities with additional data access.
|
|
|
|
See MCP Server documentation for available tools.
|