20 KiB
20 KiB
AI Chatbot Architecture
Status: historical plugin-candidate notes. The default boilerplate keeps chatbot UI disabled. Do not add web API routes or marketing chatbot surfaces from this document; new runtime links must use host-aware helpers and new service APIs belong in apps/api/src/app.ts under /v1.
This document explains a possible AI chatbot feature in ToolCTRL Boilerplate. The chatbot uses Vercel AI SDK for streaming responses and provides context-specific tools across tenant and admin contexts.
Overview
┌─────────────────────────────────────────────────────────────────────────────────┐
│ CLIENT ACCESS POINTS │
│ │
│ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ ┌───────────┐ │
│ │ Nav Bar Icon │ │ Tenant Chat │ │ Admin Chat │ │ Marketing │ │
│ │ (Dialog Mode) │ │ appPath(...) │ │ adminPath(...) │ │ disabled │ │
│ └─────────────────┘ └─────────────────┘ └─────────────────┘ └───────────┘ │
│ │ │ │ │ │
└──────────│────────────────────│────────────────────│───────────────────│────────┘
│ │ │ │
▼ ▼ ▼ ▼
┌─────────────────────────────────────────────────────────────────────────────────┐
│ CHATBOT WIDGET (React) │
│ │
│ ┌─────────────────────────────────────────────────────────────────────────┐ │
│ │ AI Elements Components (11 reusable) │ │
│ │ conversation | message | response | reasoning | tool | sources │ │
│ │ code-block | loader | actions | prompt-input | suggestion │ │
│ └─────────────────────────────────────────────────────────────────────────┘ │
│ │
└─────────────────────────────────────────────────────────────────────────────────┘
│
│ POST with context
▼
┌─────────────────────────────────────────────────────────────────────────────────┐
│ API ROUTES │
│ │
│ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────────────────────┐ │
│ │ apps/api /v1 │ │ apps/api /v1 │ │ No default marketing route │ │
│ │ tenant context │ │ admin context │ │ │ │
│ │ (Auth required) │ │ (Admin only) │ │ │ │
│ └─────────────────┘ └─────────────────┘ └─────────────────────────────────┘ │
│ │
└─────────────────────────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────────────────────┐
│ STREAMING SERVICE │
│ (chatbot-stream.server.ts) │
│ │
│ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────────────────────┐ │
│ │ Tenant Tools │ │ Admin Tools │ │ Plugin Tools │ │
│ │ - session │ │ - accounts_list │ │ - features_list │ │
│ │ - info │ │ - users_list │ │ - pricing_info │ │
│ │ - navigate │ │ - users_find │ │ │ │
│ │ - logs_recent │ │ - stats │ │ │ │
│ │ + MCP tools │ │ - navigate │ │ │ │
│ └─────────────────┘ └─────────────────┘ └─────────────────────────────────┘ │
│ │
└─────────────────────────────────────────────────────────────────────────────────┘
│
┌────────────┴────────────┐
▼ ▼
┌────────────────────────────────┐ ┌─────────────────────────────────────────────┐
│ OPENAI API │ │ DATABASE (Prisma) │
│ │ │ │
│ - GPT-4o (default) │ │ Chat ──┬── ChatMessage ── ChatMessagePart │
│ - GPT-4o-mini │ │ │ │
│ - o1, o1-mini, o1-pro │ │ └── GeneratedObject (tracking) │
│ - o3-mini │ │ │
└────────────────────────────────┘ └─────────────────────────────────────────────┘
Module Structure
apps/web/app/modules/chatbot/
├── chatbot-config.ts # Centralized configuration
├── components/
│ ├── chatbot-widget.tsx # Main chat UI component
│ └── chat-list.tsx # Chat history list
└── lib/
├── chatbot.db.server.ts # Database operations
├── chatbot-stream.server.ts # Shared streaming service
├── chatbot-dtos.ts # Prisma-based DTOs
├── chatbot-context-dtos.ts # Stream config types
├── system-prompt.ts # Context system prompts
├── capabilities/ # Capability resolver + tool registry
├── tools-admin/ # Admin prompt + tools
└── tools-tenant/ # Tenant prompt + tools
apps/web/app/modules/ai-gen/
├── db/
│ └── ai-generation.db.server.ts # Generation tracking
├── lib/
│ └── ai-models.ts # Available AI models
├── dtos/
│ └── ai-generation-dto.ts # Generation DTOs
└── services/
└── ai-tools.server.ts # AI tool utilities
apps/web/app/components/ai-elements/ # 11 reusable components
├── conversation.tsx # Chat container with scroll
├── message.tsx # Individual message wrapper
├── prompt-input.tsx # Input with model selector
├── response.tsx # AI response with markdown
├── reasoning.tsx # Collapsible reasoning display
├── tool.tsx # Tool call visualization
├── sources.tsx # Citation display
├── code-block.tsx # Syntax highlighting
├── loader.tsx # Loading indicators
├── actions.tsx # Message action buttons
└── suggestion.tsx # Quick suggestion buttons
Database Schema
model Chat {
id String @id @default(cuid())
createdAt DateTime @default(now())
updatedAt DateTime @updatedAt
type String // "tenant" | "admin" | "marketing"
tenantId String? // For tenant chats
userId String? // For authenticated users
sessionId String? // For anonymous marketing chats
title String
messages ChatMessage[]
tenant Tenant? @relation(fields: [tenantId], references: [id], onDelete: Cascade)
user User? @relation(fields: [userId], references: [id], onDelete: Cascade)
}
model ChatMessage {
id String @id @default(cuid())
createdAt DateTime @default(now())
updatedAt DateTime @updatedAt
chatId String
role String // "user" | "assistant"
parts ChatMessagePart[]
chat Chat @relation(fields: [chatId], references: [id], onDelete: Cascade)
}
model ChatMessagePart {
id String @id @default(cuid())
createdAt DateTime @default(now())
messageId String
order Int
type String // "text" | "reasoning" | "tool-*" | "source-url"
content String // JSON stringified
message ChatMessage @relation(fields: [messageId], references: [id], onDelete: Cascade)
}
model GeneratedObject {
id String @id @default(cuid())
createdAt DateTime @default(now())
userId String
chatId String? // Nullable, SetNull on delete
object String // "ChatMessage"
type String // "chat"
model String
status String // "pending" | "success" | "error"
inputTokens Int @default(0)
outputTokens Int @default(0)
costInCents Float @default(0)
timeMs Int @default(0)
user User @relation(fields: [userId], references: [id], onDelete: Cascade)
chat Chat? @relation(fields: [chatId], references: [id], onDelete: SetNull)
}
Three Chat Contexts
| Context | Route | Auth | Tools | Persistence |
|---|---|---|---|---|
| Tenant | appPath({ tenantSlug, path: "chat" }) |
Required | session, info, navigate, logs, MCP | User + Tenant ID |
| Admin | adminPath("chat") |
Admin only | accounts, users, stats, navigate | User ID |
Tenant Tools
| Tool | Description |
|---|---|
system_session |
Get current account and user info |
system_info |
Platform features and capabilities |
system_navigate |
Navigate to platform sections |
tenant_logs_recent |
View recent application logs |
| + MCP tools | Extended tools via MCP server at /mcp |
Admin Tools
| Tool | Description |
|---|---|
system_session |
Get current user profile |
admin_accounts_list |
List all tenant accounts |
admin_users_list |
List platform users |
admin_users_find |
Search users by email/name |
admin_stats_platform |
Platform statistics |
admin_navigate |
Navigate to admin sections |
Plugin Tools
No marketing chatbot tools are part of the default boilerplate. A later plugin must own its route, API, permissions, and manifest updates.
API Routes
| Endpoint | Method | Context | Auth Required |
|---|---|---|---|
apps/api /v1/ai/chat/tenant |
POST | Tenant | Yes |
apps/api /v1/ai/chat/admin |
POST | Admin | Yes (admin) |
Request Format
{
chatId?: string; // Existing chat ID or undefined for new
messages: Message[]; // Conversation history
model?: string; // Model override (optional)
}
Response Format
Server-Sent Events (SSE) stream with text deltas, tool calls, and message parts.
Configuration
Chatbot Config (chatbot-config.ts)
type ChatbotContextConfig = {
enabled: boolean; // Enable/disable this context
name: string; // Assistant name (e.g., "Sales Assistant")
defaultModel: LanguageModel; // Default AI model
maxSteps: number; // Max tool execution steps
showModelSelector: boolean; // Show model dropdown in UI
showAttachments: boolean; // Allow file attachments
showTools: boolean; // Show tool invocations in chat
suggestions: string[]; // Quick action suggestions
welcomeMessages: string[]; // Welcome messages when chat is empty
pages?: { path: string; exact: boolean }[]; // Marketing pages whitelist
};
Default Settings by Context
| Setting | Marketing | Admin | Tenant |
|---|---|---|---|
enabled |
true | true | true |
name |
Sales Assistant | Admin Assistant | User Assistant |
maxSteps |
2 | 5 | 5 |
showModelSelector |
false | false | false |
showAttachments |
false | true | true |
showTools |
false | true | true |
Environment Variables
OPENAI_API_KEY=sk-... # Required for OpenAI models
ANTHROPIC_API_KEY=... # Required for Claude models (future)
Available Models
Configured in apps/web/app/modules/ai-gen/lib/ai-models.ts:
| Model | Provider | Cost (per 1M input tokens) |
|---|---|---|
| gpt-4o | OpenAI | $0.25 |
| gpt-4o-mini | OpenAI | $0.015 |
| o1 | OpenAI | $1.50 |
| o1-mini | OpenAI | $0.30 |
| o1-pro | OpenAI | $15.00 |
| o3-mini | OpenAI | $1.10 |
Adding Custom Tools
Current capability-driven tool wiring is documented in
docs/features/chatbot-mvp.md. Context-specific legacy tools are grouped by
context in:
apps/web/app/modules/chatbot/lib/
├── capabilities/
├── tools-admin/tools-admin.server.ts
└── tools-tenant/tools-tenant.server.ts
Current capability files export focused tool builders and the registry wires them only when the resolver enables the matching capability. Context tool files export:
- context prompt helpers
- context tool definitions using Zod schemas
- context TypeScript types
Tool Definition Example
import { z } from "zod";
import { tool } from "ai";
my_custom_tool: tool({
description: "What this tool does",
inputSchema: z.object({
param: z.string().describe("Parameter description"),
}),
execute: async ({ param }) => {
// Tool logic here
return { result: "data" };
},
}),
Admin Monitoring
Routes
| Route | Purpose |
|---|---|
adminPath("ai") |
Overview + Stats |
adminPath("ai/generations") |
All AI Generations table |
adminPath("ai/chats") |
All Chats table |
adminPath("ai/chats/:id") |
Chat Detail (Read-Only) |
Stats Dashboard
- Total Generations
- Total Cost
- Average Cost per Object
- Average Time per Object
- Breakdown by Object Type
- Breakdown by Model
Request/Response Flow
1. User types message in ChatbotWidget
2. Widget calls API route with chatId + messages
3. API route calls createChatStreamResponse()
4. StreamService:
- Ensures chat exists in DB
- Saves user message
- Loads conversation history
- Creates GeneratedObject (pending)
- Calls streamText() with context tools
5. Tool Execution Loop:
- AI requests tool call
- Execute tool
- Return result to AI
6. AI streams response chunks
7. StreamService sends SSE to client
8. Widget displays streaming text
9. StreamService saves assistant message
10. Updates GeneratedObject (success + metrics)
Quick Reference
Adding Chatbot to a Page
import { ChatbotWidget } from "~/modules/chatbot/components/chatbot-widget";
<ChatbotWidget
context="tenant" // "tenant" | "admin" | "marketing"
chatId={chatId} // Optional: existing chat
initialMessages={[]} // Optional: preload messages
readOnly={false} // Optional: disable input
/>;
Creating a New Context
- Create
tools-{context}.server.tsinapps/web/app/modules/chatbot/lib/ - Export
CONTEXT_SYSTEM_PROMPT,getContextChatTools,ContextChatContext - Register the service API in
apps/api/src/app.tsunder/v1 - Add context config to
chatbot-config.ts - Create chat routes at
apps/web/app/routes/{path}/chat/
Enabling MCP Integration
The tenant chat automatically connects to the MCP server at /mcp when available. MCP tools extend the chatbot's capabilities with additional data access.
See MCP Server documentation for available tools.