nextjs-ai-chat
v0.0.27
Published
AI Chat components with voice support
Maintainers
Readme
@layouts.dev/ai-chat
Production-ready AI chat components for Next.js with real-time voice support, built on top of Vercel AI SDK and OpenAI Realtime API.
Features
- 🎯 Drop-in Components - Pre-built, customizable React components
- 🎤 Voice Mode - Real-time voice conversations with OpenAI
- 🔌 Headless Architecture - Full control over styling and behavior
- 📦 Type-Safe - Full TypeScript support with intelligent autocomplete
- ⚡ Zero Config - Works out of the box with sensible defaults
- 🎨 Fully Customizable - Render props pattern for complete control
Installation
npm install @layouts.dev/ai-chat
# or
pnpm add @layouts.dev/ai-chat
# or
yarn add @layouts.dev/ai-chatQuick Start
1. Set up environment
The OPENAI_API_KEY is required if you:
- Use voice mode (OpenAI Realtime API)
- Use the default chat API (without custom provider override)
The GLADIA_API_KEY is required if you:
- Use voice input (press-to-talk speech-to-text feature)
The Backblaze B2 keys are required if you:
- Use file uploads without providing a custom
uploadProvider
The LLAMA_CLOUD_API_KEY is required if you:
- Use voice mode AND file uploads without providing a custom
uploadTextExtractor
If you override the chat API with a custom provider (Anthropic, Google, etc.), you'll need the corresponding API key for that provider instead (e.g., ANTHROPIC_API_KEY). Voice mode still requires OPENAI_API_KEY.
# AI Provider (required for chat and voice mode)
OPENAI_API_KEY=sk-...your_openai_key...
# Voice Input (required for press-to-talk feature)
GLADIA_API_KEY=...your_gladia_key...
# File Storage (required if not providing custom uploadProvider)
B2_ACCOUNT_ID=...your_backblaze_account_id...
B2_APPLICATION_KEY=...your_backblaze_app_key...
B2_BUCKET_ID=...your_backblaze_bucket_id...
B2_BUCKET_NAME=...your_backblaze_bucket_name...
# Text Extraction (required for voice mode with uploads if not providing custom uploadTextExtractor)
LLAMA_CLOUD_API_KEY=llx-...your_llama_cloud_key...
# Or if using Anthropic:
# ANTHROPIC_API_KEY=sk-ant-...
# Or if using Google:
# GOOGLE_GENERATIVE_AI_API_KEY=...2. Set up API Routes
Create a catch-all API route to handle chat and voice requests:
// app/ai-chat/[...aiChat]/route.ts
export * from "@layouts.dev/ai-chat/endpoints";Advanced: Server-Side Configuration
By default, the package uses OpenAI models and the system prompt provided by the client (via dependencies.defaultPrompt). You can customize both the AI model and system prompt server-side using the chat() factory function.
Server-side system prompts override client-provided prompts. This is useful for:
- Enforcing specific AI behavior that clients cannot modify
- Adding server-side context (user info, permissions, etc.)
- Using different prompts based on request data
// app/ai-chat/[...aiChat]/route.ts
import chat from "@layouts.dev/ai-chat/endpoints";
import { anthropic } from "@ai-sdk/anthropic";
const endpoints = chat({
// Use a different AI provider (optional)
model: anthropic("claude-sonnet-4-5"),
// Override the system prompt server-side (optional)
// This takes precedence over client's dependencies.defaultPrompt
systemPrompt: "You are a helpful assistant. Be concise and professional.",
});
export const { POST } = endpoints;Dynamic System Prompt:
You can also provide a function that receives the request body, allowing you to generate prompts dynamically based on request data:
import chat from "@layouts.dev/ai-chat/endpoints";
import type { ChatRequestBody } from "@layouts.dev/ai-chat";
import { anthropic } from "@ai-sdk/anthropic";
const endpoints = chat({
model: anthropic("claude-sonnet-4-5"),
// Dynamic prompt based on request data
systemPrompt: (body: ChatRequestBody) => {
// Access client-provided prompt if needed
const clientPrompt = body.agentPrompt || "";
// Build server-controlled prompt
return `You are a helpful assistant.
Additional context from client: ${clientPrompt}
Always be professional and concise.`;
},
});
export const { POST } = endpoints;Async System Prompt:
The function can also be async, useful for fetching user context from a database:
import chat from "@layouts.dev/ai-chat/endpoints";
import type { ChatRequestBody } from "@layouts.dev/ai-chat";
import { anthropic } from "@ai-sdk/anthropic";
import { getUserContext } from "./db";
const endpoints = chat({
model: anthropic("claude-sonnet-4-5"),
systemPrompt: async (body: ChatRequestBody) => {
// Fetch user-specific context from database
const userContext = await getUserContext(body.userId);
return `You are assisting ${userContext.name}.
Their subscription: ${userContext.plan}
Their preferences: ${JSON.stringify(userContext.preferences)}
Be helpful and respect their preferences.`;
},
});
export const { POST } = endpoints;Request Body Properties:
The body parameter contains:
messages- Chat message historyagentPrompt- Client-provided system prompt (fromdependencies.defaultPrompt)clientTools- Client-defined toolsagentModel- Client-requested model- Plus any additional properties sent by the client
Note:
- The
systemPromptconfig applies to both text chat and voice mode - Voice mode (
realtimeendpoint) currently only supports OpenAI models, so themodelconfig only affects text chat - If
systemPromptis not provided, the client'sagentPromptis used as fallback
3. Create Chat Dependencies (Optional)
All dependencies are optional. If you don't provide any, the chat will work with default values and in-memory storage (messages won't persist).
Example with all available options:
// app/chat-dependencies.tsx
import type { ChatDependencies } from '@layouts.dev/ai-chat';
export function createChatDependencies(): ChatDependencies {
return {
// Database Integration (optional) - If omitted, messages are stored in memory only
useChatById: (chatId: string) => ({
chat: chatId ? { id: chatId, messages: [] } : null,
getChat: async () => chatId ? { id: chatId, messages: [] } : null,
updateChat: async (data) => {
// Save messages to your database
console.log('Saving chat:', chatId, data.messages);
},
loading: false,
}),
// Create new chat (optional) - If omitted, chat IDs are auto-generated
createChat: async () => {
const newChatId = `chat_${Date.now()}`;
console.log('Created new chat:', newChatId);
return newChatId;
},
// AI Configuration (optional)
defaultPrompt: 'You are a helpful AI assistant...',
defaultVoice: 'cedar', // OpenAI voice: alloy, ash, ballad, coral, echo, sage, shimmer, verse, cedar, marin
// Model Configuration (optional)
defaultAgentModel: 'gpt-4.1-mini', // OpenAI model for chat
defaultVoiceModeModel: 'gpt-realtime', // OpenAI model for voice mode
defaultVoiceModeTranscriptionModel: 'gpt-4o-transcribe', // OpenAI transcription model
// API Path (optional, default: '/ai-chat')
apiPath: '/ai-chat',
// Custom fetch function (optional) - for custom auth headers, etc.
fetch: async (input, init) => {
const headers = new Headers(init?.headers);
headers.set('Authorization', `Bearer ${getAuthToken()}`);
return globalThis.fetch(input, { ...init, headers });
},
};
}ChatDependencies Interface:
interface ChatDependencies {
// Database hooks for persisting chat history (optional)
useChatById?: (chatId: string) => {
chat: { id: string; messages: UIMessage[] } | null;
getChat: () => Promise<{ id: string; messages: UIMessage[] } | null>;
updateChat: (data: { messages: UIMessage[] }) => Promise<void>;
loading: boolean;
};
// Create a new chat session (optional)
createChat?: () => Promise<string>;
// System prompt for the AI (optional)
defaultPrompt?: string;
// OpenAI Realtime voice (for voice mode) (optional)
defaultVoice?: "alloy" | "ash" | "ballad" | "coral" | "echo" |
"sage" | "shimmer" | "verse" | "cedar" | "marin";
// Model Configuration (optional)
defaultAgentModel?: string; // OpenAI model ID for chat (default: 'gpt-4.1-mini')
defaultVoiceModeModel?: string; // OpenAI model ID for voice mode (default: 'gpt-realtime')
defaultVoiceModeTranscriptionModel?: string; // OpenAI transcription model (default: 'gpt-4o-transcribe')
// API endpoint path (optional, default: '/ai-chat')
apiPath?: string;
// Client-side tools (optional) - Custom tools that can execute any logic
clientTools?: ToolDescriptor[];
// Default active tools (optional) - If not provided, all tools are active by default
defaultActiveToolsNames?: string[];
// Custom fetch function (optional) - Override fetch for custom auth, headers, etc.
fetch?: typeof globalThis.fetch;
}Key Points:
- All properties are optional - The library provides sensible defaults for everything
useChatById(optional) - Hook to fetch and persist chat messages in your databasecreateChat(optional) - Function to create new chat sessions (returns chat ID)defaultPrompt(optional) - System instructions for the AI modeldefaultVoice(optional) - Voice for OpenAI Realtime API (used in voice mode)defaultAgentModel(optional) - OpenAI model ID for chat (default: 'gpt-4.1-mini')defaultVoiceModeModel(optional) - OpenAI model ID for voice mode (default: 'gpt-realtime')defaultVoiceModeTranscriptionModel(optional) - OpenAI transcription model (default: 'gpt-4o-transcribe')apiPath(optional) - Custom API route path (must match your API route setup)clientTools(optional) - Array of tools that execute client-side (see Client-Side Tools)defaultActiveToolsNames(optional) - List of tool names to activate by defaultfetch(optional) - Custom fetch function for adding authentication headers or custom request handling
4. Use the Components
Simplest Example (no configuration needed):
"use client";
import { AIChat } from "@layouts.dev/ai-chat"
export default function Home() {
return (
<div className="flex min-h-screen bg-zinc-50 font-sans dark:bg-black">
<AIChat.Provider>
<div className="flex flex-col h-screen w-full">
<div className="absolute m-2">
<AIChat.NewChatButton />
</div>
<div className="flex-1 overflow-y-auto p-4">
<AIChat.Messages />
</div>
<div className="p-4 border-t border-zinc-200 dark:border-zinc-800">
<div className="relative">
<div className="absolute bottom-full left-1/2 -translate-x-1/2 mb-4">
<AIChat.VoiceInputSpeaker />
<AIChat.VoiceModeSpeaker />
</div>
<AIChat.AttachedFiles />
<AIChat.Textarea />
</div>
</div>
</div>
</AIChat.Provider>
</div>
);
}Complete Example with custom dependencies:
"use client";
import { AIChat } from "@layouts.dev/ai-chat"
import { createChatDependencies } from "./chat-dependencies"
export default function Home() {
const dependencies = createChatDependencies();
return (
<div className="flex min-h-screen bg-zinc-50 font-sans dark:bg-black">
<AIChat.Provider
allowUploads={true}
allowVoiceMode={true}
allowVoiceInput={true}
dependencies={dependencies}
>
<div className="flex flex-col h-screen w-full">
<div className="absolute m-2">
<AIChat.NewChatButton />
</div>
<div className="flex-1 overflow-y-auto p-4">
<AIChat.Messages />
</div>
<div className="p-4 border-t border-zinc-200 dark:border-zinc-800">
<div className="relative">
<div className="absolute bottom-full left-1/2 -translate-x-1/2 mb-4">
<AIChat.VoiceInputSpeaker />
<AIChat.VoiceModeSpeaker />
</div>
<AIChat.AttachedFiles />
<AIChat.Textarea />
</div>
</div>
</div>
</AIChat.Provider>
</div>
);
}Components
<AIChat.Provider>
Root provider that manages chat state and voice connections.
<AIChat.Provider
dependencies={chatDependencies}
allowVoiceMode={true}
allowVoiceInput={true}
allowUploads={false}
>
{children}
</AIChat.Provider>Props:
dependencies(optional) - Chat configuration (database hooks, API path, prompts, etc.). If omitted, uses default values with in-memory storageallowVoiceMode(optional) - Enable/disable voice mode (default:true)allowVoiceInput(optional) - Enable/disable voice input using Gladia for transcription (default:true)allowUploads(optional) - Enable/disable file uploads (default:true)uploadProvider(optional) - Custom upload provider for file storageuploadTextExtractor(optional) - Custom function to extract text from files:(file: File) => Promise<string>. Important: If you provide a custom text extractor, you must also updateuploadAllowedTextExtractFileTypesto match the MIME types your extractor supports.uploadGenerateThumbnail(optional) - Custom function to generate thumbnails:(file: File) => Promise<string | undefined>onUploadOpen(optional) - Custom file picker function:(allowedFileTypes: string[]) => Promise<FileList | null>uploadAllowedFilesTypes(optional) - Array of allowed MIME types for uploads (default:NATIVE_GPT4O_MIME_TYPES). Important: If you change the chat model to a non-OpenAI model (e.g., Anthropic, Google), you must update this array to match the file types supported by your chosen model's vision capabilities.uploadAllowedTextExtractFileTypes(optional) - Array of MIME types that support text extraction (default:LLAMAPARSE_MIME_TYPES). Must match the capabilities of youruploadTextExtractor.
<AIChat.Messages>
Displays chat message history with auto-scroll.
Basic Usage (Default Styling):
<AIChat.Messages />Customizing Individual Message Types (Recommended):
Use slot components to customize only specific message types while keeping defaults for the rest:
<AIChat.Messages>
<AIChat.Messages.User>
{({ message }) => (
<div className="flex justify-end my-2">
<div className="max-w-[80%] rounded-2xl bg-blue-500 px-4 py-2 text-white">
{message}
</div>
</div>
)}
</AIChat.Messages.User>
<AIChat.Messages.Assistant>
{({ message }) => (
<div className="flex justify-start my-2">
<div className="max-w-[80%] rounded-2xl bg-gray-100 px-4 py-2">
{message}
</div>
</div>
)}
</AIChat.Messages.Assistant>
</AIChat.Messages>Available Slot Components:
<AIChat.Messages.User>- Customize user messages<AIChat.Messages.Assistant>- Customize assistant messages<AIChat.Messages.Tool>- Customize tool call displays<AIChat.Messages.File>- Customize file attachment displays
Slot Props:
Each slot receives specific props for its message type:
// User slot props
<AIChat.Messages.User>
{({ message }: { message: string }) => <div>{message}</div>}
</AIChat.Messages.User>
// Assistant slot props
<AIChat.Messages.Assistant>
{({ message }: { message: string }) => <div>{message}</div>}
</AIChat.Messages.Assistant>
// Tool slot props
<AIChat.Messages.Tool>
{({ tool }: { tool: ToolPart }) => (
<div>
<div>Tool: {tool.type.substring(5)}</div>
<div>State: {tool.state}</div>
<pre>{JSON.stringify(tool.input, null, 2)}</pre>
</div>
)}
</AIChat.Messages.Tool>
// File slot props
<AIChat.Messages.File>
{({ thumbnail, mediaType }: { thumbnail?: string; mediaType?: string }) => (
<img src={thumbnail} alt={mediaType} />
)}
</AIChat.Messages.File>Slots Accept Multiple Formats:
// Render function
<AIChat.Messages.User>
{({ message }) => <div>{message}</div>}
</AIChat.Messages.User>
// Component reference
<AIChat.Messages.User>{MyUserMessage}</AIChat.Messages.User>
// Element (props auto-injected)
<AIChat.Messages.User>
<MyUserMessage />
</AIChat.Messages.User>Legacy: Custom Rendering All Types (with render function):
You can still provide a single function to handle all message types:
<AIChat.Messages>
{({ type, message, tool }) => {
if (type === "user") {
return (
<div className="flex justify-end my-2">
<div className="max-w-[80%] rounded-2xl bg-blue-500 px-4 py-2 text-sm text-white shadow">
{message}
</div>
</div>
);
}
if (type === "assistant") {
return (
<div className="flex justify-start my-2">
<div className="max-w-[80%] rounded-2xl bg-gray-100 px-4 py-2 text-sm text-gray-900 shadow-sm">
{message}
</div>
</div>
);
}
if (type === "toolcall") {
return (
<div className="flex justify-start my-2">
<div className="max-w-[70%] rounded-xl border border-yellow-300 bg-yellow-50 px-4 py-2 text-sm text-yellow-800 shadow-sm">
<div className="font-semibold">Tool Call: {tool.type.substring(5)}</div>
<div className="text-xs text-yellow-900">
{tool.state === "input-streaming"
? `Processing: ${JSON.stringify(tool.input)}`
: `Completed: ${JSON.stringify(tool.output)}`}
</div>
</div>
</div>
);
}
}}
</AIChat.Messages>Using a Component:
const MessageComponent = ({ type, message, tool }) => {
if (type === "user") {
return <div className="user-message">{message}</div>;
}
if (type === "assistant") {
return <div className="assistant-message">{message}</div>;
}
return null;
};
// Pass as function reference
<AIChat.Messages>{MessageComponent}</AIChat.Messages>
// Or as element (props auto-injected)
<AIChat.Messages><MessageComponent /></AIChat.Messages>Render Props (Legacy API):
type- Message type:"user"|"assistant"|"toolcall"message- Message text content (optional)tool- Tool call details (whentype === "toolcall")tool.type- Tool name (string starting with "tool_" prefix)tool.state- Tool execution state:"input-streaming"|"completed"|"done"tool.input- Tool input parameterstool.output- Tool execution result
Accepted Formats:
- Slot Components:
<AIChat.Messages.User>,.Assistant,.Tool,.File- Customize individual types - Function:
{(props) => <div>...</div>}- Render function with props (handles all types) - Component:
{MyComponent}- Component function reference - Element:
<MyComponent />- React element (props auto-injected via cloneElement)
<AIChat.Textarea>
Auto-growing text input with form handling.
Basic Usage (Default Styling):
<AIChat.Textarea />Custom Textarea (with element):
<AIChat.Textarea>
<textarea
className="w-full p-3 resize-none text-sm bg-white/85 rounded-lg border border-gray-300 min-h-10 max-h-32 overflow-y-auto focus:outline-none focus:ring-2 focus:ring-blue-500"
placeholder="Type your message here..."
rows={2}
/>
</AIChat.Textarea>Using a render function:
<AIChat.Textarea>
{(props) => (
<textarea
{...props}
className="custom-textarea"
placeholder="Type here..."
/>
)}
</AIChat.Textarea>Props passed to children:
ref- Textarea ref for auto-grow functionalityvalue- Current input valueonChange- Change handleronKeyDown- Key handler (Enter to submit, Shift+Enter for new line)disabled- Disabled stateplaceholder- Placeholder text
Accepted Formats:
- Element:
<textarea />- React element (props auto-injected) - Function:
{(props) => <textarea {...props} />}- Render function with props - Component:
{MyTextareaComponent}- Component reference
<AIChat.SendButton>
Button to submit the current textarea input (same behavior as pressing Enter).
Basic Usage (Default Styling):
<AIChat.SendButton />Custom Button (with render function):
<AIChat.SendButton>
{({ onClick, disabled }) => (
<button
onClick={onClick}
disabled={disabled}
className="px-4 py-2 bg-blue-500 text-white rounded-lg disabled:bg-gray-300 disabled:cursor-not-allowed"
>
Send Message
</button>
)}
</AIChat.SendButton>Using an element:
<AIChat.SendButton>
<button className="send-button">📤</button>
</AIChat.SendButton>Render Props:
disabled- Whether button is disabled (no text, loading, voice mode active, or AI is still generating):boolean
Props injected to children:
onClick- Click handler to submit the messagedisabled- Auto-disabled when textarea is empty, loading, voice mode is active, or AI is still generating
Accepted Formats:
- Function:
{({ onClick, disabled }) => <button>...</button>}- Render function with props - Element:
<button />- React element (props auto-injected) - Component:
{MyButtonComponent}- Component reference
Note: The button shares the same
textareaInputstate with<AIChat.Textarea>and clears it after sending.
<AIChat.VoiceModeButton>
Toggle button for voice mode activation/deactivation.
Basic Usage (Default Styling):
<AIChat.VoiceModeButton />Custom Button (with render function):
<AIChat.VoiceModeButton>
{({ status }) => {
if (status === "CONNECTED") {
return (
<button className="flex items-center justify-center p-3 size-8 bg-gray-200 text-white rounded-full disabled:opacity-50 shrink-0">
<span>🔊</span>
</button>
);
}
if (status === "CONNECTING") {
return (
<button className="flex items-center justify-center p-3 size-8 bg-gray-200 text-white rounded-full disabled:opacity-50 shrink-0">
...
</button>
);
}
return (
<button className="flex items-center justify-center p-3 size-8 bg-gray-200 text-white rounded-full disabled:opacity-50 shrink-0">
<span>🔇</span>
</button>
);
}}
</AIChat.VoiceModeButton>Using an element:
<AIChat.VoiceModeButton>
<button className="voice-button">Toggle Voice</button>
</AIChat.VoiceModeButton>Render Props:
status- Connection state:"DISCONNECTED"|"CONNECTING"|"CONNECTED"
Props injected to children:
onClick- Click handler to toggle voice modedisabled- Auto-disabled during"CONNECTING"state
Accepted Formats:
- Function:
{({ status }) => <button>...</button>}- Render function with status - Element:
<button />- React element (props auto-injected) - Component:
{MyButtonComponent}- Component reference
<AIChat.VoiceModeSpeaker>
Displays who is currently speaking in voice mode.
Basic Usage (Default Styling):
<AIChat.VoiceModeSpeaker />Custom Rendering (with render function):
<AIChat.VoiceModeSpeaker>
{({ whoIsSpeaking }) => {
if (whoIsSpeaking === "user") {
return (
<div className="flex items-center gap-3 bg-linear-to-r from-green-500/10 to-emerald-500/10 backdrop-blur-sm px-6 py-3 rounded-2xl shadow-lg border border-green-200">
<span className="text-lg">🎤</span>
<span className="text-sm font-semibold text-green-700">You're speaking...</span>
<AIChat.VoiceModeMicIndicator userColor="text-green-500" />
</div>
);
}
if (whoIsSpeaking === "assistant") {
return (
<div className="flex items-center gap-3 bg-linear-to-r from-blue-500/10 to-purple-500/10 backdrop-blur-sm px-6 py-3 rounded-2xl shadow-lg border border-blue-200">
<span className="text-lg">🤖</span>
<span className="text-sm font-semibold text-blue-700">Assistant is speaking...</span>
<AIChat.VoiceModeMicIndicator assistantColor="text-blue-500" />
</div>
);
}
return null;
}}
</AIChat.VoiceModeSpeaker>Using an element:
<AIChat.VoiceModeSpeaker>
<div className="speaker-indicator" />
</AIChat.VoiceModeSpeaker>Render Props:
whoIsSpeaking- Current speaker:"user"|"assistant"|null
Props injected to children:
whoIsSpeaking- Current speaker state
Accepted Formats:
- Function:
{({ whoIsSpeaking }) => <div>...</div>}- Render function - Element:
<div />- React element (props auto-injected) - Component:
{MySpeakerComponent}- Component reference
Note: Returns
nullwhen no one is speaking. Only shows during active voice mode.
<AIChat.VoiceModeMicIndicator>
Visual FFT visualization for voice activity (animated frequency bars).
Basic Usage (Default Styling):
<AIChat.VoiceModeMicIndicator />Custom Styling (with props):
<AIChat.VoiceModeMicIndicator
barCount={24}
slideSpeed={12}
divider={4}
userColor="text-green-500"
assistantColor="text-blue-500"
size="size-6"
/>Custom Rendering (with render function):
<AIChat.VoiceModeMicIndicator>
{({ fftData, whoIsSpeaking }) => (
<div className="custom-indicator">
{/* Your custom visualization using fftData */}
<span>{whoIsSpeaking === "user" ? "🎤" : "🤖"}</span>
</div>
)}
</AIChat.VoiceModeMicIndicator>Props:
barCount- Number of frequency bars (default:8)slideSpeed- Animation speed in bars/second (default:20)divider- FFT data divider for height scaling (default:2)userColor- Tailwind text color for user speech (default:"text-green-500/80")assistantColor- Tailwind text color for assistant speech (default:"text-blue-500/80")size- Additional CSS classes for sizing (default:"size-[24px]")
Props passed to children (when using render function):
fftData- Array of frequency data (number[])whoIsSpeaking- Current speaker:"user"|"assistant"|null
Accepted Formats:
- No children: Uses default FFT visualization with props
- Function:
{({ fftData, whoIsSpeaking }) => <div>...</div>}- Custom render - Element:
<div />- React element (props auto-injected) - Component:
{MyIndicatorComponent}- Component reference
Note: Props (
barCount,slideSpeed, etc.) are only used when no children are provided. With children, onlyfftDataandwhoIsSpeakingare passed.
<AIChat.VoiceInputButton>
Press-to-talk button for speech-to-text input (requires GLADIA_API_KEY).
Basic Usage (Default Styling):
<AIChat.VoiceInputButton />Custom Button (with render function):
<AIChat.VoiceInputButton>
{({ isRecording, disabled }) => (
<button
className={`p-3 rounded-full ${isRecording ? 'bg-red-500' : 'bg-blue-500'}`}
disabled={disabled}
>
{isRecording ? '🔴' : '🎤'}
</button>
)}
</AIChat.VoiceInputButton>Using an element:
<AIChat.VoiceInputButton>
<button className="voice-input-btn">Press to Talk</button>
</AIChat.VoiceInputButton>Render Props:
isRecording- Whether currently recording audio:booleandisabled- Whether button is disabled (loading or AI is still generating):boolean
Props injected to children:
onMouseDown- Start recording on mouse/touch downonMouseUp- Stop recording on mouse/touch uponMouseLeave- Stop recording when mouse leavesonTouchStart- Start recording on touchonTouchEnd- Stop recording on touch enddisabled- Auto-disabled when loading or AI is still generating
Accepted Formats:
- Function:
{({ isRecording, disabled }) => <button>...</button>}- Render function - Element:
<button />- React element (event handlers auto-injected) - Component:
{MyButtonComponent}- Component reference
Note: This component uses Gladia's speech-to-text API. Make sure to set
GLADIA_API_KEYin your environment variables.
<AIChat.VoiceInputSpeaker>
Displays the current status of voice input recording/transcription.
Basic Usage (Default Styling):
<AIChat.VoiceInputSpeaker />Custom Rendering (with render function):
<AIChat.VoiceInputSpeaker>
{({ isListening, isTranscribing, liveTranscript }) => {
if (isListening) {
return (
<div className="flex items-center gap-3 bg-green-100 px-6 py-3 rounded-2xl">
<span className="text-lg">🎤</span>
<span className="text-sm font-semibold text-green-700">
{liveTranscript || "Listening..."}
</span>
<AIChat.VoiceInputMicIndicator color="#22c55e" />
</div>
);
}
if (isTranscribing) {
return (
<div className="flex items-center gap-3 bg-blue-100 px-6 py-3 rounded-2xl">
<span className="text-lg">⏳</span>
<span className="text-sm font-semibold text-blue-700">
Transcribing...
</span>
</div>
);
}
return null;
}}
</AIChat.VoiceInputSpeaker>Render Props:
isListening- Whether currently recording:booleanisTranscribing- Whether transcribing audio:booleanliveTranscript- Real-time transcription text:string
Props injected to children:
isListening- Recording stateisTranscribing- Transcription stateliveTranscript- Live transcript text
Accepted Formats:
- Function:
{({ isListening, isTranscribing, liveTranscript }) => <div>...</div>}- Render function - Element:
<div />- React element (props auto-injected) - Component:
{MySpeakerComponent}- Component reference
Note: Returns
nullwhen not listening or transcribing. Only shows during voice input activity.
<AIChat.UploadFileButton>
Button to open file picker and upload files to the chat.
Basic Usage (Default Styling):
<AIChat.UploadFileButton />Custom Button (with render function):
<AIChat.UploadFileButton>
{({ onClick }) => (
<button onClick={onClick} className="upload-btn">
📎 Attach Files
</button>
)}
</AIChat.UploadFileButton>Using an element:
<AIChat.UploadFileButton>
<button className="custom-upload">Upload</button>
</AIChat.UploadFileButton>Render Props:
onClick- Async click handler to open file picker and upload files:(e: React.MouseEvent<HTMLButtonElement>) => Promise<void>
Props injected to children:
onClick- File upload handler
Accepted Formats:
- Function:
{({ onClick }) => <button>...</button>}- Render function with props - Element:
<button />- React element (props auto-injected) - Component:
{MyButtonComponent}- Component reference
Note: Files are automatically uploaded and processed. In voice mode, only text-extractable files are allowed. The upload respects the
uploadAllowedFilesTypesanduploadAllowedTextExtractFileTypesconfiguration from the Provider.
<AIChat.AttachedFiles>
Displays preview of attached files before sending the message.
Basic Usage (Default Styling):
<AIChat.AttachedFiles />Custom Rendering (with render function):
<AIChat.AttachedFiles>
{({ files, onRemoveFile }) => (
<div className="file-list">
{files.map(file => (
<div key={file.fileId} className="file-item">
<span>{file.file.name}</span>
<button onClick={() => onRemoveFile(file)}>×</button>
</div>
))}
</div>
)}
</AIChat.AttachedFiles>Using a Component:
const FileList = ({ files, onRemoveFile }) => (
<div>
{files.map(file => (
<div key={file.fileId}>
{file.thumbnailBase64 && (
<img src={`data:image/jpeg;base64,${file.thumbnailBase64}`} />
)}
<span>{file.file.name}</span>
<button onClick={() => onRemoveFile(file)}>Remove</button>
</div>
))}
</div>
);
<AIChat.AttachedFiles>{FileList}</AIChat.AttachedFiles>Render Props:
files- Array of uploaded file metadata:UploadedFileMeta[]fileId- Unique file identifier:stringfile- The File object:FilethumbnailBase64- Base64 thumbnail image (if available):string | undefinedstatus- Upload status:"pending" | "uploading" | "extracting-text" | "completed" | "failed" | "aborted"progress- Upload progress (0-100):numberurl- Remote file URL:stringdata- Extracted text or file data:string
onRemoveFile- Function to remove a file:(file: UploadedFileMeta) => void
Accepted Formats:
- Function:
{({ files, onRemoveFile }) => <div>...</div>}- Render function with props - Element:
<div />- React element (props auto-injected) - Component:
{MyComponent}- Component reference
Note: Returns
nullwhen no files are attached. The component automatically shows loading states during upload and extraction.
<AIChat.VoiceInputMicIndicator>
Visual FFT visualization for voice input recording (animated frequency bars).
Basic Usage (Default Styling):
<AIChat.VoiceInputMicIndicator />Custom Styling (with props):
<AIChat.VoiceInputMicIndicator
barCount={8}
slideSpeed={20}
divider={2}
color="#22c55e"
size="24px"
/>Custom Rendering (with render function):
<AIChat.VoiceInputMicIndicator>
{({ fftData, isListening, isTranscribing }) => (
<div className="custom-indicator">
{/* Your custom visualization using fftData */}
<span>{isListening ? "🎤" : "⏳"}</span>
</div>
)}
</AIChat.VoiceInputMicIndicator>Props:
barCount- Number of frequency bars (default:8)slideSpeed- Animation speed in bars/second (default:20)divider- FFT data divider for height scaling (default:2)color- CSS color for bars (default:"#22c55e")size- Size in CSS units (default:"24px")
Props passed to children (when using render function):
fftData- Array of frequency data (number[])isListening- Whether currently recording:booleanisTranscribing- Whether transcribing:boolean
Accepted Formats:
- No children: Uses default FFT visualization with props
- Function:
{({ fftData, isListening, isTranscribing }) => <div>...</div>}- Custom render - Element:
<div />- React element (props auto-injected) - Component:
{MyIndicatorComponent}- Component reference
Note: Props (
barCount,slideSpeed, etc.) are only used when no children are provided. With children, onlyfftData,isListening, andisTranscribingare passed.
<AIChat.NewChatButton>
Button to start a new conversation.
Basic Usage (Default Styling):
<AIChat.NewChatButton />Custom Button (with element):
<AIChat.NewChatButton>
<button className="btn-primary">
+ New Chat
</button>
</AIChat.NewChatButton>Using a render function:
<AIChat.NewChatButton>
{() => <button className="custom-btn">Start New</button>}
</AIChat.NewChatButton>Props injected to children:
onClick- Click handler to create a new chat
Accepted Formats:
- Element:
<button />- React element (onClick auto-injected) - Function:
{() => <button>...</button>}- Render function - Component:
{MyButtonComponent}- Component reference
<AIChat.WithAdditionalAIContext>
Wraps part of your application to temporarily add context messages or enable specific tools only within that section. This is useful for contextual AI interactions where you want different prompts or tools based on what the user is viewing.
Basic Usage:
<AIChat.WithAdditionalAIContext
systemPrompt="You are helping the user edit their profile. Be concise and helpful."
tools={["updateProfile", "uploadAvatar"]}
>
<ProfileEditor />
</AIChat.WithAdditionalAIContext>Real-world Example:
// Different contexts for different pages
function DocumentEditor() {
return (
<AIChat.WithAdditionalAIContext
systemPrompt="You are assisting with document editing. Focus on formatting, grammar, and structure."
tools={["formatText", "insertTable", "checkGrammar"]}
>
<Editor />
</AIChat.WithAdditionalAIContext>
);
}
function ImageGallery() {
return (
<AIChat.WithAdditionalAIContext
systemPrompt="You are helping with image management. Suggest edits and organization."
tools={["cropImage", "addFilter", "organizeByDate"]}
>
<Gallery />
</AIChat.WithAdditionalAIContext>
);
}Props:
systemPrompt(optional) - Additional system message to inject into the conversation context while this component is mountedtools(optional) - Array of tool names to enable only within this contextchildren- React nodes to render
How it works:
- When mounted, adds the
systemPromptto the AI's context and enables specifiedtools - When unmounted, automatically removes the context and disables the tools
- Multiple
WithAdditionalAIContextcomponents can be nested or used in parallel - Context messages are sent with every user message while the component is active
Use cases:
- Page-specific AI behavior (different prompts per route)
- Feature-specific tools (enable photo editing tools only in gallery)
- Temporary context (show user's current selection to AI)
- Progressive disclosure (unlock advanced tools in specific UI states)
Hooks
useAIChat()
A hook to access AI chat state and handlers. Use this to build custom UI or integrate chat functionality into your application.
Basic Usage:
import { useAIChat } from '@layouts.dev/ai-chat';
function MyCustomChat() {
const { chatId, messages, loading, append, newChat } = useAIChat();
const handleSend = async () => {
await append("Hello, AI!");
};
return (
<div>
<p>Chat ID: {chatId}</p>
<p>Messages: {messages.length}</p>
<button onClick={handleSend} disabled={loading}>Send</button>
<button onClick={newChat}>New Chat</button>
</div>
);
}Returns:
| Property | Type | Description |
|----------|------|-------------|
| Core State | | |
| chatId | string | Current chat session ID |
| messages | UIMessage[] | Array of chat messages |
| loading | boolean | Whether the chat is loading or AI is generating |
| Core Actions | | |
| append | (message, role?) => Promise<void> | Send a message to the AI |
| newChat | () => Promise<void> | Start a new conversation |
| setChatId | (id: string) => void | Switch to a different chat session |
| Voice Mode | | |
| voiceModeStatus | "DISCONNECTED" \| "CONNECTING" \| "CONNECTED" | Voice mode connection state |
| startVoiceMode | () => void | Start voice mode |
| stopVoiceMode | () => void | Stop voice mode |
| interrupt | () => void | Interrupt assistant speech |
| mute | (muted: boolean) => void | Mute/unmute microphone |
| pushToTalkStart | () => void | Start push-to-talk |
| pushToTalkStop | () => void | Stop push-to-talk |
| whoIsSpeaking | "user" \| "assistant" \| null | Who is currently speaking |
| Tools | | |
| currentTools | string[] | Currently active tool names |
| addTools | (owner, tools) => void | Add tools dynamically |
| removeTools | (owner) => void | Remove tools by owner |
| Context | | |
| addContextMessage | (message: string) => void | Add context for AI |
| removeContextMessage | (message: string) => void | Remove context |
| Model Config | | |
| setVoiceModeModel | (model: string) => void | Change voice mode model |
| setVoiceModeTranscriptionModel | (model: string) => void | Change transcription model |
| setVoice | (voice: string) => void | Change AI voice |
| setAgentModel | (model: string) => void | Change chat model |
| FFT Data | | |
| fftData | number[] | User's audio frequency data |
| assistantFFTData | number[] | Assistant's audio frequency data |
Example: Custom Send Form:
function CustomMessageForm() {
const { append, loading } = useAIChat();
const [input, setInput] = useState('');
const handleSubmit = async (e: React.FormEvent) => {
e.preventDefault();
if (!input.trim() || loading) return;
await append(input);
setInput('');
};
return (
<form onSubmit={handleSubmit}>
<input
value={input}
onChange={(e) => setInput(e.target.value)}
placeholder="Type a message..."
disabled={loading}
/>
<button type="submit" disabled={loading || !input.trim()}>
{loading ? 'Sending...' : 'Send'}
</button>
</form>
);
}Example: Chat Switcher:
function ChatSwitcher() {
const { chatId, setChatId, newChat } = useAIChat();
const [chatHistory] = useState(['chat_1', 'chat_2', 'chat_3']);
return (
<div>
<p>Current: {chatId}</p>
<select value={chatId} onChange={(e) => setChatId(e.target.value)}>
{chatHistory.map(id => (
<option key={id} value={id}>{id}</option>
))}
</select>
<button onClick={newChat}>+ New Chat</button>
</div>
);
}Example: Voice Mode Controls:
function VoiceControls() {
const {
voiceModeStatus,
startVoiceMode,
stopVoiceMode,
whoIsSpeaking
} = useAIChat();
return (
<div>
<p>Status: {voiceModeStatus}</p>
<p>Speaking: {whoIsSpeaking || 'Nobody'}</p>
{voiceModeStatus === 'DISCONNECTED' ? (
<button onClick={startVoiceMode}>Start Voice Mode</button>
) : voiceModeStatus === 'CONNECTED' ? (
<button onClick={stopVoiceMode}>Stop Voice Mode</button>
) : (
<button disabled>Connecting...</button>
)}
</div>
);
}Note: This hook must be used within an
<AIChat.Provider>component.
useAIChatUpload()
A hook to programmatically manage file uploads in the chat. Useful for custom upload flows, drag-and-drop implementations, or integrating with external file sources.
Usage:
import { useAIChatUpload } from '@layouts.dev/ai-chat';
function MyCustomUploader() {
const { addFile, setFileUploadProgress, setFileUploadStatus } = useAIChatUpload();
const handleDrop = async (files: File[]) => {
for (const file of files) {
// Add file and get its ID
const fileId = await addFile(undefined, file);
// File is automatically uploaded and processed
console.log('File uploaded:', fileId);
}
};
const handleUrlUpload = async (url: string, thumbnailBase64?: string) => {
// Add file from URL (file will be downloaded and processed)
const fileId = await addFile(thumbnailBase64);
await setFileUploadStatus(fileId, 'completed', url);
};
return (
<div onDrop={handleDrop}>
Drop files here
</div>
);
}Returns:
addFile- Add a file to the upload queue:(thumbnailBase64?: string, file?: File) => Promise<string>- Returns the unique
fileIdfor the uploaded file - If
fileis provided, it will be automatically uploaded and processed - If only
thumbnailBase64is provided, you must later callsetFileUploadStatuswith a remote URL
- Returns the unique
setFileUploadProgress- Update upload progress:(fileId: string, progress: number) => void- Progress should be between 0 and 100
setFileUploadStatus- Update upload status:(fileId: string, status: UploadedFileMeta["status"], remoteUrl?: string) => Promise<void>- If status is
"completed"andremoteUrlis provided, the file will be downloaded and processed - Status can be:
"pending","uploading","extracting-text","completed","failed", or"aborted"
- If status is
Status Flow:
"pending"- File added, waiting to start"uploading"- File is being uploaded to storage"extracting-text"- Text is being extracted (for supported file types)"completed"- File ready and attached to chat"failed"or"aborted"- Upload error or cancelled
Example: Custom Drag & Drop:
function DragDropUploader() {
const { addFile } = useAIChatUpload();
const [isDragging, setIsDragging] = useState(false);
const handleDrop = async (e: React.DragEvent) => {
e.preventDefault();
setIsDragging(false);
const files = Array.from(e.dataTransfer.files);
for (const file of files) {
await addFile(undefined, file);
}
};
return (
<div
onDragOver={(e) => { e.preventDefault(); setIsDragging(true); }}
onDragLeave={() => setIsDragging(false)}
onDrop={handleDrop}
className={isDragging ? 'border-blue-500' : 'border-gray-300'}
>
Drop files here to upload
</div>
);
}Example: Upload from URL:
function UrlUploader() {
const { addFile, setFileUploadStatus } = useAIChatUpload();
const handleUrlUpload = async (imageUrl: string) => {
// First, generate a thumbnail (optional)
const thumbnail = await generateThumbnail(imageUrl);
// Add the file entry
const fileId = await addFile(thumbnail);
// Set the remote URL - file will be downloaded and processed
await setFileUploadStatus(fileId, 'completed', imageUrl);
};
return (
<button onClick={() => handleUrlUpload('https://example.com/image.jpg')}>
Upload from URL
</button>
);
}Note: This hook must be used within
<AIChat.Provider>. The hook automatically handles file processing, text extraction, and thumbnail generation based on the provider's configuration.
Architecture
This package follows a headless component pattern:
- Render Props - Components accept functions as children for full rendering control
- React Slot Pattern - Some components accept single child elements for style injection
- Dependency Injection - Configuration passed via
dependenciesprop - Separation of Concerns - UI components separate from business logic
TypeScript Support
Full TypeScript support with exported types:
import type {
ChatDependencies,
ToolDescriptor,
UseAIChatReturn,
VoiceModeStatus,
// Server-side config types
ChatConfig,
ChatRequestBody,
RealtimeRequestBody,
SystemPromptConfig,
UnifiedSystemPromptConfig,
} from '@layouts.dev/ai-chat';
import { useAIChat, useAIChatUpload } from '@layouts.dev/ai-chat';
// Server-side chat configuration
interface ChatConfig {
model: any; // AI SDK model instance
systemPrompt?: string | ((body: ChatRequestBody | RealtimeRequestBody) => string | Promise<string>);
}
// Request body for text chat endpoint
interface ChatRequestBody {
messages: any[];
agentPrompt?: string;
clientTools?: string;
agentModel?: string;
[key: string]: any;
}
// Request body for realtime/voice endpoint
interface RealtimeRequestBody {
agentPrompt?: string;
agentVoice?: string;
agentVoiceModel?: string;
agentVoiceTranscriptionModel?: string;
[key: string]: any;
}
// ChatDependencies interface - all properties are optional
interface ChatDependencies {
// Database hooks (optional)
useChatById?: (chatId: string) => {
chat: { id: string; messages: UIMessage[] } | null;
getChat: () => Promise<{ id: string; messages: UIMessage[] } | null>;
updateChat: (data: { messages: UIMessage[] }) => Promise<void>;
loading: boolean;
};
createChat?: () => Promise<string>;
// AI Configuration (optional)
defaultPrompt?: string;
defaultVoice?: "alloy" | "ash" | "ballad" | "coral" | "echo" |
"sage" | "shimmer" | "verse" | "cedar" | "marin";
// Model Configuration (optional)
defaultAgentModel?: string; // OpenAI model ID for chat (default: 'gpt-4.1-mini')
defaultVoiceModeModel?: string; // OpenAI model ID for voice mode (default: 'gpt-realtime')
defaultVoiceModeTranscriptionModel?: string; // OpenAI transcription model (default: 'gpt-4o-transcribe')
// Optional
apiPath?: string;
clientTools?: ToolDescriptor[];
defaultActiveToolsNames?: string[];
// Custom fetch function for auth headers, etc.
fetch?: typeof globalThis.fetch;
}
// ToolDescriptor for client-side tools
interface ToolDescriptor {
name: string;
description: string;
parameters: JsonSchema; // Zod schema or JSON schema
execute: (
args: any,
context: {
router: any;
pathname: string;
openChatPanel?: () => void;
sendProgressMessage: (message: string) => void;
}
) => Promise<any>;
messages?: {
preparingInput?: string | (() => string);
inputAvailable?: string | ((input: Record<string, any>) => string);
outputAvailable?: string | ((input: Record<string, any>, output: Record<string, any>) => string);
outputError?: string | ((input: Record<string, any>, output: Record<string, any>, error: string) => string);
};
}
// Voice mode status
type VoiceModeStatus = "DISCONNECTED" | "CONNECTING" | "CONNECTED";
// useAIChat() return type
interface UseAIChatReturn {
chatId: string;
messages: UIMessage[];
loading: boolean;
append: (message: string | { role: "user" | "system"; parts: any[] }, role?: "user" | "system") => Promise<void>;
newChat: () => Promise<void>;
setChatId: (id: string) => void;
voiceModeStatus: VoiceModeStatus;
startVoiceMode: () => void;
stopVoiceMode: () => void;
interrupt: () => void;
mute: (muted: boolean) => void;
pushToTalkStart: () => void;
pushToTalkStop: () => void;
whoIsSpeaking: "user" | "assistant" | null;
currentTools: string[];
addTools: (owner: string, tools: string[]) => void;
removeTools: (owner: string) => void;
addContextMessage: (message: string) => void;
removeContextMessage: (message: string) => void;
setVoiceModeModel: (model: string) => void;
setVoiceModeTranscriptionModel: (model: string) => void;
setVoice: (voice: string) => void;
setAgentModel: (model: string) => void;
fftData: number[];
assistantFFTData: number[];
}
// Message types
type MessageType = "user" | "assistant" | "toolcall";
// Speaker detection
type Speaker = "user" | "assistant" | null;
// Uploaded file metadata
interface UploadedFileMeta {
fileId: string;
thumbnailBase64?: string;
status: "pending" | "uploading" | "extracting-text" | "completed" | "failed" | "aborted";
progress: number;
file: File;
data: string;
url: string;
}
// Hook return types
// useAIChatUpload() returns:
interface AIChatUploadContextValue {
addFile: (thumbnailBase64?: string, file?: File) => Promise<string>;
setFileUploadProgress: (fileId: string, progress: number) => void;
setFileUploadStatus: (
fileId: string,
status: UploadedFileMeta["status"],
remoteUrl?: string
) => Promise<void>;
}Client-Side Tools
You can define custom tools that execute on the client side. These tools can perform any action you need:
- Call external APIs
- Interact with the UI (navigation, panels, modals)
- Access browser APIs or local storage
- Perform client-side computations
- Any custom business logic
Example:
import type { ToolDescriptor } from '@layouts.dev/ai-chat';
const saveOnMangoose: ToolDescriptor = {
name: "saveOnMangoose",
description: "Saves a string to the database (Mongoose simulation)",
parameters: {
type: "object",
properties: {
text: { type: "string" }
},
required: ["text"]
},
execute: async (args: { text: string }) => {
const text = typeof args?.text === "string"
? args.text
: String(args?.text || "");
if (!text || text.trim().length === 0) {
throw new Error("Invalid argument: text must be a non-empty string");
}
// Simulate saving and return a fake id
const fakeId = `mangoose_${Date.now()}_${Math.floor(Math.random() * 10000)}`;
return { id: fakeId, saved: text };
},
messages: {
preparingInput: "Preparing to save text to DB",
inputAvailable: (input) => `Saving text: ${input.text}`,
outputAvailable: (input, output) => `Saved with id: ${output.id}`
}
};
// Pass to dependencies
const dependencies = {
// ... other config
clientTools: [saveOnMangoose],
defaultActiveToolsNames: ["saveOnMangoose"], // Optional: activate by default
};Requirements
- Next.js 16.0+ (App Router)
- React 19.0+
- TypeScript 5.0+ (recommended)
License
UNLICENSED © Creative Robots
