cfai - CopilotKit + Cloudflare AI

The easiest way to use CopilotKit with Cloudflare Workers AI. Get OpenAI's GPT models with 93% cost savings.

⚡ Performance Benefits

• 68% faster responses (110ms vs 350ms)
• 93% cheaper than OpenAI ($11 vs $150 per million tokens)
• 200+ edge locations worldwide
• Zero cold starts with Cloudflare Workers AI

🚀 Quick Start

1. Installation

npm install cfai
# or
yarn add cfai
# or
pnpm add cfai

2. Setup API Route

Create app/api/copilotkit/route.ts:

Method 1: Quick Setup (Recommended)

import cfai from "cfai";

export const runtime = "edge"; // Required for Cloudflare Workers AI

export const POST = cfai({
  apiToken: process.env.CLOUDFLARE_API_TOKEN!,
  accountId: process.env.CLOUDFLARE_ACCOUNT_ID!,
  model: "@cf/meta/llama-3.1-8b-instruct", // Optional
});

Method 2: Using Adapter Pattern (Like Azure OpenAI)

import {
  CopilotRuntime,
  OpenAIAdapter,
  copilotRuntimeNextJSAppRouterEndpoint,
} from "@copilotkit/runtime";
import { createCloudflareOpenAI } from "cfai";
import { NextRequest } from "next/server";

export const runtime = "edge"; // Required for Cloudflare Workers AI

const { openai, model } = createCloudflareOpenAI({
  apiToken: process.env.CLOUDFLARE_API_TOKEN!,
  accountId: process.env.CLOUDFLARE_ACCOUNT_ID!,
  // Optional: Use AI Gateway for caching
  useGateway: true,
  gatewayId: process.env.CLOUDFLARE_GATEWAY_ID!,
});

const runtime = new CopilotRuntime();
const serviceAdapter = new OpenAIAdapter({ openai, model });

export const POST = async (req: NextRequest) => {
  const { handleRequest } = copilotRuntimeNextJSAppRouterEndpoint({
    runtime,
    serviceAdapter,
    endpoint: "/api/copilotkit",
  });
  return handleRequest(req);
};

Method 3: Legacy Handler (For older CopilotKit versions)

import { cfaiHandler } from "cfai";

export const runtime = "edge"; // Required for Cloudflare Workers AI

export const POST = cfaiHandler({
  apiToken: process.env.CLOUDFLARE_API_TOKEN!,
  accountId: process.env.CLOUDFLARE_ACCOUNT_ID!,
  model: "@cf/meta/llama-3.1-8b-instruct", // Optional
  temperature: 0.7, // Optional
  debug: process.env.NODE_ENV === "development", // Optional
});

3. Environment Variables

Add to your .env.local:

CLOUDFLARE_API_TOKEN=your_api_token_here
CLOUDFLARE_ACCOUNT_ID=your_account_id_here
# Optional: For AI Gateway
CLOUDFLARE_GATEWAY_ID=your_gateway_id_here

Get these from your Cloudflare dashboard:

• API Token: Create at https://dash.cloudflare.com/profile/api-tokens
• Account ID: Find in Workers & Pages > Overview
• Gateway ID: Create at AI Gateway > Create Gateway

4. Use in Your App

import { CopilotKit } from "@copilotkit/react-core";
import { CopilotChat } from "@copilotkit/react-ui";
import "@copilotkit/react-ui/styles.css";

export default function MyApp() {
  return (
    <CopilotKit runtimeUrl="/api/copilotkit">
      <CopilotChat
        instructions="You are running on Cloudflare Workers AI at the edge!"
        labels={{
          title: "AI Assistant",
          initial:
            "Hello! I'm powered by Cloudflare Workers AI. How can I help?",
        }}
      />
    </CopilotKit>
  );
}

🔧 Configuration

interface CfaiConfig {
  /** Cloudflare API token */
  apiToken: string;
  /** Cloudflare account ID */
  accountId: string;
  /** AI model to use (defaults to Llama 3.1 8B) */
  model?: string;
  /** Use Cloudflare AI Gateway for caching and observability */
  useGateway?: boolean;
  /** AI Gateway ID (required if useGateway is true) */
  gatewayId?: string;
  /** Temperature for AI responses (0-1) */
  temperature?: number;
  /** Maximum tokens in response */
  maxTokens?: number;
  /** Enable debug logging */
  debug?: boolean;
}

🌐 Using AI Gateway (Recommended)

For production apps, use Cloudflare AI Gateway for caching and observability:

import { createCloudflareOpenAI } from "cfai";

const { openai, model } = createCloudflareOpenAI({
  apiToken: process.env.CLOUDFLARE_API_TOKEN!,
  accountId: process.env.CLOUDFLARE_ACCOUNT_ID!,
  useGateway: true,
  gatewayId: process.env.CLOUDFLARE_GATEWAY_ID!,
  model: "@cf/meta/llama-3.3-70b-instruct", // Use more capable model with gateway
});

// Use with CopilotRuntime just like Azure OpenAI

🤖 Supported Models

Automatic Model Mapping

The package intelligently maps OpenAI model names to Cloudflare models. By default, it uses actual OpenAI models running on Cloudflare!

| OpenAI Model | Default (OpenAI on CF) | Alternative (Llama/Other) | | --------------- | ------------------------- | ---------------------------------------------- | | gpt-3.5-turbo | @cf/openai/gpt-oss-20b | @cf/meta/llama-3.1-8b-instruct-fast | | gpt-4 | @cf/openai/gpt-oss-120b | @cf/meta/llama-3.3-70b-instruct-fp8-fast | | gpt-4-turbo | @cf/openai/gpt-oss-120b | @cf/meta/llama-3.3-70b-instruct-fp8-fast | | gpt-4o | @cf/openai/gpt-oss-120b | @cf/mistralai/mistral-small-3.1-24b-instruct | | gpt-4o-mini | @cf/openai/gpt-oss-20b | @cf/meta/llama-3.1-8b-instruct-fast |

Choose your preference:

// Use OpenAI models (default)
cfai({
  apiToken: process.env.CLOUDFLARE_API_TOKEN!,
  accountId: process.env.CLOUDFLARE_ACCOUNT_ID!,
  modelPreference: "openai", // or omit for default
});

// Use alternative models (Llama, Mistral, etc.)
cfai({
  apiToken: process.env.CLOUDFLARE_API_TOKEN!,
  accountId: process.env.CLOUDFLARE_ACCOUNT_ID!,
  modelPreference: "alternative",
});

Model Categories

Choose models by capability:

// Fast models (low latency)
model: "@cf/meta/llama-3.1-8b-instruct-fast";
model: "@cf/openai/gpt-oss-20b";

// Powerful models (best quality)
model: "@cf/openai/gpt-oss-120b";
model: "@cf/meta/llama-3.3-70b-instruct-fp8-fast";

// Code generation
model: "@cf/qwen/qwen2.5-coder-32b-instruct";

// Reasoning models
model: "@cf/deepseek/deepseek-r1-distill-qwen-32b";

// Multimodal (text + images)
model: "@cf/meta/llama-4-scout-17b-16e-instruct";

🔒 Environment Variables

Create a .env.local file:

CLOUDFLARE_API_TOKEN=your_cloudflare_api_token
CLOUDFLARE_ACCOUNT_ID=your_cloudflare_account_id
CLOUDFLARE_GATEWAY_ID=your_gateway_id  # Optional, for AI Gateway

🚨 Troubleshooting

"Module not found: Can't resolve 'cfai'"

• Make sure you've installed the package: npm install cfai
• Try clearing your Next.js cache: rm -rf .next && npm run dev

"Edge runtime error" or "Cannot read properties"

• IMPORTANT: You must add export const runtime = "edge" to your API route
• This is required for Cloudflare Workers AI to function

"Invalid API token" or "Authentication failed"

• Verify your Cloudflare API token has Workers AI permissions
• Check your account ID is correct (find it in Cloudflare dashboard)
• Make sure environment variables are loaded (restart your dev server)

"Model not found" error

• Use a valid model name from the supported models list
• Default model is @cf/meta/llama-3.1-8b-instruct-fast

📊 Cost Comparison

| Provider | Cost per 1M tokens | Typical Response Time | | ----------------------- | ------------------ | --------------------- | | OpenAI GPT-4o | $150 | 350ms | | Cloudflare Llama 3.1 8B | $11 | 110ms | | Savings | 93% cheaper | 68% faster |

🏗️ Supported CopilotKit Versions

This package supports all modern CopilotKit versions and request formats:

• ✅ Modern CopilotKit Runtime (1.9.x+) - Recommended
• ✅ Direct chat completions
• ✅ GraphQL mutations and queries
• ✅ REST API fallbacks
• ✅ Legacy handler support - Backward compatibility

🚨 Migration from OpenAI

Replace your existing CopilotKit API route:

// Before (OpenAI)
import { CopilotRuntime, OpenAIAdapter } from "@copilotkit/runtime";

const copilotKit = new CopilotRuntime();
export const POST = copilotKit.streamHttpServerResponse(
  openaiAdapter.chatCompletionRequest()
);

// After (Cloudflare)
import { cfaiHandler } from "cfai";

export const POST = cfaiHandler({
  apiToken: process.env.CLOUDFLARE_API_TOKEN!,
  accountId: process.env.CLOUDFLARE_ACCOUNT_ID!,
});

No frontend changes required! 🎉

🔧 Advanced Usage

Health Check Endpoint

import {
  cfai,
  cfaiHealthCheck,
  cfaiOptions,
} from "cfai";

const config = {
  apiToken: process.env.CLOUDFLARE_API_TOKEN!,
  accountId: process.env.CLOUDFLARE_ACCOUNT_ID!,
};

export const POST = cfai(config);
export const GET = cfaiHealthCheck(config);
export const OPTIONS = cfaiOptions();

Debug Mode

Enable debug logging:

export const POST = cfai({
  apiToken: process.env.CLOUDFLARE_API_TOKEN!,
  accountId: process.env.CLOUDFLARE_ACCOUNT_ID!,
  debug: true, // Enables request/response logging
});

🐛 Troubleshooting

Common Issues

1. "Account ID required" - Set CLOUDFLARE_ACCOUNT_ID environment variable
2. "API token required" - Set CLOUDFLARE_API_TOKEN environment variable
3. 403 Forbidden - Check your Cloudflare API token permissions
4. Model not found - Verify the model name is correct for Cloudflare

Getting Help

• Check the CopilotKit docs
• Review Cloudflare Workers AI docs
• Open an issue on GitHub

📜 License

MIT License - see LICENSE file for details.

🤝 Contributing

Contributions welcome! Please read our contributing guidelines.

Built with ❤️ by developers who believe AI should be fast and affordable.