Cloudflare Workers AI Provider for AI SDK

A type-safe ProviderV3 implementation for Cloudflare Workers AI that integrates with the Vercel AI SDK. This provider enables you to use Cloudflare Workers AI models directly through the AI SDK's standardized interface.

Features

• ✅ Type-safe: Full TypeScript support with model capability encoding
• ✅ V3 Specification: Implements Language Model Specification V3
• ✅ Language Models: Support for text generation with streaming
• ✅ Embedding Models: Support for text embeddings
• ✅ Tool Calling: Support for function/tool calling on compatible models
• ✅ No API Keys Required: Works directly with Cloudflare Workers AI binding
• ✅ Model Registry: Built-in registry of supported models with capabilities

Installation

pnpm add @davidbrama/cloudflare-workers-ai-provider
# or
npm install @davidbrama/cloudflare-workers-ai-provider
# or
yarn add @davidbrama/cloudflare-workers-ai-provider

Prerequisites

• A Cloudflare Workers project with AI binding configured
• @cloudflare/workers-types for TypeScript support
• AI SDK v6 (beta)

Setup

1. Configure Wrangler

Add the AI binding to your wrangler.toml:

[ai]
binding = "AI"

2. Install Dependencies

pnpm add @ai-sdk/provider ai @davidbrama/cloudflare-workers-ai-provider
pnpm add -D @cloudflare/workers-types

Usage

Basic Example

import { WorkersAiProvider } from "@davidbrama/cloudflare-workers-ai-provider";
import { generateText } from "ai";

export default {
  async fetch(request: Request, env: Env) {
    // Create the provider with the AI binding
    const provider = new WorkersAiProvider({ ai: env.AI });

    // Use with AI SDK
    const { text } = await generateText({
      model: provider.languageModel("@cf/meta/llama-3-8b-instruct"),
      prompt: "Explain quantum computing in simple terms",
    });

    return new Response(text);
  },
};

Streaming Example

import { WorkersAiProvider } from "@davidbrama/cloudflare-workers-ai-provider";
import { streamText } from "ai";

export default {
  async fetch(request: Request, env: Env) {
    const provider = new WorkersAiProvider({ ai: env.AI });

    const result = streamText({
      model: provider.languageModel("@cf/meta/llama-3-8b-instruct"),
      prompt: "Write a short story about a robot",
    });

    return result.toDataStreamResponse();
  },
};

Embeddings Example

import { WorkersAiProvider } from "@davidbrama/cloudflare-workers-ai-provider";
import { embed } from "ai";

export default {
  async fetch(request: Request, env: Env) {
    const provider = new WorkersAiProvider({ ai: env.AI });

    const { embeddings } = await embed({
      model: provider.textEmbeddingModel("@cf/baai/bge-base-en-v1.5"),
      values: ["Hello world", "AI is amazing"],
    });

    return new Response(JSON.stringify(embeddings));
  },
};

Tool Calling Example

import { WorkersAiProvider } from "@davidbrama/cloudflare-workers-ai-provider";
import { generateText } from "ai";

export default {
  async fetch(request: Request, env: Env) {
    const provider = new WorkersAiProvider({ ai: env.AI });

    const { text, toolCalls } = await generateText({
      model: provider.languageModel("@cf/meta/llama-3-8b-instruct"),
      prompt: "What's the weather in San Francisco?",
      tools: {
        getWeather: {
          description: "Get the current weather in a location",
          parameters: {
            type: "object",
            properties: {
              location: {
                type: "string",
                description: "The city and state, e.g. San Francisco, CA",
              },
            },
            required: ["location"],
          },
        },
      },
    });

    // Handle tool calls...
    return new Response(text);
  },
};

Supported Models

Language Models

| Model ID | Streaming | Tools | Multimodal | | -------------------------------------- | --------- | ----- | ---------- | | @cf/meta/llama-3-8b-instruct | ✅ | ✅ | ❌ | | @cf/meta/llama-3-70b-instruct | ✅ | ✅ | ❌ | | @cf/meta/llama-3.1-8b-instruct | ✅ | ✅ | ❌ | | @cf/meta/llama-3.1-70b-instruct | ✅ | ✅ | ❌ | | @cf/mistral/mistral-7b-instruct-v0.2 | ✅ | ❌ | ❌ | | @cf/mistral/mistral-7b-instruct-v0.3 | ✅ | ❌ | ❌ | | @cf/google/gemma-7b-it | ✅ | ❌ | ❌ | | @cf/meta/llama-2-7b-chat-fp16 | ✅ | ❌ | ❌ |

Embedding Models

| Model ID | Description | | -------------------------------------- | ------------------------ | | @cf/baai/bge-base-en-v1.5 | Base English embeddings | | @cf/baai/bge-large-en-v1.5 | Large English embeddings | | @cf/baai/bge-small-en-v1.5 | Small English embeddings | | @cf/huggingface/multilingual-e5-base | Multilingual embeddings | | @cf/google/embeddinggemma-300m | Google Gemma embeddings |

For the complete list of available models, see the Cloudflare Workers AI Models documentation.

API Reference

WorkersAiProvider

The main provider class that implements ProviderV3.

Constructor

new WorkersAiProvider(settings: WorkersAiProviderSettings)

Parameters:

• settings.ai: The Cloudflare Workers AI binding (from env.AI)

Methods

languageModel(modelId: string): WorkersAiLanguageModel

Returns a language model instance for text generation.

Parameters:

• modelId: The model ID (e.g., "@cf/meta/llama-3-8b-instruct")

Throws: NoSuchModelError if the model ID is invalid or not a language model

textEmbeddingModel(modelId: string): WorkersAiEmbeddingModel

Returns an embedding model instance for text embeddings.

Parameters:

• modelId: The model ID (e.g., "@cf/baai/bge-base-en-v1.5")

Throws: NoSuchModelError if the model ID is invalid or not an embedding model

imageModel(modelId: string): ImageModelV3

Not supported. Always throws NoSuchModelError.

Type Safety

The provider includes type-safe model IDs:

import type {
  WorkersAiLanguageModelId,
  WorkersAiEmbeddingModelId,
} from "workers_provider";

// Type-safe model IDs
const languageModelId: WorkersAiLanguageModelId =
  "@cf/meta/llama-3-8b-instruct";
const embeddingModelId: WorkersAiEmbeddingModelId = "@cf/baai/bge-base-en-v1.5";

Development

Build

pnpm build

Lint

pnpm lint

Project Structure

.
├── index.ts              # Main provider implementation
├── language-model.ts     # LanguageModelV3 implementation
├── embedding-model.ts    # EmbeddingModelV3 implementation
├── conversion.ts         # Format conversion utilities
├── errors.ts             # Error handling utilities
└── types.ts              # Type definitions and model registry

Limitations

• No Image Models: Cloudflare Workers AI does not support image generation models
• Streaming: Streaming implementation may need adjustment based on actual Cloudflare Workers AI API behavior
• Multimodal: Limited multimodal support (images in prompts) - depends on model capabilities

License

ISC

Contributing

Contributions are welcome! Please ensure all code passes linting and follows the existing code style.

Related Links

• Vercel AI SDK Documentation
• Cloudflare Workers AI Documentation
• AI SDK Provider Specification