Cloudflare Worker AI Agent

This package implements a Durable Object-based Agent architecture for building AI applications on Cloudflare Workers. It supports multiple AI providers through AI SDK integration.

Architecture

    USER                         AGENT GATEWAY                 AGENT
┌──────────┐     HTTP        ┌─────────────────┐      ┌───────────────────────┐
│          │ ──────────────> │     Worker      │      │    Durable Object     │
│ Browser  │                 │ ┌─────────────┐ │ ───> │  ┌─────────────────┐  │
│          │ <─ ─ ─ ─ ─ ─ ─  │ │   Router    │ │      │  │  Agent Router   │--│---> [STATE = D1/KV/etc.]
└──────────┘     Stream      │ │    (Auth)   │ │      │  │ Business Logic  │┐ |
                             │ └─────────────┘ │      │  └─────────────────┘| │
                             └─────────────────┘      │         │           | │
                                                      │         │           | │
                                                      │         ▼           | │
                                                      │  ┌─────────────┐    | │
                                                      │  │   Services  │    | │
                                                      │  └─────┬───────┘    | │
                                                      └───────────────────────┘
                                                               │            │
                                                               ▼            ▼
                                                   ┌────────────────┐ ┌────────────────┐
                                                   │   3rd Party    │ │   AI Provider  │
                                                   │  MCP Services  │ │OpenAI/Anthropic│
                                                   └────────────────┘ └────────────────┘

Features

• AI SDK Support: Integrate with AI providers via the Vercel AI SDK
• Services: Routes for 3rd party business logic via webhooks
• Middleware: Dynamic tool injection, parameter modification, and response transformation
• Sessions: Simple session generation and management
• Auth: [Coming Soon] Authentication examples
• Events: [Coming Soon] System event handling for agents
• Cloudflare Agent: [Coming Soon] Native integration with Cloudflare Agent platform

Installation

npm install @nullshot/agent

Packages

The framework consists of the following packages:

• @nullshot/agent - Core agent framework
• @nullshot/agent/aisdk - AI SDK integration layer
• @nullshot/agent/services - Services for extending agent functionality

Basic Usage

import { XavaAgent, AgentEnv } from '@nullshot/agent';
import { AiSdkAgent } from '@nullshot/agent/aisdk';
import { ToolboxService } from '@nullshot/agent/services';
import { createOpenAI } from '@ai-sdk/openai';

// Define your environment type
type MyEnv = Env & AgentEnv;

// Create your agent class
export class MyAgent extends AiSdkAgent<MyEnv> {
	constructor(state: DurableObjectState, env: MyEnv) {
		// Initialize the AI model
		const openai = createOpenAI({
			apiKey: env.OPENAI_API_KEY,
		});
		const model = openai('gpt-4');

		// Initialize with services
		super(state, env, model, [new ToolboxService(env)]);
	}

	async processMessage(sessionId: string, messages: AIUISDKMessage): Promise<Response> {
		const result = await this.streamText(sessionId, {
			model: this.model,
			system: 'You are a helpful assistant.',
			messages: messages.messages,
			maxSteps: 10,
		});

		return result.toDataStreamResponse();
	}
}

// Worker handler
export default {
	async fetch(request, env, ctx) {
		// Apply router
		applyPermissionlessAgentSessionRouter(app);
		return app.fetch(request, env, ctx);
	},
};

Services

Services extend agent capabilities by providing specific functionality. They allow you to:

1. Expose APIs for 3rd party services (admin, webhooks, etc.)
2. Inject context for LLMs (tools, prompts, current_time, etc.)
3. Modify LLM responses (prompt retries, quality control, reactive events)
4. [Coming Soon] Fire dynamic events from 3rd party systems

Built-in Services

• Toolbox: Leverages mcp.json to manage tool injection to AI agents
• Time Context: [Coming Soon] Provides time-related context to agents

Creating a Custom Service

Services implement the Service interface or extend it with additional capabilities:

import { Service, AgentEnv } from '@nullshot/agent';
import { Hono } from 'hono';

// Basic service
export class MyService implements Service {
	name = '@my-org/agent/my-service';

	async initialize(): Promise<void> {
		// Initialize service resources
		console.log('Initializing my service');
	}
}

// External service with HTTP routes
export class MyExternalService implements ExternalService {
	name = '@my-org/agent/external-service';

	registerRoutes<E extends AgentEnv>(app: Hono<{ Bindings: E }>): void {
		app.get('/my-service/status', (c) => {
			return c.json({ status: 'ok' });
		});
	}
}

// Middleware service for AI interactions
export class MyMiddlewareService implements MiddlewareService {
	name = '@my-org/agent/middleware-service';
	middlewareVersion = '1.0.0';

	transformStreamTextTools(tools) {
		// Add or modify tools
		return {
			...tools,
			myTool: {
				description: 'My custom tool',
				execute: async (params) => {
					// Tool implementation
					return { result: 'success' };
				},
			},
		};
	}

	transformParams(params) {
		// Modify parameters for AI requests
		return {
			...params,
			temperature: 0.7,
		};
	}
}

Registering Services with an Agent

Register services when creating an agent instance:

constructor(state: DurableObjectState, env: MyEnv) {
  const services = [
    new ToolboxService(env),
    new MyCustomService(),
    new MyMiddlewareService()
  ];

  super(state, env, model, services);
}

Routers

Routers act as an API gateway for agents, handling CORS, routing logic, and authentication. They will likely be renamed to "gateways" in a future release.

Built-in Routers

• applyPermissionlessAgentSessionRouter(): Creates new sessions automatically or reuses sessions based on the /agent/chat/:sessionId endpoint.

import { Hono } from 'hono';
import { AgentEnv, applyPermissionlessAgentSessionRouter } from '@nullshot/agent';

// Create Hono app
const app = new Hono<{ Bindings: MyEnv }>();
// Apply routers / custom config
applyPermissionlessAgentSessionRouter(app);
// Export worker handler
export default {
	fetch: app.fetch,
};

Agent Environment

The AgentEnv interface provides default Durable Object naming and toolbox service configuration:

interface AgentEnv {
	AgentDurableObject: DurableObjectNamespace;
	TOOLBOX_SERVICE_MCP_SERVERS?: string;
}

Custom Environments

Implement custom environments by extending AgentEnv:

// Define a custom environment
type MyEnv = {
	OPENAI_API_KEY: string;
	MY_CUSTOM_BINDING: string;
} & AgentEnv;

// Use it in your agent
export class MyAgent extends AiSdkAgent<MyEnv> {
	// ...
}

Tools Management with MCP.json

The framework includes a CLI tool for managing Model Context Protocol (MCP) configurations.

MCP.json Structure

{
	"mcpServers": {
		"todo-list": {
			"url": "http://localhost:8788/sse"
		},
		"github": {
			"command": "npx mcp-github",
			"args": ["--port", "9000"],
			"env": {
				"GITHUB_TOKEN": "${GITHUB_TOKEN}"
			}
		}
	}
}

Tools Registry CLI

The tools-registry-cli processes mcp.json files and updates environment variables:

# Install globally
npm install -g @nullshot/agent

# Process mcp.json in current directory and update .dev.vars
npx tools-registry-cli

# Specify custom input file
npx tools-registry-cli ./my-config.json

# Output to custom file
npx tools-registry-cli --file .env.tools

# Output as JSON instead of base64
npx tools-registry-cli --format json

# Output to stdout
npx tools-registry-cli --stdout

Community and Support

Join our community on Discord to discuss features and contributions for this package in the #agent channel.