Feather Agent

Production-ready LLM agent framework with enterprise-grade reliability, multi-provider support, and intelligent orchestration capabilities.

Feather Agent is like Kubernetes for AI - it provides a unified, production-ready platform for managing multiple LLM providers with automatic failover, intelligent routing, cost optimization, and comprehensive observability.

✨ Why Feather?

🎯 Multi-Provider Strategy

• Avoid vendor lock-in with unified provider abstraction
• Intelligent failover between OpenAI, Anthropic, and custom providers
• Cost optimization through smart provider selection
• Zero downtime during provider outages

🛡️ Enterprise Reliability

• Circuit breakers prevent cascade failures
• Exponential backoff with jitter for intelligent retries
• Rate limiting with token bucket algorithm
• Automatic error classification (retryable vs permanent)

🤖 Intelligent Agents

• JSON-based planning for structured decision making
• Tool integration with caching and validation
• Memory management with PostgreSQL, Redis, or in-memory backends
• Context building with token-aware summarization

📊 Production Features

• Real-time streaming with proper abort handling
• Cost tracking per request and aggregate
• OpenTelemetry integration for distributed tracing
• Middleware system for logging, monitoring, and PII redaction

🚀 Quick Start

Installation

npm install feather-agent
# or
pnpm add feather-agent
# or
yarn add feather-agent

Requires Node.js >= 18 (uses native fetch)

Basic Usage

import { Feather, openai, anthropic } from "feather-agent";

// Initialize with multiple providers
const feather = new Feather({
  providers: {
    openai: openai({ apiKey: process.env.OPENAI_API_KEY! }),
    anthropic: anthropic({ apiKey: process.env.ANTHROPIC_API_KEY! })
  },
  limits: {
    "openai:gpt-4": { rps: 5, burst: 10 },
    "anthropic:claude-3-5-haiku": { rps: 3, burst: 5 }
  }
});

// Simple chat with automatic failover
const response = await feather.chat({
  provider: "openai",
  model: "gpt-4",
  messages: [
    { role: "user", content: "Explain quantum computing in simple terms." }
  ]
});

console.log(response.content);
console.log(`Cost: $${response.costUSD}`);

Advanced Patterns

// Fallback chain - try providers in sequence
const fallbackChain = feather.fallback([
  { provider: "openai", model: "gpt-4" },
  { provider: "anthropic", model: "claude-3-5-haiku" },
  { provider: "openai", model: "gpt-3.5-turbo" }
]);

// Race pattern - fastest response wins
const raceChain = feather.race([
  { provider: "openai", model: "gpt-4" },
  { provider: "anthropic", model: "claude-3-5-haiku" }
]);

// Batch processing with concurrency control
const results = await feather.map(
  ["task1", "task2", "task3"],
  async (task) => {
    const response = await feather.chat({
      provider: "openai",
      model: "gpt-3.5-turbo",
      messages: [{ role: "user", content: task }]
    });
    return response.content;
  },
  { concurrency: 2 }
);

🤖 Intelligent Agents

Feather includes a complete agent framework for building sophisticated AI applications:

import { Agent, InMemoryMemoryManager, createJsonPlanner, createCalcTool } from "feather-agent";

// Create a planner that makes structured decisions
const planner = createJsonPlanner({
  callModel: async ({ messages }) => {
    const response = await feather.chat({
      provider: "openai",
      model: "gpt-4",
      messages,
      temperature: 0.1
    });
    return response.content;
  },
  tools: [
    { name: "calc", description: "Evaluate arithmetic expressions" },
    { name: "web_search", description: "Search the web for information" }
  ]
});

// Create an agent with memory and tools
const agent = new Agent({
  id: "math-tutor",
  planner,
  memory: new InMemoryMemoryManager({ maxTurns: 100 }),
  tools: [createCalcTool()]
});

// Run conversations with automatic planning and tool execution
const result = await agent.run({
  sessionId: "student-123",
  input: { role: "user", content: "What is (15 + 25) * 2?" }
});

if (result.status === "completed") {
  console.log(result.output.content);
}

📖 Documentation

Core Concepts

• Quick Start Guide - Get up and running in minutes
• Memory Management - PostgreSQL, Redis, and in-memory backends
• Observability - Monitoring, tracing, and metrics
• Policies & Quotas - Guardrails and rate limiting
• Prompt Caching - Reduce costs with intelligent caching

Advanced Topics

• System Overview - Architecture and design decisions
• Troubleshooting - Common issues and solutions
• API Reference - Complete TypeScript definitions

🎯 Use Cases

Multi-Provider Applications

// A/B testing different models
const raceChain = feather.race([
  { provider: "openai", model: "gpt-4" },
  { provider: "anthropic", model: "claude-3-5-haiku" }
]);

// Cost-optimized routing
const fallbackChain = feather.fallback([
  { provider: "openai", model: "gpt-3.5-turbo" }, // Cheapest first
  { provider: "openai", model: "gpt-4" },         // Fallback to premium
  { provider: "anthropic", model: "claude-3-5-haiku" } // Cross-vendor fallback
]);

Production Chat Applications

class ChatService {
  private feather: Feather;
  
  constructor() {
    this.feather = new Feather({
      providers: {
        primary: openai({ apiKey: process.env.OPENAI_API_KEY! }),
        backup: anthropic({ apiKey: process.env.ANTHROPIC_API_KEY! })
      },
      limits: {
        "openai:gpt-4": { rps: 100, burst: 200 },
        "anthropic:claude-3-5-haiku": { rps: 50, burst: 100 }
      },
      retry: { maxAttempts: 3, baseMs: 1000, maxMs: 5000 },
      middleware: [
        this.loggingMiddleware,
        this.costTrackingMiddleware,
        this.piiRedactionMiddleware
      ]
    });
  }
  
  async chat(messages: Message[]) {
    const fallbackChain = this.feather.fallback([
      { provider: "primary", model: "gpt-4" },
      { provider: "backup", model: "claude-3-5-haiku" }
    ]);
    
    return await fallbackChain.chat({ messages });
  }
}

Agent Workflows

// Sequential agent chain
const research = await feather.chat({
  provider: "openai",
  model: "gpt-4",
  messages: [{ role: "user", content: "Research quantum computing" }]
});

const article = await feather.chat({
  provider: "anthropic", 
  model: "claude-3-5-haiku",
  messages: [{ role: "user", content: `Write article: ${research.content}` }]
});

// Parallel agent processing
const [analysis, strategy, critique] = await Promise.all([
  feather.chat({ provider: "openai", model: "gpt-4", messages: analysisPrompt }),
  feather.chat({ provider: "anthropic", model: "claude-3-5-haiku", messages: strategyPrompt }),
  feather.chat({ provider: "openai", model: "gpt-3.5-turbo", messages: critiquePrompt })
]);

🛠️ Configuration

Provider Configuration

const feather = new Feather({
  providers: {
    openai: openai({ 
      apiKey: process.env.OPENAI_API_KEY!,
      baseUrl: "https://api.openai.com/v1", // Optional custom endpoint
      pricing: { inputPer1K: 0.005, outputPer1K: 0.015 }
    }),
    anthropic: anthropic({ 
      apiKey: process.env.ANTHROPIC_API_KEY!,
      pricing: { inputPer1K: 0.008, outputPer1K: 0.024 }
    })
  }
});

Rate Limiting & Reliability

const feather = new Feather({
  providers: { /* ... */ },
  limits: {
    "openai:gpt-4": { rps: 10, burst: 20 },      // 10 req/sec, burst to 20
    "openai:gpt-3.5-turbo": { rps: 50, burst: 100 },
    "anthropic:claude-3-5-haiku": { rps: 5, burst: 10 }
  },
  retry: {
    maxAttempts: 3,        // Try up to 3 times
    baseMs: 1000,         // Start with 1 second delay
    maxMs: 10000,         // Max 10 second delay
    jitter: "full"        // Add randomness to prevent thundering herd
  },
  timeoutMs: 30000         // 30 second timeout
});

Configuration File

Create feather.config.json for declarative configuration:

{
  "policy": "cheapest",
  "providers": {
    "openai": {
      "apiKeyEnv": "OPENAI_API_KEY",
      "models": [
        {
          "name": "gpt-4",
          "aliases": ["smart", "expensive"],
          "inputPer1K": 0.03,
          "outputPer1K": 0.06
        },
        {
          "name": "gpt-3.5-turbo", 
          "aliases": ["fast", "cheap"],
          "inputPer1K": 0.001,
          "outputPer1K": 0.002
        }
      ]
    },
    "anthropic": {
      "apiKeyEnv": "ANTHROPIC_API_KEY",
      "models": [
        {
          "name": "claude-3-5-haiku",
          "aliases": ["fast", "balanced"],
          "inputPer1K": 0.008,
          "outputPer1K": 0.024
        }
      ]
    }
  }
}

Use semantic model names:

import { buildRegistry } from "feather-agent";
import config from "./feather.config.json" assert { type: "json" };

const registry = buildRegistry(config);
const feather = new Feather({ registry });

// Use semantic aliases - orchestrator picks best option
const response = await feather.chat({
  model: "fast",  // Will pick cheapest "fast" model
  messages: [{ role: "user", content: "Hello!" }]
});

🖥️ CLI Usage

Install globally or use with npx:

# Install globally
npm install -g feather-agent

# Use with npx
npx feather chat -m gpt-4 -q "What is machine learning?"

# With specific provider
npx feather chat -p openai -m gpt-4 -q "Hello world"

# With config file
npx feather chat -c ./my-config.json -m fast -q "Explain AI"

CLI Options

feather chat [options]

Options:
  -p, --provider <provider>  Provider name (optional with config)
  -m, --model <model>        Model name or alias
  -q, --query <query>        User message
  -c, --config <file>        Config file path (default: feather.config.json)
  -h, --help                 Show help

🔧 Adding Custom Providers

Create providers for any LLM service:

import { ChatProvider } from "feather-agent";

export function customProvider(config: { apiKey: string }): ChatProvider {
  return {
    id: "custom",
    
    async chat(req: ChatRequest): Promise<ChatResponse> {
      const response = await fetch("https://api.custom-llm.com/chat", {
        method: "POST",
        headers: {
          "Authorization": `Bearer ${config.apiKey}`,
          "Content-Type": "application/json"
        },
        body: JSON.stringify({
          model: req.model,
          messages: req.messages,
          temperature: req.temperature,
          max_tokens: req.maxTokens
        })
      });
      
      if (!response.ok) {
        throw new Error(`Custom API error: ${response.status}`);
      }
      
      const data = await response.json();
      return {
        content: data.choices[0].message.content,
        tokens: {
          input: data.usage.prompt_tokens,
          output: data.usage.completion_tokens
        },
        costUSD: calculateCost(data.usage),
        raw: data
      };
    },
    
    async *stream(req: ChatRequest): AsyncIterable<ChatDelta> {
      // Implement streaming if supported
      const response = await fetch("https://api.custom-llm.com/chat/stream", {
        method: "POST",
        headers: {
          "Authorization": `Bearer ${config.apiKey}`,
          "Content-Type": "application/json"
        },
        body: JSON.stringify({
          model: req.model,
          messages: req.messages,
          stream: true
        })
      });
      
      const reader = response.body?.getReader();
      const decoder = new TextDecoder();
      
      while (true) {
        const { value, done } = await reader!.read();
        if (done) break;
        
        const chunk = decoder.decode(value, { stream: true });
        const lines = chunk.split('\n');
        
        for (const line of lines) {
          if (line.startsWith('data: ')) {
            const data = JSON.parse(line.slice(6));
            if (data.choices?.[0]?.delta?.content) {
              yield { content: data.choices[0].delta.content };
            }
          }
        }
      }
    },
    
    price: {
      inputPer1K: 0.001,   // $0.001 per 1K input tokens
      outputPer1K: 0.002  // $0.002 per 1K output tokens
    }
  };
}

// Use your custom provider
const feather = new Feather({
  providers: {
    custom: customProvider({ apiKey: "your-api-key" })
  }
});

🧪 Testing

Unit Tests

import { describe, it, expect, vi } from "vitest";
import { Feather } from "feather-agent";

describe("Feather Orchestrator", () => {
  it("should handle fallback correctly", async () => {
    const mockProvider = {
      id: "mock",
      async chat() {
        throw new Error("Provider failed");
      }
    };
    
    const feather = new Feather({
      providers: {
        fail: mockProvider,
        success: {
          id: "success",
          async chat() {
            return { content: "Success!" };
          }
        }
      }
    });
    
    const fallbackChain = feather.fallback([
      { provider: "fail", model: "test" },
      { provider: "success", model: "test" }
    ]);
    
    const response = await fallbackChain.chat({
      messages: [{ role: "user", content: "test" }]
    });
    
    expect(response.content).toBe("Success!");
  });
});

Integration Tests

import { Feather, openai } from "feather-agent";

describe("Integration Tests", () => {
  it("should work with real OpenAI API", async () => {
    const feather = new Feather({
      providers: {
        openai: openai({ apiKey: process.env.OPENAI_API_KEY! })
      }
    });
    
    const response = await feather.chat({
      provider: "openai",
      model: "gpt-3.5-turbo",
      messages: [{ role: "user", content: "Say hello" }]
    });
    
    expect(response.content).toContain("hello");
    expect(response.costUSD).toBeGreaterThan(0);
  });
});

📊 Monitoring & Observability

Cost Tracking

class CostTracker {
  private costs: Map<string, number> = new Map();
  
  async trackCost(provider: string, cost: number) {
    const current = this.costs.get(provider) || 0;
    this.costs.set(provider, current + cost);
    
    // Send to your metrics system
    await this.sendToMetrics({
      provider,
      cost,
      total: current + cost,
      timestamp: new Date()
    });
  }
  
  getTotalCost(): number {
    return Array.from(this.costs.values()).reduce((sum, cost) => sum + cost, 0);
  }
}

const costTracker = new CostTracker();

const feather = new Feather({
  providers: { /* ... */ },
  middleware: [
    async (ctx, next) => {
      await next();
      if (ctx.response?.costUSD) {
        await costTracker.trackCost(ctx.provider, ctx.response.costUSD);
      }
    }
  ]
});

OpenTelemetry Integration

import { trace, context } from '@opentelemetry/api';

export function withTelemetry<T>(
  operation: string,
  fn: () => Promise<T>
): Promise<T> {
  const tracer = trace.getTracer('feather');
  const span = tracer.startSpan(operation);
  
  return context.with(trace.setSpan(context.active(), span), async () => {
    try {
      const result = await fn();
      span.setStatus({ code: SpanStatusCode.OK });
      return result;
    } catch (error) {
      span.setStatus({ code: SpanStatusCode.ERROR, message: error.message });
      span.recordException(error);
      throw error;
    } finally {
      span.end();
    }
  });
}

🔒 Security Best Practices

API Key Management

// ✅ Good: Use environment variables
const feather = new Feather({
  providers: {
    openai: openai({ apiKey: process.env.OPENAI_API_KEY! })
  }
});

// ❌ Bad: Hardcode API keys
const feather = new Feather({
  providers: {
    openai: openai({ apiKey: "sk-1234567890abcdef" })
  }
});

PII Redaction

const feather = new Feather({
  providers: { /* ... */ },
  middleware: [
    async (ctx, next) => {
      // Redact sensitive information
      ctx.request.messages = ctx.request.messages.map(msg => ({
        ...msg,
        content: msg.content
          .replace(/\b\d{4}-\d{4}-\d{4}-\d{4}\b/g, '[CARD]')  // Credit cards
          .replace(/\b\d{3}-\d{2}-\d{4}\b/g, '[SSN]')         // SSNs
          .replace(/\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Z|a-z]{2,}\b/g, '[EMAIL]')  // Emails
      }));
      await next();
    }
  ]
});

Rate Limiting

// Prevent abuse with strict rate limits
const feather = new Feather({
  providers: { /* ... */ },
  limits: {
    "openai:gpt-4": { rps: 1, burst: 2 },  // Very conservative limits
    "anthropic:claude-3-5-haiku": { rps: 2, burst: 3 }
  }
});

🚀 Deployment

Docker

FROM node:18-alpine

WORKDIR /app

COPY package*.json ./
RUN npm ci --only=production

COPY dist/ ./dist/

EXPOSE 3000

CMD ["node", "dist/index.js"]

Environment Variables

# Production environment
OPENAI_API_KEY=sk-proj-...
ANTHROPIC_API_KEY=sk-ant-...

# Optional: Custom configurations
FEATHER_CONFIG_PATH=/app/config/feather.config.json
FEATHER_LOG_LEVEL=info
FEATHER_RATE_LIMIT_ENABLED=true

Kubernetes ConfigMap

apiVersion: v1
kind: ConfigMap
metadata:
  name: feather-config
data:
  feather.config.json: |
    {
      "policy": "cheapest",
      "providers": {
        "openai": {
          "apiKeyEnv": "OPENAI_API_KEY",
          "models": [
            {
              "name": "gpt-4",
              "aliases": ["smart"],
              "inputPer1K": 0.03,
              "outputPer1K": 0.06
            }
          ]
        }
      }
    }

📈 Performance

Benchmarks

• Latency: < 50ms overhead for orchestration
• Throughput: 1000+ requests/second with proper rate limiting
• Memory: < 10MB base footprint
• Bundle Size: < 100KB gzipped

Optimization Tips

• Use prompt caching to reduce redundant API calls
• Enable tool caching for repeated computations
• Configure appropriate rate limits per provider
• Use semantic model aliases for cost optimization

🤝 Contributing

We welcome contributions! Please see our Contributing Guide for details.

Quick Start for Contributors

1. Fork the repository
2. Create a feature branch: git checkout -b feature/amazing-feature
3. Make your changes and add tests
4. Run tests: npm test
5. Commit: git commit -m 'Add amazing feature'
6. Push: git push origin feature/amazing-feature
7. Open a Pull Request

Adding New Providers

1. Create a new file in src/providers/
2. Implement the ChatProvider interface
3. Add tests in tests/providers/
4. Update documentation with usage examples

📄 License

Licensed under the Apache License, Version 2.0. See LICENSE for details.

🙏 Acknowledgments

• Built with ❤️ for the developer community
• Inspired by the need for reliable LLM orchestration
• Thanks to all contributors and users

Need help?

• 📖 Check the documentation
• 🐛 Report issues on GitHub
• 💬 Join our Discord community
• 📧 Contact us at [email protected]

Star us on GitHub ⭐ to show your support!