ultra-claude-toolkit

Ultra-type-safe TypeScript toolkit for building AI applications with Anthropic Claude.

Quality Assured: Every commit runs lint, TypeScript checks, and full test suite automatically via pre-commit hooks.

Features

• 🚀 Ultra-type-safe: Perfect TypeScript inference with zero casting
• 🛠️ Tool Creation: Easy creation of validation and executable tools
• 🔄 Workflow Management: Single and multi-turn AI workflows
• 🔁 Smart Retry Logic: Uses Anthropic rate limit headers for optimal timing
• 📊 Production Metrics: Comprehensive observability and performance tracking
• 🛡️ Enhanced Error Handling: Rich error context with debugging information
• 📝 Zod Integration: Schema validation for inputs and outputs
• 🎯 Type Safety: Discriminated unions for tool results
• 🏗️ Clean Architecture: Modular, maintainable codebase structure

Installation

npm install ultra-claude-toolkit

Peer Dependencies

npm install @anthropic-ai/sdk zod

Quick Start

import { z } from 'zod'
import Anthropic from '@anthropic-ai/sdk'
import { 
  createValidationAnthropicTool, 
  createToolset,
  executeSingleToolWorkflow 
} from 'ultra-claude-toolkit'

// Create a validation tool
const mathTool = createValidationAnthropicTool({
  name: 'calculate',
  description: 'Return the result of a mathematical calculation',
  inputSchema: z.object({
    operation: z.enum(['add', 'subtract', 'multiply', 'divide']),
    a: z.number(),
    b: z.number(),
    result: z.number(),
    explanation: z.string().optional()
  })
})

// Create a toolset
const toolset = createToolset(mathTool)

// Use in a workflow
const client = new Anthropic({ apiKey: 'your-api-key' })

const result = await executeSingleToolWorkflow(client, {
  model: {
    model: 'claude-3-5-haiku-latest',
    max_tokens: 256,
    temperature: 0
  },
  system: 'You are a helpful math assistant.',
  messages: [{ role: 'user', content: 'What is 15 + 27?' }],
  toolset,
  expectedToolName: 'calculate'
})

// result is perfectly typed: { operation: 'add' | 'subtract' | ..., a: number, b: number, result: number }
console.log(`${result.a} ${result.operation} ${result.b} = ${result.result}`)

📊 Production Metrics & Observability

Track performance, success rates, and operational health in production:

import { 
  enableMetrics, 
  getMetricsCollector,
  executeSingleToolWorkflow 
} from 'ultra-claude-toolkit'

// Enable metrics collection (optional, zero overhead when disabled)
enableMetrics()

// Execute workflows - metrics automatically collected
const result = await executeSingleToolWorkflow(client, { /*...*/ })

// Access comprehensive metrics
const metrics = getMetricsCollector().getMetrics()

console.log(`Success rate: ${metrics.workflows.successRate * 100}%`)
console.log(`Average execution time: ${metrics.tools['my_tool'].averageExecutionTime}ms`)
console.log(`Rate limit hits: ${metrics.retries.rateLimitHits}`)
console.log(`Total workflows: ${metrics.workflows.singleTurnCount}`)

// Export metrics for monitoring systems
const exportedMetrics = getMetricsCollector().export()

What's Tracked:

• ⏱️ Tool execution times (with sliding window averages)
• 📈 Success/failure rates per tool and workflow type
• 🔄 Retry behavior (rate limits, server errors, recovery success)
• 🎯 Workflow performance metrics
• 🛡️ Error patterns and debugging context

API Reference

Tool Creation

createValidationAnthropicTool(params)

Creates a validation-only tool that parses and validates AI responses.

const tool = createValidationAnthropicTool({
  name: 'extract_data',
  description: 'Extract structured data from text',
  inputSchema: z.object({
    title: z.string(),
    summary: z.string(),
    tags: z.array(z.string())
  })
})

createExecutableAnthropicTool(params)

Creates an executable tool with a handler function.

const tool = createExecutableAnthropicTool({
  name: 'fetch_weather',
  description: 'Get current weather for a city',
  inputSchema: z.object({ city: z.string() }),
  outputSchema: z.object({ 
    temperature: z.number(), 
    condition: z.string() 
  }),
  handler: async ({ city }) => {
    // Your implementation
    return { temperature: 22, condition: 'sunny' }
  }
})

Toolsets

createToolset(...tools)

Groups tools together for use in workflows.

const toolset = createToolset(tool1, tool2, tool3)

// Type-safe tool execution
const result = await toolset.executeToolCall(block)
if (result.name === 'tool1') {
  // result.result is typed for tool1
}

Workflows

executeSingleToolWorkflow(client, params)

Execute a single-turn conversation expecting one specific tool call.

const result = await executeSingleToolWorkflow(client, {
  model: { model: 'claude-3-5-haiku-latest', max_tokens: 256, temperature: 0 },
  system: 'System prompt',
  messages: [{ role: 'user', content: 'User message' }],
  toolset,
  expectedToolName: 'specific_tool',
  signal: abortSignal,  // optional
  onRetry: (info) => console.log('Retrying...', info)  // optional
})

executeMultiTurnToolWorkflow(client, params)

Execute a multi-turn conversation until a specific final tool is called.

const result = await executeMultiTurnToolWorkflow(client, {
  model: { model: 'claude-3-5-sonnet-latest', max_tokens: 4000, temperature: 0 },
  system: 'System prompt',
  initialMessages: [{ role: 'user', content: 'Start the workflow' }],
  toolset,
  finalToolName: 'final_result',
  maxIterations: 20,  // optional, default: 20
  additionalTools: [  // optional, for built-in tools like web_search
    { type: 'web_search_20250305', name: 'web_search', max_uses: 5 }
  ],
  onToolCall: (toolName, input, result) => {  // optional
    console.log(`Tool ${toolName} called with:`, input, 'Result:', result)
  }
})

Utilities

withRetry(operation, opts)

Retry wrapper with intelligent rate limiting.

const result = await withRetry(
  async () => client.messages.create({ /* params */ }),
  {
    maxAttempts: 5,     // default: 5
    baseDelayMs: 1000,  // default: 1000
    maxDelayMs: 15000,  // default: 15000
    onRetry: (info) => console.log('Retrying...', info)
  }
)

Metrics & Observability

enableMetrics() / disableMetrics()

Enable or disable automatic metrics collection:

import { enableMetrics, disableMetrics } from 'ultra-claude-toolkit'

// Enable metrics collection
enableMetrics()

// Your workflows automatically tracked...

// Disable when not needed (zero overhead)
disableMetrics()

getMetricsCollector()

Access the current metrics collector:

import { getMetricsCollector } from 'ultra-claude-toolkit'

const collector = getMetricsCollector()

// Get current metrics snapshot
const metrics = collector.getMetrics()

// Get specific tool metrics
const toolMetrics = collector.getToolMetrics('my_tool')

// Export as JSON
const exportedData = collector.export()

// Reset all metrics
collector.reset()

Custom Metrics Collection

Implement your own metrics collector:

import { MetricsCollector, setMetricsCollector } from 'ultra-claude-toolkit'

class CustomMetricsCollector implements MetricsCollector {
  recordEvent(event: MetricsEvent): void {
    // Send to your monitoring system
    this.sendToDatadog(event)
  }
  
  // ... implement other methods
}

// Use custom collector
setMetricsCollector(new CustomMetricsCollector())

Advanced Usage

Custom Model Configurations

// Define reusable model configurations
const MODELS = {
  FAST: {
    model: 'claude-3-5-haiku-latest',
    max_tokens: 1024,
    temperature: 0
  },
  POWERFUL: {
    model: 'claude-3-5-sonnet-latest', 
    max_tokens: 4000,
    temperature: 0.1
  }
}

const result = await executeSingleToolWorkflow(client, {
  model: MODELS.FAST,
  // ... other params
})

Enhanced Error Handling

The toolkit provides intelligent error handling with rich debugging context:

Automatic Retry Logic:

• 🎯 Smart Rate Limiting: Uses Anthropic's response headers for optimal retry timing
• 🔄 Server Error Recovery: Exponential backoff for 500/529 errors
• 📊 Metrics Integration: All retry attempts tracked for observability

Rich Error Context: Every error includes detailed debugging information:

• 🕒 Timestamps and attempt numbers
• 🛠️ Tool names and model information
• 📋 Original error objects preserved
• 📈 Metadata for operational analysis

import { withRetry, ErrorContext } from 'ultra-claude-toolkit'

try {
  const result = await executeSingleToolWorkflow(client, params)
} catch (error) {
  // Errors now include rich context for debugging
  console.log('Error occurred at:', error.context?.timestamp)
  console.log('Tool involved:', error.context?.toolName)
  console.log('Attempt number:', error.context?.attempt)
  console.log('Original error:', error.originalError)
}

// Custom retry with context
const result = await withRetry(
  async () => client.messages.create({ /* params */ }),
  {
    maxAttempts: 5,
    onRetry: (info) => console.log(`Retry ${info.attempt}: ${info.message}`)
  },
  { toolName: 'my_tool', modelUsed: 'claude-3-5-sonnet-latest' }
)

TypeScript Support

This library is built with TypeScript-first design:

• Perfect inference: No manual type annotations needed
• Discriminated unions: Tool results are precisely typed
• Generic constraints: Tool names are validated at compile time

// All these are perfectly typed without manual annotations
const toolset = createToolset(tool1, tool2)
const result = await executeSingleToolWorkflow(client, { 
  toolset, 
  expectedToolName: 'tool1'  // ✅ TypeScript knows this is valid
  // expectedToolName: 'invalid'  // ❌ TypeScript error
})
// result.someProperty  // ✅ Perfectly typed based on tool1's schema

🏗️ Architecture & Design

The toolkit is built with a clean, modular architecture:

src/
├── core/              # Tool and toolset fundamentals
├── workflow/          # Single/multi-turn workflow orchestration
├── retry/             # Enhanced error handling & rate limiting  
├── metrics/           # Production observability system
├── utils/             # Schema and utility functions
└── index.ts           # Clean, organized exports

Design Principles:

• 🎯 Ultra-type-safe: Perfect TypeScript inference
• 🧩 Modular: Focused modules with single responsibilities
• 🔒 Backward Compatible: All existing code continues to work
• 📊 Observable: Built-in metrics and debugging support
• ⚡ Performance: Optional features have zero overhead when disabled

🚀 Production Usage

Monitoring & Alerting:

// Set up monitoring
enableMetrics()

// Check health periodically
setInterval(() => {
  const metrics = getMetricsCollector().getMetrics()
  
  if (metrics.workflows.successRate < 0.95) {
    alert('Workflow success rate below 95%')
  }
  
  if (metrics.retries.rateLimitHits > 100) {
    alert('High rate limiting detected')
  }
}, 60000)

Error Tracking:

// Enhanced error context for debugging
try {
  await executeSingleToolWorkflow(client, params)
} catch (error) {
  logger.error('Workflow failed', {
    timestamp: error.context?.timestamp,
    toolName: error.context?.toolName,
    modelUsed: error.context?.modelUsed,
    originalError: error.originalError
  })
}

Performance Optimization:

// Monitor tool performance
const metrics = getMetricsCollector().getMetrics()

Object.entries(metrics.tools).forEach(([toolName, stats]) => {
  if (stats.averageExecutionTime > 5000) {
    console.warn(`Tool ${toolName} is slow: ${stats.averageExecutionTime}ms avg`)
  }
})

License

MIT