@betterlu/ai-broker-client

Client SDK for AI Broker. Generate text synchronously, asynchronously, or via streaming, and manage providers and use-cases.

Installation

npm install @betterlu/ai-broker-client

Requires Node.js >= 18 (uses native fetch).

Quick Start

import { AIBrokerClient, AIBrokerAdmin } from '@betterlu/ai-broker-client';

const client = new AIBrokerClient({
  baseUrl: 'https://your-broker.example.com/api',
});

const admin = new AIBrokerAdmin({
  baseUrl: 'https://your-broker.example.com/api',
});

You can also set the AI_BROKER_URL environment variable instead of passing baseUrl:

export AI_BROKER_URL=https://your-broker.example.com/api

const client = new AIBrokerClient(); // reads AI_BROKER_URL from env

Generation

Sync

const result = await client.generate({
  prompt: 'Explain quantum computing in one paragraph',
  useCase: 'coding',
  temperature: 0.7,
});

console.log(result.response?.content);

Streaming

for await (const chunk of client.generateStream({ prompt: 'Tell me a story' })) {
  switch (chunk.type) {
    case 'start':
      console.log('Request:', chunk.requestId);
      break;
    case 'content_delta':
      process.stdout.write(chunk.content);
      break;
    case 'reasoning_delta':
      // thinking model output
      break;
    case 'usage':
      console.log('Tokens:', chunk.usage);
      break;
    case 'done':
      console.log('\nFinished:', chunk.finishReason);
      break;
    case 'error':
      console.error('Error:', chunk.error);
      break;
  }
}

Async Job

// Enqueue
const job = await client.enqueueJob({
  prompt: 'Write a detailed analysis of...',
  useCase: 'analysis',
});

// Poll for status
const status = await client.getJob(job.jobId);
console.log(status.status); // 'queued' | 'running' | 'succeeded' | 'failed'

// Get result when done
const result = await client.getJobResult(job.jobId);
console.log(result.response?.content);

With Tool Loop

const result = await client.generate({
  prompt: 'What is the weather in Tokyo?',
  toolLoop: {
    enabled: true,
    maxSteps: 4,
    tools: ['web_search', 'web_fetch'],
  },
});

console.log(result.response?.content);
console.log(result.response?.toolLoop?.steps, 'tool loop steps');

Admin: Providers

const admin = new AIBrokerAdmin({ baseUrl: 'https://your-broker.example.com/api' });

// List configured providers
const providers = await admin.listProviders();

// Create a new provider
await admin.createProvider({
  name: 'my-openai',
  displayName: 'My OpenAI',
  type: 'openai_compatible',
  baseUrl: 'https://api.openai.com/v1',
  apiKey: 'sk-...',
  defaultModel: 'gpt-4o',
});

// List models available from a provider
const models = await admin.listProviderModels('my-openai');

// Update a provider
await admin.updateProvider('provider-id', { defaultModel: 'gpt-4o-mini' });

// Delete a provider
await admin.deleteProvider('provider-id');

Admin: Use-Case Rules

Use-case rules define preset configurations that callers can reference by name.

// Create a use-case rule
await admin.createUseCase({
  useCase: 'coding',
  providerId: 'my-openai',
  model: 'gpt-4o',
  temperature: 0.3,
  system: 'You are an expert programmer.',
  toolLoop: { enabled: true, maxSteps: 3, tools: ['web_search'] },
});

// List all rules
const rules = await admin.listUseCases();

// Update a rule
await admin.updateUseCase('rule-id', { temperature: 0.5 });

// Delete a rule
await admin.deleteUseCase('rule-id');

Call History

const calls = await client.listCalls();
const call = await client.getCall('call-id');

Health & Defaults

const health = await client.health();
console.log(health.status); // 'ok' | 'degraded' | 'error'

const defaults = await client.defaults();
console.log(defaults.provider, defaults.model, defaults.temperature);

Error Handling

import { AIBrokerError } from '@betterlu/ai-broker-client';

try {
  await client.generate({ prompt: 'Hello' });
} catch (err) {
  if (err instanceof AIBrokerError) {
    console.error(`API error ${err.status}: ${err.message}`);
  } else {
    throw err;
  }
}

Custom Fetch

Pass a custom fetch implementation (useful for Node < 18 or custom middleware):

import fetch from 'node-fetch';

const client = new AIBrokerClient({
  baseUrl: 'https://your-broker.example.com/api',
  fetch: fetch as unknown as typeof globalThis.fetch,
});

API Reference

AIBrokerClient

| Method | Returns | Description | |--------|---------|-------------| | health() | HealthResponse | Check broker health | | defaults() | DefaultsResponse | Get default config | | providers() | ProviderInfo[] | List providers (summary) | | generate(req) | GenerateResponse | Sync generation | | generateStream(req) | AsyncGenerator<StreamChunk> | Streaming generation | | enqueueJob(req) | JobSubmitResponse | Enqueue async job | | listJobs() | JobInfo[] | List recent jobs | | getJob(id) | JobInfo | Get job status | | getJobResult(id) | JobResult | Get completed job result | | listCalls() | CallInfo[] | List recent calls | | getCall(id) | CallInfo | Get a specific call |

AIBrokerAdmin

| Method | Returns | Description | |--------|---------|-------------| | listProviders() | ProviderInfo[] | List providers (summary) | | listProviderRecords() | ProviderRecord[] | List full provider records | | createProvider(req) | ProviderRecord | Create a provider | | updateProvider(id, req) | ProviderRecord | Update a provider | | deleteProvider(id) | { success: boolean } | Delete a provider | | listProviderModels(id) | ProviderModelsResponse | List provider models | | listUseCases() | UseCaseRule[] | List use-case rules | | createUseCase(req) | UseCaseRule | Create a use-case rule | | updateUseCase(id, req) | UseCaseRule | Update a use-case rule | | deleteUseCase(id) | { success: boolean } | Delete a use-case rule |

License

MIT