@gotong/llm

LlmAgent base class + neutral LlmProvider interface for Gotong. Builds an LLM-backed Participant in three lines.

The Hub does not call LLMs. LlmAgent does. The provider is the only place vendor SDKs are imported — pick one:

• @gotong/llm-anthropic — Anthropic Claude
• @gotong/llm-openai — OpenAI

Or roll your own by implementing LlmProvider.stream(req) (returns AsyncIterable<LlmStreamChunk>). If you just want the folded final response, pipe through the exported drainStream(provider.stream(req)) helper.

Install

pnpm add @gotong/llm
# plus a provider:
pnpm add @gotong/llm-anthropic @anthropic-ai/sdk

Use

import { Hub } from '@gotong/core'
import { LlmAgent } from '@gotong/llm'
import { AnthropicProvider } from '@gotong/llm-anthropic'

const hub = new Hub()
await hub.start()

hub.register(new LlmAgent({
  id: 'writer',
  capabilities: ['draft'],
  provider: new AnthropicProvider(),                  // reads ANTHROPIC_API_KEY
  system: 'You write one terse sentence.',
}))

const draft = await hub.dispatch({
  from: 'system',
  strategy: { kind: 'capability', capabilities: ['draft'] },
  payload: { topic: 'distributed agents' },
})

Override points

Subclass LlmAgent and override either of these to customize prompt assembly or output shaping:

• buildRequest(task): LlmRequest — translate Task.payload into an LlmRequest. Default reads { prompt } / { topic } / { history } and injects the agent-level system.
• parseResponse(response, task): unknown — translate LlmResponse into the task output. Default returns { text, stopReason, by, usage }.

For full control (multi-step reasoning, tool loops, retries) override handleTask(task) directly.

MockLlmProvider

Ships in this package — a deterministic in-process provider for tests and no-key demos.

import { MockLlmProvider } from '@gotong/llm'

const provider = new MockLlmProvider({
  reply: (req) => `mock reply to ${req.messages.at(-1)?.content}`,
})

License

MIT