@startsimpli/llm

Provider-agnostic LLM client for the StartSimpli Next.js apps. Wraps OpenAI (GPT-5.x, GPT-4o family, fine-tuned ft:* models) and Anthropic (Claude 4 + Claude 3.x) behind a single LLMService<TOutput> that does:

1. provider selection with preferred + fallback,
2. raw text generation or Zod-validated structured output,
3. JSON extraction from messy LLM responses (direct → markdown code block → brace boundary),
4. automatic retry with parse-error correction and schema-error correction prompts when the model returns invalid output,
5. typed LLMProviderError classification (RATE_LIMITED, CONTEXT_LENGTH_EXCEEDED, TIMEOUT, etc.) with a helper to map onto HTTP status codes for Next API routes.

Streaming is not supported — every call is a single non-streaming request. Bring-your-own batching (see Batching below).

A MockLLMProvider ships in-box for testing.

Install

Workspace dep:

"dependencies": {
  "@startsimpli/llm": "workspace:*"
}

Only runtime dep is zod ^4 (for schema validation). HTTP is native fetch — no @anthropic-ai/sdk or openai SDK is pulled in, which keeps the package edge-runtime friendly.

Current consumer status. As of this writing, no app in the monorepo consumes @startsimpli/llm at runtime — it is transpiled by market-simpli/next.config.ts but not imported. raise-simpli/web-app/src/integrations/ still ships an app-local OpenAI provider that duplicates this package. Migrating that app onto @startsimpli/llm is open work; until then, treat the package's own test suite as the canonical usage reference.

Public surface

| Export | Type | Description | | --- | --- | --- | | LLMService<TOutput> | class | Top-level service: provider selection + Zod-validated generation + retries | | GenerationResult<TOutput> / GenerationMetadata | types | Discriminated success/error union returned by .generate() | | LLMServiceConfig | type | { maxRetries?, preferredProvider?, defaultModel?, defaultTemperature?, defaultMaxTokens? } | | ILLMProvider | interface | Contract every provider implements — generate, isAvailable, optional estimateCost | | OpenAIProvider / createOpenAIProvider() | class + factory | OpenAI implementation; factory returns null if OPENAI_API_KEY is unset | | AnthropicProvider / createAnthropicProvider() | class + factory | Anthropic implementation; factory returns null if ANTHROPIC_API_KEY is unset | | MockLLMProvider / setMockBehavior / resetMockBehavior / MockBehavior | class + helpers | Deterministic test provider — control delay, errors, custom responses, invalid-JSON simulation | | LLMProviderError / LLMErrorCode | class + union | Typed error with code, retryable, statusCode, provider | | LLMResponse / TokenUsage / GenerateOptions / ProviderConfig / JsonExtractionResult / ValidationResult<T> | types | Provider/response types | | LLM_DEFAULTS | const | { openaiModel: 'gpt-4o-mini', anthropicModel: 'claude-sonnet-4-20250514', temperature: 0.7, maxTokens: 8192, timeoutMs: 60000 } | | extractJson(content) | function | Robust JSON extractor (direct → ```json block → brace boundary) | | llmErrorToHttpStatus(code) | function | Maps LLMErrorCode → 400/422/429/500/503/504 | | buildSchemaErrorCorrectionPrompt / buildParseErrorCorrectionPrompt / buildGenericRetryPrompt | functions | Domain-agnostic retry prompt builders |

Models recognised for cost estimation

| Provider | Models with cost table baked in | | --- | --- | | OpenAI | gpt-5.2, gpt-5.1, gpt-5, gpt-4o, gpt-4o-mini. Base gpt-5* are auto-forced to temperature=1.0; fine-tuned ft:* clones use the configured temperature. | | Anthropic | claude-opus-4-6, claude-opus-4-20250514, claude-sonnet-4-6, claude-sonnet-4-20250514, claude-3-5-sonnet-20241022, claude-3-5-haiku-20241022, claude-3-opus-20240229, claude-3-sonnet-20240229, claude-3-haiku-20240307. |

Configuration

The factories read from process.env and return null when nothing is configured, so a missing key silently drops a provider from the pool.

| Variable | Used by | Default | | --- | --- | --- | | OPENAI_API_KEY | createOpenAIProvider() | unset → provider disabled | | OPENAI_BASE_URL | createOpenAIProvider() | https://api.openai.com/v1 | | OPENAI_MODEL | createOpenAIProvider() | gpt-4o-mini (LLM_DEFAULTS.openaiModel) | | ANTHROPIC_API_KEY | createAnthropicProvider() | unset → provider disabled | | ANTHROPIC_BASE_URL | createAnthropicProvider() | https://api.anthropic.com/v1 | | ANTHROPIC_MODEL | createAnthropicProvider() | claude-sonnet-4-20250514 (LLM_DEFAULTS.anthropicModel) | | LLM_PROVIDER | LLMService constructor | When set to mock, only the MockLLMProvider is registered |

You can also bypass env wiring entirely by new OpenAIProvider({ apiKey, … }) + service.registerProvider('openai', …).

Batching

Pack N items into one prompt with echoed ids — never loop client.create() per item.

LLMService does one HTTP round-trip per .generate() call (plus up to maxRetries correction attempts on parse/schema failure). It does not transparently batch — calling .generate() in a loop will issue N requests.

Canonical pattern: instead of items.map(item => service.generate(item, …)), build one prompt that contains the full set with stable ids, and a Zod schema for the bulk envelope:

const Bulk = z.object({
  results: z.array(z.object({
    id: z.string(),         // echo back the input id
    score: z.number(),
    label: z.string(),
  })),
})

const userPrompt = `Score each item. Echo the id verbatim.\n\n` +
  items.map(i => `[id=${i.id}] ${i.text}`).join('\n')

const result = await service.generate(userPrompt, systemPrompt, Bulk)
if (result.success) {
  for (const row of result.data.results) {
    // match row.id back to the input
  }
}

This collapses N calls into 1, which is usually 5–50× cheaper in latency and dollars and avoids per-call rate-limit headwind.

Usage

Structured generation with a Zod schema (canonical pattern, lifted from the package's own test suite src/__tests__/llm-service.test.ts):

import { z } from 'zod'
import { LLMService } from '@startsimpli/llm'

const Pattern = z.object({
  name: z.string(),
  value: z.number(),
})

const service = new LLMService<z.infer<typeof Pattern>>({
  preferredProvider: 'anthropic',
  maxRetries: 2,
})

const result = await service.generate(
  'Generate a sample object',
  'You return JSON matching the requested schema.',
  Pattern,
)

if (result.success) {
  console.log(result.data, result.metadata) // { name, value }, { model, promptTokens, … }
} else {
  console.error(result.error.code, result.error.retryable)
}

Raw chat (no schema):

const response = await service.chat(userPrompt, systemPrompt, { model: 'gpt-4o-mini' })
console.log(response.content, response.usage)

Mapping errors to HTTP in a Next API route:

import { llmErrorToHttpStatus, LLMProviderError } from '@startsimpli/llm'

try {
  // ...
} catch (e) {
  if (e instanceof LLMProviderError) {
    return new Response(e.message, { status: llmErrorToHttpStatus(e.code) })
  }
  throw e
}

Testing with the mock provider:

import { LLMService, MockLLMProvider, setMockBehavior } from '@startsimpli/llm'

const mock = new MockLLMProvider()
const service = new LLMService({ preferredProvider: 'mock' })
service.registerProvider('mock', mock)

setMockBehavior({ customResponse: JSON.stringify({ name: 'Alice', value: 42 }) })
const result = await service.generate('prompt', 'system', Pattern)

Verification

cd packages/llm
pnpm vitest run
pnpm tsc --noEmit

Test suite ships 32 tests across 4 files: json-extraction.test.ts, mock-provider.test.ts, retry.test.ts, and llm-service.test.ts. They exercise the JSON extraction strategies, the mock provider's behavior matrix, retry-prompt construction, and the success/failure paths through LLMService.generate.

Shared-first

Per CLAUDE.md rule 9, LLM client wiring never belongs in app src/. If you find a lib/openai.ts or an LLMService clone inside an app (today: raise-simpli/web-app/src/integrations/providers/openai/), the right move is to delete it and import from @startsimpli/llm. Domain prompts and Zod schemas stay in the app; provider abstraction, retries, JSON extraction, and cost accounting stay here.