@forgeframework/ai-llm

Multi-framework LLM access for the Forge Framework.

Part of the Forge Framework — a TypeScript framework for backend microservices and web applications.

Installation

npm install @forgeframework/ai-llm

Usage

import { DefaultAIProviderFactory, AIFramework, AIModelProvider, TokenTracker } from '@forgeframework/ai-llm';
import { ConfigLoader } from '@forgeframework/config';
import { LoggerFactory } from '@forgeframework/logging';
import { z } from 'zod';

const config = await new ConfigLoader({ sources: [{ type: 'env', prefix: 'FORGE' }] }).load();
const factory = new DefaultAIProviderFactory(config, LoggerFactory);
const provider = factory.createInstance({
  framework: AIFramework.NATIVE,
  provider: AIModelProvider.OPENAI,
  model: 'gpt-4o',
  apiKey: config.get('ai.openai.apiKey'),
  parameters: { temperature: 0.7, maxTokens: 2048 },
});

const response = await provider.chat([
  { role: 'system', content: 'You are a helpful assistant.' },
  { role: 'user', content: 'Explain TypeScript generics.' },
]);

for await (const chunk of provider.chatStream([{ role: 'user', content: 'Write a haiku.' }])) {
  process.stdout.write(chunk.delta);
}

const Sentiment = z.object({ sentiment: z.enum(['positive', 'negative', 'neutral']), confidence: z.number() });
const structured = await provider.structuredOutput(
  [{ role: 'user', content: 'Analyze: "This framework is amazing!"' }],
  { schema: Sentiment, retryOnParseError: true, maxRetries: 3 },
);

const cost = new TokenTracker().estimateCost(provider.getTokenUsage(), 'gpt-4o');

Documentation

Full API reference and guides: https://carbonforge.io/framework/docs/ai/ai-llm

(Documentation site launching soon.)

License

MIT © Carbon Forge