@inteye/llm-hub

轻量级前端大模型接入 SDK。支持用户自助配置 API Key，内置 18 个国内外主流与本地大模型平台，并提供模型能力标签、OpenAI/Anthropic 自定义接入和 token plan 元信息。

Lightweight frontend LLM integration SDK. It lets users configure their own API keys, ships with 18 built-in providers, and includes model capability metadata, local providers, OpenAI/Anthropic-compatible custom providers, and token plan metadata.

中文

特性

• 统一 chat / chatStream API
• 浏览器与 Node.js/SSR 可用，Node.js 需要 18+
• 零运行时依赖
• 内置 18 个 provider：OpenAI、Anthropic、Gemini、DeepSeek、Qwen、Zhipu、Moonshot、Groq、MiniMax、Mistral、Baichuan、Yi、Step、xAI、Ollama、LM Studio、LocalAI、Custom Provider
• 支持 OpenAI-compatible 和 Anthropic-compatible 自定义接入
• 支持本地 provider：Ollama、LM Studio、LocalAI，以及完全自定义平台
• 支持模型能力判断、标签筛选、多模态判断
• 支持动态拉取模型列表
• API Access 与 Token Plan 分离保存；Token Plan 可作为独立接入凭据或仅作为套餐/账单元信息

安装到业务项目

下面命令用于你的业务项目，不是用于本仓库根目录。

本仓库是 pnpm workspace。如果你是在这个仓库里测试 core，请跳到“本地开发与验证”，不要在仓库根目录运行 npm install @inteye/llm-hub。混用 npm 和 pnpm workspace 可能触发 npm 的 ERESOLVE 依赖解析错误。

# npm
npm install @inteye/llm-hub

# pnpm
pnpm add @inteye/llm-hub

# yarn
yarn add @inteye/llm-hub

快速开始

浏览器中可以使用 localStorage 或 sessionStorage 保存用户配置。

import { createClient } from '@inteye/llm-hub'
import { deepseek, openai, qwen } from '@inteye/llm-hub/providers'

const client = createClient({
  storage: 'localStorage',
  providers: [openai, deepseek, qwen]
})

await client.configure('deepseek', {
  apiKey: 'sk-xxx',
  defaultModel: 'deepseek-v4-flash'
})

const response = await client.chat({
  provider: 'deepseek',
  model: 'deepseek-v4-flash',
  messages: [
    { role: 'user', content: '你好！' }
  ]
})

console.log(response.content)

Node.js / SSR

Node.js/SSR 场景建议使用 memory 存储，并从环境变量读取 API Key。

import { createClient } from '@inteye/llm-hub'
import { openai } from '@inteye/llm-hub/providers'

const client = createClient({
  storage: 'memory',
  providers: [openai]
})

await client.configure('openai', {
  apiKey: process.env.OPENAI_API_KEY || '',
  defaultModel: 'gpt-5.4-mini'
})

CommonJS 也可以使用：

const { createClient } = require('@inteye/llm-hub')
const { openai } = require('@inteye/llm-hub/providers')

流式响应

const stream = client.chatStream({
  provider: 'openai',
  model: 'gpt-5.4-mini',
  messages: [{ role: 'user', content: '写一首短诗' }]
})

for await (const chunk of stream) {
  process.stdout.write(chunk.delta)
}

动态获取模型

fetchModels(provider) 会优先从平台 API 获取模型；如果 provider 不支持或请求失败，会返回内置预置列表。未保存配置时，可以传入临时配置。

const models = await client.fetchModels('openai', {
  apiKey: 'sk-xxx'
})

if (client.canFetchModels('openai')) {
  console.log('OpenAI supports remote model listing.')
}

模型能力与标签

ModelInfo 支持 capabilities、tags、modalities，同时兼容旧字段 supportsVision、supportsStreaming、supportsFunctionCalling。

client.isModelMultimodal('openai', 'gpt-5.4')
client.modelSupports('openai', 'gpt-5.4', 'vision')
client.modelSupports('deepseek', 'deepseek-v4-pro', 'reasoning')

const visionModels = client.filterModels({ capabilities: ['vision'] })
const longContextModels = client.filterModels({ tags: ['long-context'] })
const allPresets = client.listAllModels()

内置派生标签包括 multimodal、long-context、reasoning、coding。自定义模型可以添加任意业务标签。

自定义 provider

推荐使用 createCustomProvider 创建 OpenAI-compatible 或 Anthropic-compatible 接入。

import { createClient } from '@inteye/llm-hub'
import { createCustomProvider } from '@inteye/llm-hub/providers'

const openaiProxy = createCustomProvider({
  protocol: 'openai',
  name: 'openai-proxy',
  displayName: 'OpenAI Proxy',
  defaultBaseUrl: 'https://api.example.com/v1',
  models: [
    {
      id: 'my-model',
      name: 'My Model',
      contextWindow: 32768,
      supportsStreaming: true,
      capabilities: ['text', 'streaming', 'json-mode'],
      tags: ['custom', 'low-cost']
    }
  ]
})

const claudeProxy = createCustomProvider({
  protocol: 'anthropic',
  name: 'claude-proxy',
  displayName: 'Claude Proxy',
  defaultBaseUrl: 'https://claude.example.com',
  anthropicVersion: '2023-06-01',
  models: [
    {
      id: 'claude-compatible',
      name: 'Claude Compatible',
      contextWindow: 200000,
      supportsVision: true,
      supportsStreaming: true
    }
  ]
})

const client = createClient({
  providers: [openaiProxy, claudeProxy]
})

也可以直接使用底层 OpenAICompatibleProvider。

API Access 与 Token Plan

普通 API Key、Base URL、headers、organization、project、apiVersion 属于 API Access。Token Plan / Coding Plan / 订阅 Key 属于独立的 Plan 配置，可能和普通按量 API Key 互斥，也可能只作为账单、额度、套餐元信息存在。

client.configure(provider, config) 只保存普通 API Access。client.configureTokenPlan(provider, plan) 单独保存 Plan，不要求先配置 API Key。

如果某个平台的 Token Plan Key 可以直接调用模型，在 plan.access 中填写 key/baseUrl/protocol，并设置 useAsProviderAccess: true。当普通 API Access 不存在，或该开关显式启用时，client.chat() 会使用这个 Plan credential。protocol 会参与实际请求路由：openai 使用 OpenAI-compatible /chat/completions，anthropic 使用 Anthropic-compatible /v1/messages。

await client.configure('openai', {
  apiKey: process.env.OPENAI_API_KEY || '',
  organization: 'org_xxx',
  project: 'proj_xxx'
})

await client.configureTokenPlan('qwen', {
  type: 'token-plan',
  name: 'Model Studio Token Plan',
  access: {
    apiKey: 'token-plan-key',
    baseUrl: 'https://token-plan.cn-beijing.maas.aliyuncs.com/compatible-mode/v1',
    protocol: 'openai'
  },
  useAsProviderAccess: true,
  quota: {
    totalTokens: 10000000,
    resetAt: '2026-07-01'
  }
})

await client.configureTokenPlan('minimax', {
  type: 'token-plan',
  name: 'MiniMax Token Plan',
  access: {
    apiKey: 'token-plan-key',
    baseUrl: 'https://api.minimax.io/v1',
    protocol: 'openai'
  },
  useAsProviderAccess: true
})

const accessProviders = await client.getConfiguredAccessProviders()
const planProviders = await client.getConfiguredPlanProviders()
const qwenPlanPresets = client.listTokenPlanPresets('qwen')

本地与完全自定义平台

内置 ollama、lmstudio、localai 和 custom。本地平台默认不要求 API Key；custom 可通过 baseUrl、apiKey、protocol 和 defaultModel 接入任意 OpenAI-compatible 或 Anthropic-compatible 服务。支持 /models 的 OpenAI-compatible 服务可用 client.fetchModels(provider) 拉取模型列表；拉取不到时仍可在 defaultModel 中手动填写模型 ID。

await client.configure('ollama', {
  defaultModel: 'llama3.2'
})

await client.configure('custom', {
  baseUrl: 'http://localhost:4000/v1',
  protocol: 'openai',
  defaultModel: 'my-local-model'
})

内置 provider 与预置模型

下面是 SDK 内置预置名单，不等同于各平台实时全量模型。支持 /models 的平台可用 client.fetchModels(provider) 拉取当前账号实际可用模型。

| Provider | 协议 | 预置模型 ID | |----------|------|-------------| | openai | OpenAI | gpt-5.5, gpt-5.4, gpt-5.4-mini, gpt-5.4-nano, gpt-4o, gpt-4o-mini | | anthropic | Anthropic | claude-opus-4-8, claude-sonnet-4-6, claude-haiku-4-5-20251001 | | gemini | Gemini | gemini-3.1-pro-preview, gemini-3.5-flash, gemini-3-flash-preview, gemini-3.1-flash-lite, gemini-2.5-pro, gemini-2.5-flash | | deepseek | OpenAI | deepseek-v4-pro, deepseek-v4-flash, deepseek-chat, deepseek-reasoner | | qwen | OpenAI | qwen3.7-max, qwen3.7-plus, qwen3.6-plus, qwen3.6-flash, qwen3-coder-plus, qwen3-coder-flash | | zhipu | OpenAI | glm-5.1, glm-5, glm-5-turbo, glm-4.7, glm-4.6, glm-4.6v-flash | | moonshot | OpenAI | kimi-k2.6, kimi-k2.5, kimi-k2-thinking, kimi-k2-thinking-turbo, moonshot-v1-128k | | groq | OpenAI | groq/compound, groq/compound-mini, openai/gpt-oss-120b, openai/gpt-oss-20b, meta-llama/llama-4-scout-17b-16e-instruct, llama-3.3-70b-versatile, llama-3.1-8b-instant | | minimax | OpenAI | MiniMax-M3, MiniMax-M2.7, MiniMax-M2.7-highspeed, MiniMax-M2.5, MiniMax-M2.5-highspeed | | mistral | OpenAI | mistral-medium-3-5, mistral-large-2512, mistral-small-2603, magistral-medium-2509, ministral-3-14b-latest, codestral-2512 | | baichuan | OpenAI | Baichuan4, Baichuan3-Turbo, Baichuan3-Turbo-128k, Baichuan2-Turbo | | yi | OpenAI | yi-large, yi-medium, yi-medium-200k, yi-spark, yi-large-turbo, yi-vision | | step | OpenAI | step-1-8k, step-1-32k, step-1-128k, step-1-flash, step-2-16k | | xai | OpenAI | grok-4.3, grok-4.3-latest, grok-4.1-fast, grok-build-0.1 | | ollama | OpenAI-compatible local | llama3.2, qwen2.5-coder:7b, deepseek-r1:8b | | lmstudio | OpenAI-compatible local | local-model | | localai | OpenAI-compatible local | local-model | | custom | OpenAI 或 Anthropic compatible | custom-model |

API 速查

| API | 说明 | |-----|------| | createClient(options) | 创建客户端 | | client.registerProvider(adapter) | 注册 provider | | client.configure(provider, config) | 保存普通 API Access 配置 | | client.getConfigAsync(provider) | 异步读取 API Access；如有 Plan 会附带 tokenPlan | | client.getProviderAccess(provider) | 只读取普通 API Access | | client.getEffectiveProviderAccess(provider) | 读取最终会用于调用的凭据，可能来自 API Access，也可能来自 active Plan credential | | client.getAvailableProviders() | 获取已有可调用凭据的 provider，包含 API Access 或可用 Plan credential | | client.getConfiguredAccessProviders() | 获取已配置普通 API Access 的 provider | | client.getConfiguredPlanProviders() | 获取已配置 Token Plan 的 provider | | client.chat(options) | 非流式对话 | | client.chatStream(options) | 流式对话 | | client.validateProvider(provider, configOverride?) | 验证配置，返回布尔值 | | client.testProvider(provider, configOverride?, { model }) | 发起最小 chat 测试并抛出上游具体错误，适合 UI 调试 | | client.fetchModels(provider, configOverride?) | 拉取模型列表 | | client.listModels(provider) | 获取内置模型列表 | | client.listAllModels(filter?) / client.filterModels(filter) | 跨 provider 筛选模型 | | client.getModel(provider, model) | 获取模型元数据 | | client.isModelMultimodal(provider, model) | 判断多模态 | | client.modelSupports(provider, model, capability) | 判断模型能力 | | client.getModelTags(provider, model) | 获取模型标签 | | client.configureTokenPlan(provider, tokenPlan) | 单独保存 Token Plan / Coding Plan / 订阅元信息或 Plan credential | | client.getTokenPlan(provider) | 读取 Token Plan | | client.listTokenPlanPresets(provider) | 获取平台内置的 Token Plan / Coding Plan 接入预设 |

本地开发与验证

在仓库根目录运行：

pnpm install
pnpm --filter @inteye/llm-hub lint
pnpm --filter @inteye/llm-hub test
pnpm --filter @inteye/llm-hub build

全仓验证：

pnpm lint
pnpm test
pnpm build

许可证

MIT

English

Features

• Unified chat / chatStream API
• Browser and Node.js/SSR support, Node.js 18+
• Zero runtime dependencies
• 18 built-in providers: OpenAI, Anthropic, Gemini, DeepSeek, Qwen, Zhipu, Moonshot, Groq, MiniMax, Mistral, Baichuan, Yi, Step, xAI, Ollama, LM Studio, LocalAI, and Custom Provider
• OpenAI-compatible and Anthropic-compatible custom providers
• Local providers for Ollama, LM Studio, LocalAI, plus a fully custom provider
• Model capability detection, tag filtering, and multimodal checks
• Remote model listing when a provider supports it
• Separate API Access and Token Plan storage; token plans can be call credentials or billing/account metadata

Install in an App Project

The commands below are for your application project, not for this repository root.

This repository uses a pnpm workspace. If you are testing core inside this repository, skip to "Local Development and Verification" and do not run npm install @inteye/llm-hub at the repository root. Mixing npm with this pnpm workspace can trigger npm ERESOLVE dependency resolution errors.

# npm
npm install @inteye/llm-hub

# pnpm
pnpm add @inteye/llm-hub

# yarn
yarn add @inteye/llm-hub

Quick Start

Use localStorage or sessionStorage in browser apps.

import { createClient } from '@inteye/llm-hub'
import { deepseek, openai, qwen } from '@inteye/llm-hub/providers'

const client = createClient({
  storage: 'localStorage',
  providers: [openai, deepseek, qwen]
})

await client.configure('deepseek', {
  apiKey: 'sk-xxx',
  defaultModel: 'deepseek-v4-flash'
})

const response = await client.chat({
  provider: 'deepseek',
  model: 'deepseek-v4-flash',
  messages: [
    { role: 'user', content: 'Hello!' }
  ]
})

console.log(response.content)

Node.js / SSR

Use memory storage in Node.js/SSR and read secrets from environment variables.

import { createClient } from '@inteye/llm-hub'
import { openai } from '@inteye/llm-hub/providers'

const client = createClient({
  storage: 'memory',
  providers: [openai]
})

await client.configure('openai', {
  apiKey: process.env.OPENAI_API_KEY || '',
  defaultModel: 'gpt-5.4-mini'
})

CommonJS is supported:

const { createClient } = require('@inteye/llm-hub')
const { openai } = require('@inteye/llm-hub/providers')

Streaming

const stream = client.chatStream({
  provider: 'openai',
  model: 'gpt-5.4-mini',
  messages: [{ role: 'user', content: 'Write a short poem.' }]
})

for await (const chunk of stream) {
  process.stdout.write(chunk.delta)
}

Model Listing

fetchModels(provider) tries the provider API first. If listing is unsupported or the request fails, it returns the built-in preset list. You can pass a temporary config before saving credentials.

const models = await client.fetchModels('openai', {
  apiKey: 'sk-xxx'
})

if (client.canFetchModels('openai')) {
  console.log('OpenAI supports remote model listing.')
}

Model Capabilities and Tags

ModelInfo supports capabilities, tags, and modalities, while remaining compatible with supportsVision, supportsStreaming, and supportsFunctionCalling.

client.isModelMultimodal('openai', 'gpt-5.4')
client.modelSupports('openai', 'gpt-5.4', 'vision')
client.modelSupports('deepseek', 'deepseek-v4-pro', 'reasoning')

const visionModels = client.filterModels({ capabilities: ['vision'] })
const longContextModels = client.filterModels({ tags: ['long-context'] })
const allPresets = client.listAllModels()

Derived tags include multimodal, long-context, reasoning, and coding. Custom models can add any product-specific tags.

Custom Providers

Use createCustomProvider for OpenAI-compatible or Anthropic-compatible APIs.

import { createClient } from '@inteye/llm-hub'
import { createCustomProvider } from '@inteye/llm-hub/providers'

const openaiProxy = createCustomProvider({
  protocol: 'openai',
  name: 'openai-proxy',
  displayName: 'OpenAI Proxy',
  defaultBaseUrl: 'https://api.example.com/v1',
  models: [
    {
      id: 'my-model',
      name: 'My Model',
      contextWindow: 32768,
      supportsStreaming: true,
      capabilities: ['text', 'streaming', 'json-mode'],
      tags: ['custom', 'low-cost']
    }
  ]
})

const client = createClient({
  providers: [openaiProxy]
})

API Access and Token Plans

Normal API keys, Base URLs, headers, organization, project, and API version belong to API Access. Token Plan, Coding Plan, subscription keys, and credit/quota metadata are stored as a separate Plan config. Depending on the provider, a plan key can be mutually exclusive with the normal pay-as-you-go API key, or it can be metadata only.

client.configure(provider, config) saves normal API Access. client.configureTokenPlan(provider, plan) saves Plan config independently and does not require an API key to exist first.

If a provider's Token Plan key can call model APIs, put its key/baseUrl/protocol in plan.access and set useAsProviderAccess: true. client.chat() can then use that plan credential when normal API Access is absent or explicitly preferred. protocol affects the real request route: openai uses OpenAI-compatible /chat/completions, and anthropic uses Anthropic-compatible /v1/messages.

await client.configure('openai', {
  apiKey: process.env.OPENAI_API_KEY || '',
  organization: 'org_xxx',
  project: 'proj_xxx'
})

await client.configureTokenPlan('qwen', {
  type: 'token-plan',
  name: 'Model Studio Token Plan',
  access: {
    apiKey: 'token-plan-key',
    baseUrl: 'https://token-plan.cn-beijing.maas.aliyuncs.com/compatible-mode/v1',
    protocol: 'openai'
  },
  useAsProviderAccess: true,
  quota: {
    totalTokens: 10000000,
    resetAt: '2026-07-01'
  }
})

await client.configureTokenPlan('minimax', {
  type: 'token-plan',
  name: 'MiniMax Token Plan',
  access: {
    apiKey: 'token-plan-key',
    baseUrl: 'https://api.minimax.io/v1',
    protocol: 'openai'
  },
  useAsProviderAccess: true
})

const accessProviders = await client.getConfiguredAccessProviders()
const planProviders = await client.getConfiguredPlanProviders()
const qwenPlanPresets = client.listTokenPlanPresets('qwen')

Local and Fully Custom Providers

Built-in providers include ollama, lmstudio, localai, and custom. Local providers do not require an API key by default. custom can connect to any OpenAI-compatible or Anthropic-compatible service with baseUrl, apiKey, protocol, and defaultModel. OpenAI-compatible services that expose /models can use client.fetchModels(provider); when the remote list is unavailable, set defaultModel manually.

await client.configure('ollama', {
  defaultModel: 'llama3.2'
})

await client.configure('custom', {
  baseUrl: 'http://localhost:4000/v1',
  protocol: 'openai',
  defaultModel: 'my-local-model'
})

Built-in Providers and Preset Models

The table below is the SDK preset list, not a real-time full catalog for each platform. Use client.fetchModels(provider) for account-specific availability where supported.

| Provider | Protocol | Preset model IDs | |----------|----------|------------------| | openai | OpenAI | gpt-5.5, gpt-5.4, gpt-5.4-mini, gpt-5.4-nano, gpt-4o, gpt-4o-mini | | anthropic | Anthropic | claude-opus-4-8, claude-sonnet-4-6, claude-haiku-4-5-20251001 | | gemini | Gemini | gemini-3.1-pro-preview, gemini-3.5-flash, gemini-3-flash-preview, gemini-3.1-flash-lite, gemini-2.5-pro, gemini-2.5-flash | | deepseek | OpenAI | deepseek-v4-pro, deepseek-v4-flash, deepseek-chat, deepseek-reasoner | | qwen | OpenAI | qwen3.7-max, qwen3.7-plus, qwen3.6-plus, qwen3.6-flash, qwen3-coder-plus, qwen3-coder-flash | | zhipu | OpenAI | glm-5.1, glm-5, glm-5-turbo, glm-4.7, glm-4.6, glm-4.6v-flash | | moonshot | OpenAI | kimi-k2.6, kimi-k2.5, kimi-k2-thinking, kimi-k2-thinking-turbo, moonshot-v1-128k | | groq | OpenAI | groq/compound, groq/compound-mini, openai/gpt-oss-120b, openai/gpt-oss-20b, meta-llama/llama-4-scout-17b-16e-instruct, llama-3.3-70b-versatile, llama-3.1-8b-instant | | minimax | OpenAI | MiniMax-M3, MiniMax-M2.7, MiniMax-M2.7-highspeed, MiniMax-M2.5, MiniMax-M2.5-highspeed | | mistral | OpenAI | mistral-medium-3-5, mistral-large-2512, mistral-small-2603, magistral-medium-2509, ministral-3-14b-latest, codestral-2512 | | baichuan | OpenAI | Baichuan4, Baichuan3-Turbo, Baichuan3-Turbo-128k, Baichuan2-Turbo | | yi | OpenAI | yi-large, yi-medium, yi-medium-200k, yi-spark, yi-large-turbo, yi-vision | | step | OpenAI | step-1-8k, step-1-32k, step-1-128k, step-1-flash, step-2-16k | | xai | OpenAI | grok-4.3, grok-4.3-latest, grok-4.1-fast, grok-build-0.1 | | ollama | OpenAI-compatible local | llama3.2, qwen2.5-coder:7b, deepseek-r1:8b | | lmstudio | OpenAI-compatible local | local-model | | localai | OpenAI-compatible local | local-model | | custom | OpenAI or Anthropic compatible | custom-model |

API Reference

| API | Description | |-----|-------------| | createClient(options) | Create a client | | client.registerProvider(adapter) | Register a provider | | client.configure(provider, config) | Save normal API Access config | | client.getConfigAsync(provider) | Read API Access asynchronously; includes tokenPlan when one exists | | client.getProviderAccess(provider) | Read only normal API Access | | client.getEffectiveProviderAccess(provider) | Read the credential that will actually be used for calls, from API Access or an active Plan credential | | client.getAvailableProviders() | List providers with usable call credentials, including API Access or Plan credentials | | client.getConfiguredAccessProviders() | List providers with normal API Access | | client.getConfiguredPlanProviders() | List providers with Token Plans | | client.chat(options) | Non-streaming chat | | client.chatStream(options) | Streaming chat | | client.validateProvider(provider, configOverride?) | Validate config and return a boolean | | client.testProvider(provider, configOverride?, { model }) | Send a minimal chat test and throw the upstream error, useful for UI debugging | | client.fetchModels(provider, configOverride?) | Fetch models | | client.listModels(provider) | List built-in models for one provider | | client.listAllModels(filter?) / client.filterModels(filter) | Filter models across providers | | client.getModel(provider, model) | Get model metadata | | client.isModelMultimodal(provider, model) | Check multimodal support | | client.modelSupports(provider, model, capability) | Check a model capability | | client.getModelTags(provider, model) | Get model tags | | client.configureTokenPlan(provider, tokenPlan) | Save Token Plan / Coding Plan / subscription metadata or Plan credentials independently | | client.getTokenPlan(provider) | Read a Token Plan | | client.listTokenPlanPresets(provider) | List built-in Token Plan / Coding Plan access presets |

Local Development and Verification

Run from the repository root:

pnpm install
pnpm --filter @inteye/llm-hub lint
pnpm --filter @inteye/llm-hub test
pnpm --filter @inteye/llm-hub build

Full workspace verification:

pnpm lint
pnpm test
pnpm build

License

MIT