@llamagate/ai-sdk-provider

LlamaGate provider for the Vercel AI SDK.

Access 26+ open-source LLMs including Llama, Qwen, DeepSeek, Mistral, and more through a simple, OpenAI-compatible API.

Installation

npm install @llamagate/ai-sdk-provider

Setup

1. Sign up at llamagate.dev to get your API key
2. Set your API key as an environment variable:

export LLAMAGATE_API_KEY=your-api-key

Or pass it directly when creating the provider:

import { createLlamaGate } from '@llamagate/ai-sdk-provider';

const llamagate = createLlamaGate({
  apiKey: 'your-api-key',
});

Usage

Text Generation

import { generateText } from 'ai';
import { llamagate } from '@llamagate/ai-sdk-provider';

const { text } = await generateText({
  model: llamagate('llama-3.1-8b'),
  prompt: 'Explain quantum computing in simple terms.',
});

console.log(text);

Streaming

import { streamText } from 'ai';
import { llamagate } from '@llamagate/ai-sdk-provider';

const { textStream } = await streamText({
  model: llamagate('qwen3-8b'),
  prompt: 'Write a haiku about programming.',
});

for await (const chunk of textStream) {
  process.stdout.write(chunk);
}

Embeddings

import { embed } from 'ai';
import { llamagate } from '@llamagate/ai-sdk-provider';

const { embedding } = await embed({
  model: llamagate.textEmbeddingModel('nomic-embed-text'),
  value: 'The quick brown fox jumps over the lazy dog.',
});

console.log(embedding);

Vision Models

import { generateText } from 'ai';
import { llamagate } from '@llamagate/ai-sdk-provider';

const { text } = await generateText({
  model: llamagate('qwen3-vl-8b'),
  messages: [
    {
      role: 'user',
      content: [
        { type: 'text', text: 'What is in this image?' },
        { type: 'image', image: new URL('https://example.com/image.jpg') },
      ],
    },
  ],
});

Available Models

Chat Models

| Model | Description | Context | |-------|-------------|---------| | llama-3.1-8b | Llama 3.1 8B Instruct | 131K | | llama-3.2-3b | Llama 3.2 3B | 131K | | qwen3-8b | Qwen 3 8B | 32K | | mistral-7b-v0.3 | Mistral 7B Instruct v0.3 | 32K | | deepseek-r1-8b | DeepSeek R1 8B (Reasoning) | 64K | | deepseek-r1-7b-qwen | DeepSeek R1 Distill Qwen 7B | 131K | | openthinker-7b | OpenThinker 7B (Reasoning) | 32K | | dolphin3-8b | Dolphin 3 8B | 128K |

Code Models

| Model | Description | Context | |-------|-------------|---------| | qwen2.5-coder-7b | Qwen 2.5 Coder 7B | 32K | | codellama-7b | CodeLlama 7B | 16K | | deepseek-coder-6.7b | DeepSeek Coder 6.7B | 16K | | codegemma-7b | CodeGemma 7B | 8K | | starcoder2-7b | StarCoder2 7B | 16K |

Vision Models

| Model | Description | Context | |-------|-------------|---------| | qwen3-vl-8b | Qwen 3 VL 8B | 32K | | llava-7b | LLaVA 1.5 7B | 4K | | olmocr-7b | olmOCR 7B (OCR) | 8K | | ui-tars-7b | UI-TARS 1.5 7B | 32K | | gemma3-4b | Gemma 3 4B | 128K |

Embedding Models

| Model | Description | Dimensions | |-------|-------------|------------| | nomic-embed-text | Nomic Embed Text | 768 | | embeddinggemma-300m | EmbeddingGemma 300M | 768 | | qwen3-embedding-8b | Qwen 3 Embedding 8B | 4096 |

Pricing

LlamaGate offers competitive pricing starting at $0.02 per 1M tokens. See llamagate.dev/pricing for full details.

Documentation

• LlamaGate Documentation
• Vercel AI SDK Documentation

License

MIT