@friendliai/ai-provider

A provider to use FriendliAI models with the Vercel AI SDK and OpenAI-compatible APIs.

Installation

npm i @friendliai/ai-provider

Example

import { friendli } from '@friendliai/ai-provider';
import { generateText } from 'ai';

const { text } = await generateText({
  model: friendli('meta-llama-3.3-70b-instruct'),
  prompt: 'What is the meaning of life?',
  maxTokens: 20,
  topK: 10,
  providerOptions: {
    friendliai: {
      minP: 0.2,
    },
  },
});
console.log(text);

Provider Options

You can pass Friendli-specific parameters using providerOptions.friendliai. For backward compatibility, the friendli key is also supported.

Supported options (camelCase):

• minP
• repetitionPenalty
• xtcThreshold
• xtcProbability
• parallelToolCalls
• regex (BETA: Force output to satisfy a regular expression)
• chat_template_kwargs

For more details, see the official documentation.