Groq Model Provider Plugin for Genkit

Firebase Genkit framework plugin for Groq APIs (Fast LLM inference Provider)

Installation

npm install genkitx-plugin-groq

Quick Start

1. Set up your environment

export GROQ_API_KEY="your-api-key-here"

2. Create a Genkit application

import { genkit } from "genkit";
import { z } from "genkit";
import { groq, groqModel } from "genkitx-plugin-groq";

const ai = genkit({
  plugins: [groq()],
});

// Define a flow
export const myFlow = ai.defineFlow(
  {
    name: "myFlow",
    inputSchema: z.string(),
    outputSchema: z.string(),
  },
  async (input) => {
    const response = await ai.generate({
      model: groqModel("llama-3.1-8b-instant"),
      prompt: `Tell me about: ${input}`,
      config: {
        temperature: 0.7,
        maxOutputTokens: 512,
      },
    });

    return response.text;
  },
);

3. Run the Developer UI

genkit start -- npm run dev

Configuration

Plugin Options

interface GroqPluginOptions {
  apiKey?: string; // Groq API key (defaults to GROQ_API_KEY env var)
}

Usage Examples

Basic Text Generation

const response = await ai.generate({
  model: groqModel("llama-3.1-8b-instant"),
  prompt: "Write a haiku about programming",
});

console.log(response.text);

Multi-turn Conversation

const response = await ai.generate({
  model: groqModel("llama-3.1-8b-instant"),
  messages: [
    { role: "user", content: "What is quantum computing?" },
    {
      role: "assistant",
      content: "Quantum computing uses quantum mechanics principles...",
    },
    { role: "user", content: "Can you explain superposition?" },
  ],
});

console.log(response.text);

System Prompt

const response = await ai.generate({
  model: groqModel("llama-3.1-8b-instant"),
  system: "You are a helpful programming assistant.",
  prompt: "How do I reverse a string in JavaScript?",
});

console.log(response.text);

Limitations

• Media input (images, videos, audio) is not currently supported
• Tool calling is not currently supported (can be added in future versions)
• Maximum context window varies by model (see Groq documentation)

Troubleshooting

API Key Not Found

Error: Must supply either `options.apiKey` or set `GROQ_API_KEY` environment variable.

Solution: Set the GROQ_API_KEY environment variable or pass it to the plugin options.

Resources

• Groq API Documentation
• Genkit Documentation

Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

License

Apache License 2.0 - See LICENSE file for details