Genkit

Genkit is a framework for building AI-powered applications. It provides open source libraries for Node.js and Go, along with tools to help you debug and iterate quickly.

Quick Start

Install the following Genkit dependencies to use Genkit in your project:

• genkit - Genkit core capabilities.
• A model plugin, e.g. @genkit-ai/google-genai for Google AI Gemini models.

npm install genkit @genkit-ai/google-genai

Set up your API key:

export GOOGLE_API_KEY=your-api-key

Make your first request:

import { genkit } from 'genkit';
import { googleAI } from '@genkit-ai/google-genai';

const ai = genkit({ plugins: [googleAI()] });

const { text } = await ai.generate({
  model: googleAI.model('gemini-flash-latest'),
  prompt: 'Why is Genkit awesome?',
});

console.log(text);

Key Features

Structured Output

Generate strongly-typed, schema-validated output using Zod schemas:

import { genkit, z } from 'genkit';
import { googleAI } from '@genkit-ai/google-genai';

const ai = genkit({ plugins: [googleAI()] });

const RecipeSchema = z.object({
  title: z.string(),
  ingredients: z.array(z.string()),
  instructions: z.array(z.string()),
});

const { output } = await ai.generate({
  model: googleAI.model('gemini-flash-latest'),
  prompt: 'Invent a new pasta recipe',
  output: { schema: RecipeSchema },
});

console.log(output?.title); // fully typed

Streaming

Stream responses in real time with generateStream:

const { response, stream } = ai.generateStream({
  model: googleAI.model('gemini-flash-latest'),
  prompt: 'Write a short story about a robot',
});

for await (const chunk of stream) {
  process.stdout.write(chunk.text);
}

Tools (Function Calling)

Define tools that models can call automatically to access external data or perform actions:

const getWeather = ai.defineTool(
  {
    name: 'getWeather',
    description: 'Gets the current weather for a given city',
    inputSchema: z.object({ city: z.string() }),
    outputSchema: z.object({ temperature: z.number(), condition: z.string() }),
  },
  async ({ city }) => {
    // your implementation here
    return { temperature: 72, condition: 'sunny' };
  }
);

const { text } = await ai.generate({
  model: googleAI.model('gemini-flash-latest'),
  prompt: 'What should I wear in Tokyo today?',
  tools: [getWeather],
});

Interrupts (Human-in-the-Loop)

Beta feature: Interrupts require importing from genkit/beta instead of genkit:

import { genkit } from 'genkit/beta';

Interrupts pause model processing and return control to the caller, enabling human-in-the-loop workflows. There are two patterns:

Basic Interrupts

Use defineInterrupt to create a tool that always pauses. The caller provides a response with .respond():

const confirmAction = ai.defineInterrupt({
  name: 'confirmAction',
  description: 'Confirm an action with the user before proceeding',
  inputSchema: z.object({ action: z.string(), reason: z.string() }),
  outputSchema: z.object({ approved: z.boolean() }),
});

let response = await ai.generate({
  model: googleAI.model('gemini-flash-latest'),
  prompt: 'Book a table for 2 at 7pm tonight',
  tools: [confirmAction],
});

// The model triggered an interrupt - get user approval
if (response.interrupts.length) {
  const interrupt = response.interrupts[0];
  console.log(interrupt.toolRequest.input); // { action: '...', reason: '...' }

  // Resume with the user's response (bypasses tool execution)
  response = await ai.generate({
    model: googleAI.model('gemini-flash-latest'),
    messages: response.messages,
    tools: [confirmAction],
    resume: {
      respond: confirmAction.respond(interrupt, { approved: true }),
    },
  });
}

Restartable Tools

Regular tools can conditionally interrupt using interrupt() and be re-executed with .restart(). The resumed flag lets the tool know it's been approved:

const sendEmail = ai.defineTool(
  {
    name: 'sendEmail',
    description: 'Sends an email',
    inputSchema: z.object({ to: z.string(), body: z.string() }),
    outputSchema: z.object({ sent: z.boolean() }),
  },
  async (input, { interrupt, resumed }) => {
    if (!resumed) {
      interrupt({ message: `Send email to ${input.to}?` });
    }
    // Approved - proceed with sending
    return { sent: true };
  }
);

let response = await ai.generate({
  model: googleAI.model('gemini-flash-latest'),
  prompt: 'Send a hello email to [email protected]',
  tools: [sendEmail],
});

if (response.interrupts.length) {
  const interrupt = response.interrupts[0];
  // Restart re-executes the tool, this time with resumed=true
  response = await ai.generate({
    model: googleAI.model('gemini-flash-latest'),
    messages: response.messages,
    tools: [sendEmail],
    resume: { restart: [sendEmail.restart(interrupt)] },
  });
}

The toolApproval middleware from @genkit-ai/middleware automates this pattern, interrupting any tool not in an approved list:

import { toolApproval } from '@genkit-ai/middleware';
import { restartTool } from 'genkit';

let response = await ai.generate({
  model: googleAI.model('gemini-flash-latest'),
  prompt: 'Send a hello email to [email protected]',
  tools: [sendEmail, readInbox],
  use: [toolApproval({ approved: ['readInbox'] })], // sendEmail not approved
});

// sendEmail was interrupted - get user approval, then restart
if (response.interrupts.length) {
  response = await ai.generate({
    model: googleAI.model('gemini-flash-latest'),
    messages: response.messages,
    tools: [sendEmail, readInbox],
    resume: {
      restart: response.interrupts.map((i) =>
        restartTool(i, { toolApproved: true })
      ),
    },
    use: [toolApproval({ approved: ['readInbox'] })],
  });
}

Prompts (Dotprompt)

Manage prompts as code with embedded schemas, model configuration, and Handlebars templating:

---
model: googleai/gemini-flash-latest
input:
  schema:
    topic: string
output:
  schema:
    title: string
    summary: string
---
Write a blog post about {{topic}}.

const blogPrompt = ai.prompt('blog');
const { output } = await blogPrompt({ topic: 'AI safety' });

Flows

Build strongly typed, fully observable workflows that can be served as APIs and accessed from the client:

import { genkit, z } from 'genkit';
import { googleAI } from '@genkit-ai/google-genai';

const ai = genkit({
  plugins: [googleAI()],
  model: googleAI.model('gemini-flash-latest'),
});

const RecipeSchema = z.object({
  title: z.string(),
  ingredients: z.array(z.string()),
  instructions: z.array(z.string()),
});

export const recipeFlow = ai.defineFlow(
  {
    name: 'recipeFlow',
    inputSchema: z.object({ ingredient: z.string() }),
    outputSchema: RecipeSchema,
  },
  async (input) => {
    const { output } = await ai.generate({
      prompt: `Create a recipe using ${input.ingredient}`,
      output: { schema: RecipeSchema },
    });
    if (!output) throw new Error('Failed to generate recipe');
    return output;
  }
);

Serve flows as an API:

import { startFlowServer } from '@genkit-ai/express'; // npm i @genkit-ai/express

startFlowServer({ flows: [recipeFlow] });

Access from the client:

import { streamFlow } from 'genkit/beta/client';

const { stream } = streamFlow({
  url: 'http://localhost:3500/recipeFlow',
  input: { ingredient: 'avocado' },
});

for await (const chunk of stream) {
  console.log(chunk);
}

Agents

Beta feature: Agents require importing from genkit/beta instead of genkit:

import { genkit } from 'genkit/beta';

Agents are stateful, multi-turn conversations built on top of prompts and tools. defineAgent bundles a system prompt, tools, and (optionally) a session store into a single ergonomic chat API. A chat carries state across turns automatically, so you don't have to thread message history by hand:

import { genkit, z, FileSessionStore } from 'genkit/beta';
import { googleAI } from '@genkit-ai/google-genai';

const ai = genkit({ plugins: [googleAI()] });

const getWeather = ai.defineTool(
  {
    name: 'getWeather',
    description: 'Get the current weather for a given location.',
    inputSchema: z.object({ location: z.string() }),
    outputSchema: z.object({ weather: z.string() }),
  },
  async ({ location }) => ({ weather: `Sunny in ${location}` })
);

const weatherAgent = ai.defineAgent({
  name: 'weatherAgent',
  system: 'You are a helpful weather assistant. Use the getWeather tool.',
  tools: [getWeather],
  // Optional: persist each turn as a resumable snapshot.
  store: new FileSessionStore('./.snapshots'),
});

const chat = weatherAgent.chat();

// First turn (streaming).
const turn = chat.sendStream('What is the weather in London?');
for await (const chunk of turn.stream) {
  process.stdout.write(chunk.text ?? '');
}
await turn.response;

// Follow-up turn reuses the same chat - state is carried automatically.
const res = await chat.send('Now say that in French');
console.log(res.text);

Agents can be served over HTTP and accessed from the client with remoteAgent, which returns the exact same chat API as the server:

import { remoteAgent } from 'genkit/beta/client';

const AGENT_URL = 'YOUR_AGENT_URL';
const agent = remoteAgent({ url: AGENT_URL });
const chat = agent.chat();
const res = await chat.send('Weather in Tokyo?');
console.log(res.text);

Learn more in the Agents documentation.

Middleware

The @genkit-ai/middleware package provides ready-made middleware to add common functionality to your AI requests:

• retry - Automatically retry failed requests with exponential backoff.
• fallback - Fall back to alternative models on specific error statuses.
• toolApproval - Restrict tool execution to an approved list, interrupting unapproved calls for review.
• filesystem - Give the model sandboxed read/write access to a directory on the filesystem.
• skills - Scan for skill definitions and inject them as available tools.
• agents - Delegate to sub-agents by exposing each as a dedicated delegation tool.
• artifacts - Give the model tools to read, create, and update session artifacts.

npm install @genkit-ai/middleware

import { retry } from '@genkit-ai/middleware';

const { text } = await ai.generate({
  model: googleAI.model('gemini-flash-latest'),
  prompt: 'Why is Genkit awesome?',
  use: [
    retry({
      maxRetries: 3,
      initialDelayMs: 1000,
      backoffFactor: 2,
    }),
  ],
});

Developer Tools

Genkit comes with a powerful CLI and Developer UI for locally testing, debugging, and iterating on your AI features:

npx genkit start -- npx tsx src/index.ts

The Developer UI lets you visually test flows, inspect traces, and experiment with prompts - all in your browser.

Plugins

Genkit supports a growing ecosystem of plugins for model providers, vector stores, and more:

| Category | Plugins | |---|---| | Models | @genkit-ai/google-genai, @genkit-ai/vertexai, @genkit-ai/compat-oai, genkitx-anthropic, genkitx-ollama | | Deployment | @genkit-ai/express, @genkit-ai/fetch, @genkit-ai/firebase, @genkit-ai/cloud-run | | Monitoring | @genkit-ai/google-cloud |

Browse all plugins: npmjs.com/search?q=keywords:genkit-plugin

Deployment

Genkit flows can be deployed anywhere Node.js runs:

• Express - Deploy to Node.js
• Firebase - Deploy to Firebase
• Cloud Run - Deploy to Cloud Run

Next Steps

• Developer tools: Set up and use Genkit's CLI and developer UI.
• Generating content: Use Genkit's unified generation API.
• Building agents: Build stateful, multi-turn agents.
• Creating flows: Build observable workflows with rich debugging.
• Managing prompts: Manage prompts and configuration as code.

Learn more at genkit.dev

License: Apache 2.0