Genkit

Genkit is a framework for building AI-powered applications. It provides open source libraries for Node.js and Go, along with tools to help you debug and iterate quickly.

Quick Start

Install the following Genkit dependencies to use Genkit in your project:

• genkit - Genkit core capabilities.
• A model plugin, e.g. @genkit-ai/google-genai for Google AI Gemini models.

npm install genkit @genkit-ai/google-genai

Set up your API key:

export GOOGLE_API_KEY=your-api-key

Make your first request:

import { genkit } from 'genkit';
import { googleAI } from '@genkit-ai/google-genai';

const ai = genkit({ plugins: [googleAI()] });

const { text } = await ai.generate({
  model: googleAI.model('gemini-flash-latest'),
  prompt: 'Why is Genkit awesome?',
});

console.log(text);

Key Features

Structured Output

Generate strongly-typed, schema-validated output using Zod schemas:

import { genkit, z } from 'genkit';
import { googleAI } from '@genkit-ai/google-genai';

const ai = genkit({ plugins: [googleAI()] });

const RecipeSchema = z.object({
  title: z.string(),
  ingredients: z.array(z.string()),
  instructions: z.array(z.string()),
});

const { output } = await ai.generate({
  model: googleAI.model('gemini-flash-latest'),
  prompt: 'Invent a new pasta recipe',
  output: { schema: RecipeSchema },
});

console.log(output?.title); // fully typed

Streaming

Stream responses in real time with generateStream:

const { response, stream } = ai.generateStream({
  model: googleAI.model('gemini-flash-latest'),
  prompt: 'Write a short story about a robot',
});

for await (const chunk of stream) {
  process.stdout.write(chunk.text);
}

Tools (Function Calling)

Define tools that models can call automatically to access external data or perform actions:

const getWeather = ai.defineTool(
  {
    name: 'getWeather',
    description: 'Gets the current weather for a given city',
    inputSchema: z.object({ city: z.string() }),
    outputSchema: z.object({ temperature: z.number(), condition: z.string() }),
  },
  async ({ city }) => {
    // your implementation here
    return { temperature: 72, condition: 'sunny' };
  }
);

const { text } = await ai.generate({
  model: googleAI.model('gemini-flash-latest'),
  prompt: 'What should I wear in Tokyo today?',
  tools: [getWeather],
});

Interrupts (Human-in-the-Loop)

Beta feature: Interrupts require importing from genkit/beta instead of genkit:

import { genkit } from 'genkit/beta';

Interrupts pause model processing and return control to the caller, enabling human-in-the-loop workflows. There are two patterns:

Basic Interrupts

Use defineInterrupt to create a tool that always pauses. The caller provides a response with .respond():

const confirmAction = ai.defineInterrupt({
  name: 'confirmAction',
  description: 'Confirm an action with the user before proceeding',
  inputSchema: z.object({ action: z.string(), reason: z.string() }),
  outputSchema: z.object({ approved: z.boolean() }),
});

let response = await ai.generate({
  model: googleAI.model('gemini-flash-latest'),
  prompt: 'Book a table for 2 at 7pm tonight',
  tools: [confirmAction],
});

// The model triggered an interrupt - get user approval
if (response.interrupts.length) {
  const interrupt = response.interrupts[0];
  console.log(interrupt.toolRequest.input); // { action: '...', reason: '...' }

  // Resume with the user's response (bypasses tool execution)
  response = await ai.generate({
    model: googleAI.model('gemini-flash-latest'),
    messages: response.messages,
    tools: [confirmAction],
    resume: {
      respond: confirmAction.respond(interrupt, { approved: true }),
    },
  });
}

Restartable Tools

Regular tools can conditionally interrupt using interrupt() and be re-executed with .restart(). The resumed flag lets the tool know it's been approved:

const sendEmail = ai.defineTool(
  {
    name: 'sendEmail',
    description: 'Sends an email',
    inputSchema: z.object({ to: z.string(), body: z.string() }),
    outputSchema: z.object({ sent: z.boolean() }),
  },
  async (input, { interrupt, resumed }) => {
    if (!resumed) {
      interrupt({ message: `Send email to ${input.to}?` });
    }
    // Approved - proceed with sending
    return { sent: true };
  }
);

let response = await ai.generate({
  model: googleAI.model('gemini-flash-latest'),
  prompt: 'Send a hello email to [email protected]',
  tools: [sendEmail],
});

if (response.interrupts.length) {
  const interrupt = response.interrupts[0];
  // Restart re-executes the tool, this time with resumed=true
  response = await ai.generate({
    model: googleAI.model('gemini-flash-latest'),
    messages: response.messages,
    tools: [sendEmail],
    resume: { restart: [sendEmail.restart(interrupt)] },
  });
}

The toolApproval middleware from @genkit-ai/middleware automates this pattern, interrupting any tool not in an approved list:

import { toolApproval } from '@genkit-ai/middleware';
import { restartTool } from 'genkit';

let response = await ai.generate({
  model: googleAI.model('gemini-flash-latest'),
  prompt: 'Send a hello email to [email protected]',
  tools: [sendEmail, readInbox],
  use: [toolApproval({ approved: ['readInbox'] })], // sendEmail not approved
});

// sendEmail was interrupted - get user approval, then restart
if (response.interrupts.length) {
  response = await ai.generate({
    model: googleAI.model('gemini-flash-latest'),
    messages: response.messages,
    tools: [sendEmail, readInbox],
    resume: {
      restart: response.interrupts.map((i) =>
        restartTool(i, { toolApproved: true })
      ),
    },
    use: [toolApproval({ approved: ['readInbox'] })],
  });
}

Prompts (Dotprompt)

Manage prompts as code with embedded schemas, model configuration, and Handlebars templating:

---
model: googleai/gemini-flash-latest
input:
  schema:
    topic: string
output:
  schema:
    title: string
    summary: string
---
Write a blog post about {{topic}}.

const blogPrompt = ai.prompt('blog');
const { output } = await blogPrompt({ topic: 'AI safety' });

Flows

Build strongly typed, fully observable workflows that can be served as APIs and accessed from the client:

import { genkit, z } from 'genkit';
import { googleAI } from '@genkit-ai/google-genai';

const ai = genkit({
  plugins: [googleAI()],
  model: googleAI.model('gemini-flash-latest'),
});

const RecipeSchema = z.object({
  title: z.string(),
  ingredients: z.array(z.string()),
  instructions: z.array(z.string()),
});

export const recipeFlow = ai.defineFlow(
  {
    name: 'recipeFlow',
    inputSchema: z.object({ ingredient: z.string() }),
    outputSchema: RecipeSchema,
  },
  async (input) => {
    const { output } = await ai.generate({
      prompt: `Create a recipe using ${input.ingredient}`,
      output: { schema: RecipeSchema },
    });
    if (!output) throw new Error('Failed to generate recipe');
    return output;
  }
);

Serve flows as an API:

import { startFlowServer } from '@genkit-ai/express'; // npm i @genkit-ai/express

startFlowServer({ flows: [recipeFlow] });

Access from the client:

import { streamFlow } from 'genkit/beta/client';

const { stream } = streamFlow({
  url: 'http://localhost:3500/recipeFlow',
  input: { ingredient: 'avocado' },
});

for await (const chunk of stream) {
  console.log(chunk);
}

Middleware

The @genkit-ai/middleware package provides ready-made middleware to add common functionality to your AI requests:

• retry - Automatically retry failed requests with exponential backoff.
• fallback - Fall back to alternative models on specific error statuses.
• toolApproval - Restrict tool execution to an approved list, interrupting unapproved calls for review.
• filesystem - Give the model sandboxed read/write access to a directory on the filesystem.
• skills - Scan for skill definitions and inject them as available tools.

npm install @genkit-ai/middleware

import { retry } from '@genkit-ai/middleware';

const { text } = await ai.generate({
  model: googleAI.model('gemini-flash-latest'),
  prompt: 'Why is Genkit awesome?',
  use: [
    retry({
      maxRetries: 3,
      initialDelayMs: 1000,
      backoffFactor: 2,
    }),
  ],
});

Developer Tools

Genkit comes with a powerful CLI and Developer UI for locally testing, debugging, and iterating on your AI features:

npx genkit start -- npx tsx src/index.ts

The Developer UI lets you visually test flows, inspect traces, and experiment with prompts - all in your browser.

Plugins

Genkit supports a growing ecosystem of plugins for model providers, vector stores, and more:

| Category | Plugins | |---|---| | Models | @genkit-ai/google-genai, @genkit-ai/vertexai, @genkit-ai/compat-oai, genkitx-anthropic, genkitx-ollama | | Deployment | @genkit-ai/express, @genkit-ai/fetch, @genkit-ai/firebase, @genkit-ai/cloud-run | | Monitoring | @genkit-ai/google-cloud |

Browse all plugins: npmjs.com/search?q=keywords:genkit-plugin

Deployment

Genkit flows can be deployed anywhere Node.js runs:

• Express - Deploy to Node.js
• Firebase - Deploy to Firebase
• Cloud Run - Deploy to Cloud Run

Next Steps

• Developer tools: Set up and use Genkit's CLI and developer UI.
• Generating content: Use Genkit's unified generation API.
• Creating flows: Build observable workflows with rich debugging.
• Managing prompts: Manage prompts and configuration as code.

Learn more at genkit.dev

License: Apache 2.0