typefuse

Typed, multi-provider prompt client for Langfuse prompt management.

Your prompts, models, and parameters live in Langfuse; typefuse executes them through the Vercel AI SDK and generates a fully typed TypeScript client from your prompt library — variables and JSON schemas become interfaces, so calling a prompt is as type-safe as calling a function.

const { summary } = await prompts.summarizeArticle({ input });
//      ^ typed from the prompt's json_schema   ^ typed from {{input}} in the prompt

• Config-driven execution — model, temperature, structured output schema, timeouts, and more are read from each Langfuse prompt's config, so you change them in the Langfuse console without deploying.
• Multi-provider — provider/model ids route to OpenAI, Anthropic, Google, or Vertex (each an optional peer dependency; bring your own via providers).
• Model fallback — set fallback_model on a prompt and transient provider failures (quota, 5xx, timeouts) retry once on a different provider instead of failing the request.
• Tracing built in — every execution creates or joins a Langfuse trace with the compiled messages, model parameters, token usage, and the model that actually answered. Usage is reported as Langfuse usage details with cached tokens split into cache_read_input_tokens / cache_creation_input_tokens, so cache hit rates are visible and cost inference can apply discounted cache prices.
• Template logic — Langfuse variables are flat; typefuse adds Mustache sections ({{#flag}}…{{/flag}}) on top, typed as booleans in the generated client.
• Codegen — a CLI that turns your Langfuse prompt library into a committed, reviewable, typed client file.

Install

npm install typefuse ai langfuse
# plus the providers you use:
npm install @ai-sdk/openai @ai-sdk/google-vertex

Setup

Configuration is entirely env-var driven — no init code needed:

| Variable | Effect | Default | | --- | --- | --- | | TYPEFUSE_LABEL | Prompt fetch label (production, latest, …) | production | | TYPEFUSE_ENVIRONMENT | Stamped on traces' environment | Langfuse default | | TYPEFUSE_DEFAULT_MODEL | Used when a prompt config has no model | openai/gpt-5-mini | | TYPEFUSE_TRACING | false/0 disables always-on tracing | true | | TYPEFUSE_SECRET_KEY | Langfuse secret key | LANGFUSE_SECRET_KEY | | TYPEFUSE_PUBLIC_KEY | Langfuse public key | LANGFUSE_PUBLIC_KEY | | TYPEFUSE_BASE_URL | Langfuse host | LANGFUSE_BASE_URL, then Langfuse Cloud |

Provider API keys use each SDK's conventions: OPENAI_API_KEY, ANTHROPIC_API_KEY, GOOGLE_API_KEY/GEMINI_API_KEY, GOOGLE_VERTEX_API_KEY.

The label defaults to production on purpose: a missing env var can never point production at a moving label like latest. Set TYPEFUSE_LABEL=latest on staging/dev.

Generate the typed client

npx typefuse generate --label latest --out src/lib/langfusePrompts.ts

This fetches every prompt in your Langfuse project at the given label and writes a typed module. Commit the file: schema changes then show up in PR diffs, and builds need no network access or credentials.

// generated: src/lib/langfusePrompts.ts
import { createPromptHandler } from "typefuse";

interface SummarizeArticleVariables {
  input: string;
  includeSources: boolean;
}
interface SummarizeArticleResponse {
  summary: string;
  keyPoints: string[];
}

const summarizeArticleHandler = createPromptHandler<
  SummarizeArticleVariables,
  SummarizeArticleResponse
>("SummarizeArticle");

const prompts = { summarizeArticle: summarizeArticleHandler };
export default prompts;

Use it:

import prompts from "./lib/langfusePrompts";

const { summary } = await prompts.summarizeArticle({ input: "...", includeSources: true });

Handlers also accept optional prior messages and an existing Langfuse trace to nest under:

await prompts.summarizeArticle({ input, includeSources: false }, previousMessages, trace);

Template logic (Mustache)

Langfuse's native templating is flat {{variable}} substitution — no conditionals. typefuse renders any prompt containing a Mustache section tag with mustache.js instead, so prompts can carry optional blocks that the caller switches on and off:

You are a helpful summarizer.

{{#includeSources}}
Cite the source for every key point.
{{/includeSources}}
{{^includeSources}}
Do not include citations.
{{/includeSources}}

Article: {{input}}

Codegen types section variables as boolean and inline variables as string, so the generated handler forces every call site to decide each flag:

await prompts.summarizeArticle({ input, includeSources: true });

Behavior notes:

• Detection is automatic: a prompt is Mustache-rendered when every message parses cleanly and at least one contains a section tag. Sections must open and close within the same chat message (each message renders independently). Malformed templates fall back to Langfuse's native flat compile. Set templating: "mustache" in the prompt config to force Mustache for section-free prompts (fail-fast on malformed ones).
• HTML escaping is disabled — {{name}} and {{{name}}} are equivalent. Prompts are plain text, not HTML.
• Under Mustache, unknown {{variables}} render as empty strings; Langfuse's native compile leaves them as literal text. Keep literal braces out of Mustache-rendered prompts.
• The Langfuse Playground only renders flat variables — prompts using sections show raw tags there. Preview by running the handler (the compiled messages are on the trace).
• A name used both as a section and inline ({{#x}}…{{x}}…{{/x}}) is typed string — truthiness drives the section. Names that aren't valid TS identifiers (include-sources) are emitted as quoted keys.

Need the underlying Langfuse client (e.g. to flushAsync() before shutdown, or to open traces)?

import { getTypefuse } from "typefuse";

await getTypefuse().langfuse.flushAsync();

Prompt config reference

Set these on each prompt's Config in the Langfuse console. All keys are snake_case:

| Field | Type | Effect | | --- | --- | --- | | model | string | provider/model, e.g. openai/gpt-5-mini, vertex/gemini-3.5-flash (required) | | fallback_model | string | Retried once when the primary model call throws | | temperature | number | Sampling temperature | | frequency_penalty | number | Frequency penalty (OpenAI-style models) | | json_schema | object | JSON Schema (or { name, schema }) for structured output via generateObject | | max_seconds_timeout | number | Per-attempt timeout | | reasoning_effort | string | OpenAI reasoning effort | | service_tier | "flex" \| "default" | OpenAI service tier | | thinking_config | object | Gemini thinking config (google/vertex models), e.g. { "thinking_budget": 0 } | | templating | "mustache" | Force Mustache rendering (automatic when the prompt contains a section tag) | | enable_tracing | boolean | Force tracing for this prompt when client-level tracing is off | | provider_options | object | Per-provider overrides, e.g. { "openai": { "reasoning_effort": "high" } } |

Prompts without json_schema resolve to the raw response text.

Provider options

provider_options is a generic passthrough to the AI SDK's providerOptions: the entry matching the executing model's provider (so the fallback model gets its own provider's entry) is applied with keys deep-converted snake_case → camelCase. Any provider option the AI SDK supports works without a typefuse release:

{
  "model": "vertex/gemini-2.5-flash",
  "provider_options": {
    "vertex": { "thinking_config": { "thinking_budget": 1024, "include_thoughts": true } }
  }
}

For Gemini thinking specifically, the top-level thinking_config shorthand does the same and is routed to the google or vertex key automatically based on the model. thinking_budget: 0 disables thinking on models that allow it; Gemini 3 models use thinking_level ("low"/"high") instead of a budget. A provider_options thinking_config wins over the top-level shorthand.

Per-call options

Prompt config covers anything static. For values only the caller knows at request time, pass a 4th argument:

await summarizeArticle({ article }, undefined, trace, {
  providerOptions: { vertex: { cachedContent: handle } },
});

These are merged over the config-derived provider options, per provider, with the caller winning on conflict. Only the executing model's provider entry applies.

Context caching (Vertex/Google)

Passing cachedContent — a handle from caches.create() — changes request assembly, because a context cache carries the system instruction itself:

• the system message is dropped from the request (sending it inline as well is either rejected or silently duplicated)
• if the handle turns out to be expired, deleted, or from another region, the request is retried on the same model without it. It is deliberately not allowed to trigger fallback_model — a lapsed cache must never quietly change which model answers
• the handle is stripped automatically on a genuine fallback attempt, since a handle is bound to its creating model

Pair it with onCacheUnusable so your own bookkeeping stays in step — without it, a dead handle costs every subsequent request a failed attempt until it expires:

{
  providerOptions: { vertex: { cachedContent: handle } },
  onCacheUnusable: () => cacheStore.invalidate(),
}

typefuse does not create or own caches — lifecycle (creation, TTL, invalidation on prompt-version change) stays with the caller. Note that Vertex rejects explicit caches below 4,096 tokens, and that a cache keyed to an old prompt version will keep answering under superseded instructions with nothing erroring, so include the version in whatever key you store it under.

Advanced: explicit client

For anything env vars can't express — pinned prompt versions, custom provider factories, or several clients in one process — create an instance and generate with --import-from:

// src/lib/typefuse.ts
import { createTypefuse } from "typefuse";

const tf = createTypefuse({
  label: "latest",
  pinnedVersions: { SummarizeArticle: 24 },
  providers: { myGateway },
});

export const createPromptHandler = tf.createPromptHandler;

npx typefuse generate --label latest --import-from ./typefuse

Explicit options always win over env vars.

License

MIT