@mate-academy/prompt-client

Provider-agnostic LLM prompt management and tracing client. Consumers code against two stable interfaces — LLMPromptClient (fetch + compile prompts, caching, fallbacks) and LLMTracer (traces + generations with usage/cost details) — and pick a provider from PromptManagementProviders. The provider can be swapped without touching call sites.

Providers:

• Langfuse — the real provider. One shared Langfuse SDK client backs both the prompt client and the tracer.
• InMemory — a deterministic test double: seedable prompts and a recording tracer for assertions. Use it in unit/integration tests instead of hand-rolled mocks.

Install

npm install @mate-academy/prompt-client

Quick start

import {
  createPromptManagement,
  PromptManagementProviders,
} from '@mate-academy/prompt-client';

const promptManagement = createPromptManagement({
  provider: PromptManagementProviders.Langfuse,
  options: {
    credentials: {
      publicKey: process.env.LANGFUSE_PUBLIC_KEY ?? '',
      secretKey: process.env.LANGFUSE_SECRET_KEY ?? '',
      baseUrl: process.env.LANGFUSE_BASE_URL ?? '',
    },
  },
  logger,
});

const prompt = await promptManagement.promptClient.getPrompt('my-prompt');
const instructions = prompt.compile({ leadName: 'Maria' });

createPromptManagement returns a PromptManagementBundle:

| Member | Purpose | |---|---| | promptClient | LLMPromptClient — getPrompt(name, options?), shutdown() | | tracer | LLMTracer — createTrace, createGeneration, flush, shutdown | | flush() | Flush pending trace/generation events without shutting down | | shutdown() | Flush and stop the underlying client (whole bundle) |

promptClient.shutdown() and tracer.shutdown() stop the same shared client; prefer bundle.shutdown().

Prompts

const prompt = await promptClient.getPrompt('chatAgent.instructions', {
  label: 'production',      // default; mutually exclusive with version
  version: 4,               // pin an exact version (label is then omitted)
  cacheTtlSeconds: 60,      // default 60s, SDK-side cache + background refresh
  fallback: FALLBACK_TEXT,  // never throw: return this text on any failure
});

prompt.name;        // 'chatAgent.instructions'
prompt.version;     // Langfuse version, or 0 when the fallback was used
prompt.isFallback;  // true when the fallback was served
prompt.config;      // the config object stored on the Langfuse prompt
prompt.compile({ leadName: 'Maria' }); // mustache-style {{var}} substitution

Error model (only when NO fallback is provided):

• LLMPromptNotFoundError — the prompt does not exist (safe to use as an existence probe).
• LLMPromptFetchError — infrastructure failure (network/HTTP) or a non-text prompt. Both extend LLMPromptError and carry promptName + cause.

With fallback set, getPrompt never rejects: on any failure it resolves to the fallback text with version: 0 and isFallback: true, and the logger receives a warning.

Tracing

const trace = tracer.createTrace({
  name: 'conversation-turn',
  sessionId: chatId,      // groups turns of one conversation in Langfuse
  userId,
  tags: ['sdr'],
  input: inboundMessage,
});

const generation = tracer.createGeneration(trace, {
  name: 'chat-completion',
  model: modelName,
  input: messages,
  prompt,                 // links the generation to the Langfuse prompt version
});

generation.end({
  output: completion,
  usageDetails: usageToUsageDetails(response.usage),
  costDetails: costToCostDetails(response.cost),
});

trace.update({ output: decision });

usageToUsageDetails / costToCostDetails convert @mate-academy/llm-gateway usage/cost results into the shape Langfuse expects; their input types are structural, so no llm-gateway dependency is required.

Usage in a long-lived server (api)

A process can talk to several Langfuse projects. A project is a runtime argument, never baked into the client: build one bundle per project (one shared client each), memoize them behind a registry keyed by your own project enum, and pass the project at the call site. Flush + stop every live bundle on SIGTERM.

const bundles = new Map<LangfuseProject, PromptManagementBundle>();

const getPromptManagement = (
  project: LangfuseProject,
): PromptManagementBundle => {
  const existing = bundles.get(project);

  if (existing) {
    return existing;
  }

  const bundle = createPromptManagement({
    provider: PromptManagementProviders.Langfuse,
    options: { credentials: appConfig.langfuse.projects[project] },
    logger: rootLogger.child(`PromptManagement:${project}`),
  });

  bundles.set(project, bundle);

  return bundle;
};

// at a call site — pick the project you need:
const { promptClient, tracer } = getPromptManagement(LangfuseProject.SalesQA);

// in graceful shutdown:
await Promise.all([...bundles.values()].map((bundle) => bundle.shutdown()));

Memoizing per project is correctness, not caching: each bundle owns a live SDK client with a background flush timer and buffered events, so the registry's Map is the client's lifetime — one client per project per process.

Typed error handling keeps existing semantics: catch LLMPromptNotFoundError for "is this conversation scorable?" probes and treat LLMPromptFetchError as an infrastructure alert.

Usage in a Lambda

Create the bundle at module scope so it stays warm across invocations, use per-prompt fallbacks so a Langfuse outage can never break a turn, and flush at the end of every invocation — buffered trace events are lost when the sandbox freezes:

const promptManagement = createPromptManagement({
  provider: PromptManagementProviders.Langfuse,
  options: {
    credentials: {
      publicKey: appConfig.langfusePublicKey,
      secretKey: appConfig.langfuseSecretKey,
      baseUrl: appConfig.langfuseBaseUrl,
    },
    flushAt: 1, // send events immediately; Lambdas have no idle time to batch
  },
  logger,
});

export const handler = async (event: SQSEvent): Promise<void> => {
  try {
    const prompt = await promptManagement.promptClient.getPrompt(
      PromptKey.ChatAgentInstructions,
      { fallback: FALLBACKS[PromptKey.ChatAgentInstructions] },
    );
    // ... run the turn, create trace/generations with sessionId: chatId
  } finally {
    await promptManagement.flush();
  }
};

Testing consumers with the InMemory provider

const promptManagement = createPromptManagement({
  provider: PromptManagementProviders.InMemory,
  options: {
    prompts: {
      'chatAgent.instructions': { prompt: 'Reply to {{leadName}}', version: 3 },
    },
    // GenerateStub: unknown names resolve to `Mock prompt for <name>`
    missingPromptBehavior: InMemoryMissingPromptBehaviors.GenerateStub,
  },
});

// The InMemory bundle is typed with the concrete classes:
promptManagement.promptClient.setPrompt('closing', { prompt: 'Bye!' });
promptManagement.tracer.traces;          // recorded traces + their updates
promptManagement.tracer.generations;     // recorded generations + end payloads
promptManagement.tracer.flushCallCount;  // flush()/shutdown() call counters
promptManagement.tracer.reset();

Provider notes (Langfuse)

• label and version are mutually exclusive; when version is set the label (including the production default) is omitted automatically.
• Fallback prompts have version: 0 and isFallback: true (SDK semantics).
• Only text prompts are supported; a chat prompt is rejected with LLMPromptFetchError. Chat support would arrive as an additive getChatPrompt method.
• Pinned to the langfuse v3 SDK. Its only dynamic imports target Node built-ins (fs, crypto) and are marked webpackIgnore, so the package is safe to bundle with webpack/serverless-bundle for Lambdas.
• No SDK types leak through the public API; a future SDK swap stays inside this package.

Development

npm run lint
npm run type-check
npm test                  # unit tests, no network
npm run test:integration  # requires .env.test (see .env.test.example)
npm run build

Publishing

npm run patch   # or minor / major / canary

First-ever publish of this scoped package must pass --access public: npm publish --access public.