@mate-academy/prompt-client

Provider-agnostic LLM prompt management client. Consumers code against one stable interface — LLMPromptClient (fetch + compile text and chat prompts, caching, fallbacks, a codegen-facing prompt catalog) — and pick a provider from PromptManagementProviders. The provider can be swapped without touching call sites.

Providers:

• Langfuse — the real provider, built on @langfuse/client v5.
• InMemory — a deterministic test double: seedable text and chat prompts. Use it in unit/integration tests instead of hand-rolled mocks.

Tracing moved out of this package. @mate-academy/llm-tracer now hosts the Langfuse tracer surface (traces, generations, usage/cost details); this package stays prompts-only.

Install

npm install @mate-academy/prompt-client

Migrating from 1.x

2.0.0 is a breaking release: the entire tracer surface was removed from this package and moved to @mate-academy/llm-tracer. Update imports as follows:

| Removed from prompt-client | New home | |---|---| | LLMTracer, LLMTrace, LLMGeneration | @mate-academy/llm-tracer | | LLMTraceOptions, LLMTraceUpdateOptions | @mate-academy/llm-tracer | | LLMGenerationOptions, LLMGenerationEndOptions | @mate-academy/llm-tracer | | LLMTracerError | @mate-academy/llm-tracer | | InMemoryTracer, InMemoryRecordedTrace, InMemoryRecordedGeneration | @mate-academy/llm-tracer | | usageToUsageDetails, costToCostDetails, LLMUsageInput, LLMCostInput | @mate-academy/llm-tracer | | PromptManagementBundle.tracer / .flush() | @mate-academy/llm-tracer's own bundle | | LangfuseProviderOptions.flushAt / .flushIntervalMs | @mate-academy/llm-tracer's own provider options |

getPrompt(name, options?) itself is unchanged and byte-identical to 1.x — only the surrounding bundle shape and the SDK underneath changed. Existing 1.x consumers (api/src/gateways/PromptManagement/, api/src/modules/salesQa/, serverless/services/sdrChatbot) are unaffected and keep running on the published 1.x line until they migrate.

Quick start

import {
  createPromptManagement,
  PromptManagementProviders,
} from '@mate-academy/prompt-client';

const promptManagement = createPromptManagement({
  provider: PromptManagementProviders.Langfuse,
  options: {
    credentials: {
      publicKey: process.env.LANGFUSE_PUBLIC_KEY ?? '',
      secretKey: process.env.LANGFUSE_SECRET_KEY ?? '',
      baseUrl: process.env.LANGFUSE_BASE_URL ?? '',
    },
  },
  logger,
});

const prompt = await promptManagement.promptClient.getPrompt('my-prompt');
const instructions = prompt.compile({ leadName: 'Maria' });

createPromptManagement returns a PromptManagementBundle:

| Member | Purpose | |---|---| | promptClient | LLMPromptClient — getPrompt, getChatPrompt, listPrompts, getPromptRecord, shutdown() | | shutdown() | Stop the underlying provider client |

There is no flush() on this bundle — a prompts-only client buffers nothing. promptClient.shutdown() and bundle.shutdown() stop the same shared client; prefer bundle.shutdown().

Prompts

const prompt = await promptClient.getPrompt('chatAgent.instructions', {
  label: 'production',      // default; mutually exclusive with version
  version: 4,               // pin an exact version (label is then omitted)
  cacheTtlSeconds: 60,      // default 60s, SDK-side cache + background refresh
  fallback: FALLBACK_TEXT,  // never throw: return this text on any failure
});

prompt.name;        // 'chatAgent.instructions'
prompt.version;     // Langfuse version, or 0 when the fallback was used
prompt.isFallback;  // true when the fallback was served
prompt.config;      // the config object stored on the Langfuse prompt
prompt.compile({ leadName: 'Maria' }); // mustache-style {{var}} substitution

Error model (only when NO fallback is provided):

• LLMPromptNotFoundError — the prompt does not exist (safe to use as an existence probe).
• LLMPromptFetchError — infrastructure failure (network/HTTP) or a wrong-shape prompt (a chat prompt fetched via getPrompt, or vice versa). Both extend LLMPromptError and carry promptName. cause is set when an underlying provider error is available.

LLMPromptListError (thrown only by listPrompts, see Catalog) sits outside this hierarchy on purpose: it is selection-scoped, not prompt-scoped, so it extends Error directly and carries the requested label + tag plus cause instead of promptName. A broad catch (error) { if (error instanceof LLMPromptError) ... } will not catch it — handle it explicitly, or match on error.name === 'LLMPromptListError'.

With fallback set, getPrompt never rejects: it resolves to the fallback text with version: 0 and isFallback: true, and logs the fallback resolution. If the SDK throws instead of returning its fallback, the client also logs the underlying fetch error before building the same fallback locally.

Chat prompts

getChatPrompt is the additive counterpart of getPrompt for role-tagged message prompts:

const prompt = await promptClient.getChatPrompt('chatAgent.conversation', {
  label: 'production',
  fallback: [
    { role: LLMPromptMessageRoles.System, content: 'You are a helpful agent.' },
  ],
});

prompt.messages;    // LLMPromptMessage[] — { role, content }
prompt.compile({ leadName: 'Maria' }); // substitutes {{var}} in each message's content

Fallback is an LLMPromptMessage[] instead of a string, with the same never-throw / version: 0 / isFallback: true semantics as getPrompt. Langfuse placeholder entries and messages with an unrecognized role are filtered when the prompt is fetched. A warning is logged once at fetch time so the gap is visible without breaking the call; messages and compile() then use the filtered result.

Catalog (for codegen)

listPrompts and getPromptRecord expose the raw prompt catalog for Langfuse-prompt codegen (the gateway's generateSnapshot):

const { promptNames, page, totalPages } = await promptClient.listPrompts({
  label: 'production',
  tag: 'catalog', // optional; omit to list every prompt carrying the label
  page: 1,      // default 1
  pageSize: 100, // default 100
});

const record = await promptClient.getPromptRecord('chatAgent.instructions', {
  label: 'production',
});
// record.type === LLMPromptTypes.Text | LLMPromptTypes.Chat

label is required and tag narrows the same listing further — a prompt is returned only when it carries both. The filter is applied by the provider, never by widening the query and post-filtering names, so an unmatched tag yields an empty promptNames page rather than the full label listing. Pagination applies to the filtered set.

Both methods are uncached and never fall back — they throw on failure (LLMPromptListError for listPrompts, LLMPromptNotFoundError / LLMPromptFetchError for getPromptRecord) so a codegen run can fail loudly in CI and fall back to the last-committed snapshot artifact instead of silently baking in stale or partial data.

Usage in a long-lived server (api)

A process can talk to several Langfuse projects. A project is a runtime argument, never baked into the client: build one bundle per project (one shared client each), memoize them behind a registry keyed by your own project enum, and pass the project at the call site. Stop every live bundle on SIGTERM.

const bundles = new Map<LangfuseProject, PromptManagementBundle>();

const getPromptManagement = (
  project: LangfuseProject,
): PromptManagementBundle => {
  const existing = bundles.get(project);

  if (existing) {
    return existing;
  }

  const bundle = createPromptManagement({
    provider: PromptManagementProviders.Langfuse,
    options: { credentials: appConfig.langfuse.projects[project] },
    logger: rootLogger.child(`PromptManagement:${project}`),
  });

  bundles.set(project, bundle);

  return bundle;
};

// at a call site — pick the project you need:
const { promptClient } = getPromptManagement(LangfuseProject.SalesQA);

// in graceful shutdown:
await Promise.all([...bundles.values()].map((bundle) => bundle.shutdown()));

Memoizing per project is correctness, not caching: each bundle owns a live SDK client, so the registry's Map is the client's lifetime — one client per project per process.

Typed error handling keeps existing semantics: catch LLMPromptNotFoundError for "is this conversation scorable?" probes and treat LLMPromptFetchError as an infrastructure alert.

Usage in a Lambda

Create the bundle at module scope so it stays warm across invocations, and use per-prompt fallbacks so a Langfuse outage can never break a turn:

const promptManagement = createPromptManagement({
  provider: PromptManagementProviders.Langfuse,
  options: {
    credentials: {
      publicKey: appConfig.langfusePublicKey,
      secretKey: appConfig.langfuseSecretKey,
      baseUrl: appConfig.langfuseBaseUrl,
    },
  },
  logger,
});

export const handler = async (event: SQSEvent): Promise<void> => {
  const prompt = await promptManagement.promptClient.getPrompt(
    PromptKey.ChatAgentInstructions,
    { fallback: FALLBACKS[PromptKey.ChatAgentInstructions] },
  );
  // ... run the turn
};

Testing consumers with the InMemory provider

const promptManagement = createPromptManagement({
  provider: PromptManagementProviders.InMemory,
  options: {
    prompts: {
      'chatAgent.instructions': { prompt: 'Reply to {{leadName}}', version: 3 },
    },
    chatPrompts: {
      'chatAgent.conversation': {
        messages: [{ role: LLMPromptMessageRoles.System, content: 'Reply to {{leadName}}' }],
      },
    },
    // GenerateStub: unknown names resolve to `Mock prompt for <name>`
    missingPromptBehavior: InMemoryMissingPromptBehaviors.GenerateStub,
  },
});

// The InMemory bundle is typed with the concrete class:
promptManagement.promptClient.setPrompt('closing', { prompt: 'Bye!' });
promptManagement.promptClient.setChatPrompt('closing.chat', {
  messages: [{ role: LLMPromptMessageRoles.System, content: 'Bye!' }],
});

listPrompts / getPromptRecord on the InMemory client read only the seeded prompts (labels default to ['production']); missingPromptBehavior never applies to getPromptRecord — a missing seed always throws LLMPromptNotFoundError, since the catalog must report seeded truth for codegen tests.

Seeds accept an optional tags array so listPrompts({ label, tag }) filters the same way it does against Langfuse. Tags have no default: a seed without tags matches every tag-less listing and no tag-filtered one.

Provider notes (Langfuse)

• label and version are mutually exclusive; when version is set the label (including the production default) is omitted automatically.
• Fallback prompts have version: 0 and isFallback: true (SDK semantics).
• Built on @langfuse/client v5. Error classification is structural, never instanceof: v5's Fern-generated errors never assign a distinctive error.name (every one reports 'Error' at runtime), so classification duck-types on error.statusCode — 404 maps to LLMPromptNotFoundError, everything else to LLMPromptFetchError. This survives a duplicate SDK copy in a webpack/serverless bundle, where instanceof would silently fail.
• Behavior change vs 1.x: a non-404 HTTP error (5xx, 401, 403) on the prompt endpoint now correctly classifies as LLMPromptFetchError instead of falling into the default LLMPromptNotFoundError bucket.
• The SDK's type: 'text' | 'chat' option on prompt.get is a compile-time overload discriminator only — the runtime still returns whatever prompt the server has. getPrompt therefore still rejects a chat prompt (and getChatPrompt a text prompt) with LLMPromptFetchError.
• No SDK types leak through the public API; a future SDK swap stays inside this package.

Development

npm run lint
npm run type-check
npm test                  # unit tests, no network
npm run test:integration  # requires .env.test (see .env.test.example)
npm run build

Publishing

npm run patch   # or minor / major / canary

First-ever publish of this scoped package must pass --access public: npm publish --access public.