@coopah/bentley-langfuse

LangFuse observability integration for Bentley, built on OpenTelemetry. Traces agent invocations, scores runs, manages prompt versions, and runs evaluation experiments.

Uses @langfuse/otel + @langfuse/client (the modern OTel-based API), not the deprecated v3 Langfuse class.

Install

pnpm add @coopah/bentley-langfuse

Dependencies

• @coopah/bentley-core
• @langfuse/client ^4.6.1
• @langfuse/otel ^4.6.1
• @opentelemetry/api ^1.9.0
• @opentelemetry/sdk-node ^0.213.0

Usage

import { initBentleyTracing, shutdownBentleyTracing } from "@coopah/bentley-langfuse";

// Initialize before creating your Bentley instance
initBentleyTracing({
  publicKey: process.env.LANGFUSE_PUBLIC_KEY!,
  secretKey: process.env.LANGFUSE_SECRET_KEY!,
  baseUrl: process.env.LANGFUSE_BASE_URL, // optional, defaults to LangFuse cloud
});

// ... use bentley ...

// On shutdown
await shutdownBentleyTracing();

Tracing Middleware

import { bentleyTracingMiddleware } from "@coopah/bentley-langfuse";

// Add to your shell middleware pipeline for automatic span collection
const bentley = createBentley({
  plugins: [/* ... */],
  middleware: [bentleyTracingMiddleware],
});

Traced Functions

import { bentleyTraced } from "@coopah/bentley-langfuse";

const myFunction = bentleyTraced(async () => {
  // This function call is wrapped in a trace span
  return await doExpensiveWork();
});

API

Setup

• initBentleyTracing(config: BentleyLangfuseConfig) — Initialize OTel + LangFuse
• shutdownBentleyTracing() — Flush pending spans and shut down
• getSpanProcessor() — Access the raw OTel span processor

Tracing

• bentleyTracingMiddleware — Shell middleware for automatic tracing
• bentleyTraced(fn) — Wrap any async function with a trace span

Prompts

• BentleyPromptManager — Manage and compile versioned LangFuse prompts

Scoring & Cost

• BentleyScoringService — Score agent runs programmatically
• bentleyAutoScoringMiddleware — Automatic scoring middleware
• calculateCost(model, usage) — Token cost calculation
• MODEL_COSTS — Cost-per-token constants for supported models

Experiments

• BentleyExperimentRunner — Run A/B evaluation experiments
• BentleyExperimentEvaluator — Custom evaluator interface

Configuration

interface BentleyLangfuseConfig {
  publicKey: string;
  secretKey: string;
  baseUrl?: string;       // LangFuse endpoint
  environment?: string;
  exportMode?: "batched" | "immediate";
}

Related Packages

| Package | Role | |---------|------| | @coopah/bentley-core | Core runtime (required) | | @coopah/bentley-server | Server with built-in LangFuse support |

License

MIT