@runtime-judgement/openai-agents

OpenAI Agents SDK bridge for Runtime Judgement — register one trace processor and every agent run is automatically captured for failure attribution.

Installation

npm install @runtime-judgement/openai-agents
# or
pnpm add @runtime-judgement/openai-agents

Requires OpenAI SDK v4+ (openai >= 4.0.0).

Quick start

npm install @runtime-judgement/openai-agents

import { rjInit } from "@runtime-judgement/openai-agents"

// At app startup — one line to enable automatic trace attribution
rjInit({ apiKey: process.env.RJ_API_KEY })

Set RJ_API_KEY=rj_live_... (get your key from https://runtime-judgement.app/app/settings/api-keys).

Every agent run is now automatically ingested. Failures surface in your failure clusters within ~30 seconds.

Pattern 1 — Trace processor (recommended)

Register once at app startup. Every agent run is captured automatically.

import { RJTraceProcessor } from "@runtime-judgement/openai-agents"
import { registerTraceProcessor } from "openai/agents"   // or @openai/agents

registerTraceProcessor(new RJTraceProcessor({
  apiKey: process.env.RJ_API_KEY,   // Required: your RJ API key
  suiteId: "production-suite",      // Optional: tag all traces to a snapshot suite
}))

Your agent code is unchanged — the processor hooks into the OpenAI Agents SDK tracing pipeline automatically.

Pattern 2 — Manual span (fallback)

For any OpenAI call that doesn't go through the Agents SDK pipeline:

import { rjSpan } from "@runtime-judgement/openai-agents"

const response = await rjSpan("product-search", async () => {
  return await openai.chat.completions.create({
    model: "gpt-4o",
    messages: [{ role: "user", content: "Find blue running shoes under $100" }],
  })
}, { apiKey: process.env.RJ_API_KEY })

The return value is passed through unchanged. If the function throws, the error is recorded in the span and re-thrown.

Configuration

| Option | Type | Default | Description | |---|---|---|---| | apiKey | string | process.env.RJ_API_KEY | Your Runtime Judgement API key. Generate one at /app/settings/api-keys. | | suiteId | string | — | Optional snapshot suite ID to tag all traces with. | | endpoint | string | https://runtime-judgement.app/api/ingest/otlp/v1/traces | OTLP ingest endpoint. Override for self-hosted deployments. | | disabled | boolean | false | When true, no spans are sent. Useful in test environments. | | capturePrompts | boolean | false | When true, include prompt/input content in spans. Off by default — see Privacy note below. |

What data is captured

For each agent span, the processor captures:

• gen_ai.system — always "openai"
• gen_ai.request.model — the model name (e.g. "gpt-4o")
• gen_ai.usage.prompt_tokens — input token count
• gen_ai.usage.completion_tokens — output token count
• gen_ai.response.finish_reasons — "stop", "tool_calls", or "error"
• gen_ai.output.value — first 2,000 characters of the agent's output
• rj.span_kind — "agent_run", "tool_call", or "llm_call"
• error.message — error text when the span failed

What is NOT captured by default

• Full prompt content — prompts may contain PII, secrets, or confidential user data. Prompts are never sent unless you explicitly set capturePrompts: true.
• Raw HTTP request/response bodies — only the structured span attributes above are forwarded.
• API keys or credentials — the bridge never reads or logs your OpenAI key.

Privacy note

All data is sent over HTTPS to your Runtime Judgement account. Only you and your team can access it. The 2,000-character output truncation limit is a hard cap — long completions are always truncated before transmission.

If capturePrompts: false (the default), the span contains no user-supplied text beyond the output truncated to 2,000 chars. This is the safe default for production deployments handling user data.

TypeScript support

Full TypeScript support. All public types are exported from the package root:

import type {
  RJConfig,
  AgentTrace,
  AgentSpan,
  RjSpanKind,
} from "@runtime-judgement/openai-agents"

Compatibility

• OpenAI SDK v4+
• Node.js 18+
• Works with both the openai/agents sub-path and the @openai/agents standalone package

Self-hosted / custom endpoint

Point the processor at your own RJ instance:

new RJTraceProcessor({
  apiKey: process.env.RJ_API_KEY,
  endpoint: "https://your-rj-instance.example.com/api/ingest/otlp/v1/traces",
})

License

MIT