@noetaris/harness-openai

OpenAI adapter for @noetaris/harness.

Overview

@noetaris/harness-openai provides an OpenAI class that implements the LLM and ObserverAware interfaces from @noetaris/harness. It handles translation between the harness message format and the OpenAI chat.completions.create format, and emits telemetry events (token usage, model ID) through an attached Observer.

Installation

pnpm add @noetaris/harness-openai

Peer dependencies:

pnpm add @noetaris/harness @noetaris/harness-types @noetaris/harness-openai-models

@noetaris/harness-openai-models supplies the context-window lookup table used to populate contextWindowSize on responses and usage events.

Requires Node.js ≥ 22.

Usage

import { OpenAI } from '@noetaris/harness-openai'

// The model ID is the required first argument; options are optional.
const llm = new OpenAI('gpt-4o-mini', {
  apiKey: process.env.OPENAI_API_KEY, // defaults to the OPENAI_API_KEY env var
})

// Wire into a harness provider slot
h.provide('model', runtime())

const agent = createAgent(h, { prompts: { system: '...' } })
const run = agent.run(initialState, { model: llm })

API

OpenAI

new OpenAI(model: string, options?: OpenAIOptions)

Implements LLM and ObserverAware. OpenAIOptions accepts apiKey and the generation parameters temperature, maxTokens, topP, and seed.

• invoke(messages, options?) — translates harness Message[] and Tool[] to OpenAI format, calls chat.completions.create(), and maps the response back to an LLMResponse (including tool_calls extraction and a required usage: { inputTokens, outputTokens, contextWindowSize? } field; contextWindowSize is resolved from @noetaris/harness-openai-models).
• bindObserver(observer) — attaches an Observer. Each invoke emits an "llm.request" event ({ modelId, providerName: 'openai' }) before the call and an "llm.response" event ({ tokens: { input, output }, modelId, stopReason, providerName, contextWindowSize? }) from the response usage after it.
• setStepContext(ctx) — sets the StepContext attached to emitted events; called by the harness before each step.

Image input

Image input needs @noetaris/harness-types 0.5.0 or later.

A user message can carry images next to text. Pass blocks instead of a string:

import { readFileSync } from 'node:fs'
import { OpenAI } from '@noetaris/harness-openai'

const llm = new OpenAI('gpt-4o-mini')

const response = await llm.invoke([
  {
    role: 'user',
    content: [
      { type: 'text', text: 'What is in this picture?' },
      { type: 'image', data: readFileSync('photo.png').toString('base64'), mediaType: 'image/png' },
    ],
  },
])

• Accepted types: image/jpeg, image/png, image/gif, image/webp. The match is exact and case-sensitive. OpenAI's vision guide lists PNG, JPEG, WEBP and non-animated GIF.
• Anything else throws before any request is sent, and no llm.request event is emitted: Error: Unsupported image media type "image/bmp" for OpenAI. Supported: image/jpeg, image/png, image/gif, image/webp.
• data is raw base64. The adapter builds the data:<mediaType>;base64,<data> URL itself and sends it as an image_url part, with no detail field. If data already starts with data:, the prefix is not removed and ends up doubled.
• API shape: the adapter calls Chat Completions (image_url parts). OpenAI's vision guide shows the Responses API (input_image), so do not copy its snippet.
• String content is sent as before. An array becomes one part per block, in the same order.
• The adapter does not check that the model can read images. The provider decides.
• Images kept in session state are stored again on every run. See Large media in session state.

Known limitation (fork/join branches): setStepContext stores the given StepContext in an instance field and invoke() reads it back when emitting onEvent. This is safe when one step runs at a time, but not when a single OpenAI instance is shared across concurrent fork branches (the normal setup — one instance provided once via h.provide('llm', ...)): concurrent branches can race on that stored field, and telemetry may be attributed to the wrong branch/step. Not fixed in this release — a correct fix means passing StepContext through the call instead of storing it, which is a larger, separate change. If you use @noetaris/harness's fork/join feature with this adapter, treat per-step LLM telemetry as unreliable for concurrently-running branches.

MockOpenAI

A deterministic test double for use in tests and demos without a real API key.

Related Packages

• @noetaris/harness — core execution engine
• @noetaris/harness-types — shared LLM type contract
• @noetaris/harness-openai-models — OpenAI context-window lookup table
• @noetaris/harness-anthropic — Anthropic Claude adapter
• @noetaris/harness-google — Google Gemini adapter
• @noetaris/harness-otel — OpenTelemetry observer bridge

License

MIT