OpenAI Agents SDK Adapter

A comprehensive Agent Harness adapter for testing multi-agent workflows using the OpenAI Agents SDK.

Features

• Multi-agent workflows - Create and test complex agent interactions
• Tool execution - Mock, template, and real tool implementations
• Agent handoffs - Test delegation between agents
• Guardrails - Input/output validation and content filtering
• Lifecycle events - Track agent execution and tool calls
• Structured outputs - Enforce JSON schema with Zod validation
• MCP server support - Integrate Model Context Protocol servers
• Streaming - Real-time response streaming

Installation

npm install @wavespec/adapter-openai-agents @openai/agents openai

Configuration

Set your OpenAI API key:

export OPENAI_API_KEY=your-api-key

Create a test suite YAML file:

suite:
  name: "My Agent Test"
  adapter: "openai-agents"

tests:
  - name: "Create a helpful assistant"
    operation: create_agent
    params:
      name: "Assistant"
      instructions: "You are a helpful assistant"
      model: "gpt-4.1"
    assertions:
      - type: success

Supported Operations

create_agent

Create an agent with instructions, tools, and settings:

- operation: create_agent
  params:
    name: "WeatherBot"
    instructions: "You provide weather information"
    model: "gpt-4.1"
    tools:
      - name: "get_weather"
        description: "Get weather for a location"
        parameters:
          location: { type: "string", description: "City name" }
        mode: "mock"  # or "template" or "module"

run_agent

Execute an agent with a message:

- operation: run_agent
  params:
    agent: "{{agent_id}}"
    message: "What's the weather in San Francisco?"
    stream: false

create_tool

Create a reusable tool:

- operation: create_tool
  params:
    name: "calculate"
    description: "Perform calculations"
    parameters:
      expression: { type: "string" }
    mode: "template"
    template: "Result: {{args.expression}}"

Tool Execution Modes

• mock - Returns success without execution (for testing)
• template - Safe string interpolation with {{args.field}} syntax
• module - Load and execute external JavaScript/TypeScript functions

Model Settings

Configure model behavior:

- operation: create_agent
  params:
    name: "Agent"
    instructions: "..."
    model: "gpt-4.1"
    modelSettings:
      temperature: 0.7      # 0-2, lower = more deterministic
      topP: 0.9            # 0-1, nucleus sampling
      maxTokens: 1000      # Maximum response length
      frequencyPenalty: 0  # -2 to 2, reduce repetition
      presencePenalty: 0   # -2 to 2, encourage diversity

Agent Handoffs

Delegate between agents:

- operation: create_agent
  params:
    name: "Receptionist"
    instructions: "Triage user requests"
    handoffSettings:
      handoffTo:
        - agent: "{{specialist_id}}"
          description: "Technical questions"

Guardrails

Add input/output validation:

- operation: create_agent
  params:
    name: "Agent"
    instructions: "..."
    guardrails:
      input:
        - type: "content_filter"
          policy: "block_harmful"
      output:
        - type: "format_validator"
          schema: { type: "object" }

Lifecycle Events

Track execution events:

- operation: run_agent
  params:
    agent: "{{agent_id}}"
    message: "Hello"
    captureLifecycleEvents: true

Events include:

• Agent creation
• Tool invocations
• Handoffs
• Errors

Built-in Tools

Use OpenAI's built-in tools:

- operation: create_agent
  params:
    name: "ResearchBot"
    instructions: "Research topics"
    builtinTools:
      - type: "web_search"
      - type: "file_search"

Available built-in tools:

• web_search - Search the web
• file_search - Search uploaded files
• code_interpreter - Execute Python code
• image_generation - Generate images

Structured Outputs

Enforce response format with Zod schemas:

- operation: create_agent
  params:
    name: "DataExtractor"
    instructions: "Extract structured data"
    outputType:
      schema:
        type: "object"
        properties:
          name: { type: "string" }
          age: { type: "number" }
        required: ["name", "age"]

Example: Multi-Agent Workflow

suite:
  name: "Customer Service Workflow"
  adapter: "openai-agents"

tests:
  - name: "Create receptionist agent"
    operation: create_agent
    params:
      name: "Receptionist"
      instructions: "Triage customer requests"
      model: "gpt-4.1"
    store:
      receptionist_id: "response.raw.agentId"

  - name: "Create specialist agent"
    operation: create_agent
    params:
      name: "TechSupport"
      instructions: "Resolve technical issues"
      model: "gpt-4.1"
    store:
      specialist_id: "response.raw.agentId"

  - name: "Configure handoff"
    operation: create_agent
    params:
      name: "Receptionist"
      instructions: "Triage and delegate"
      handoffSettings:
        handoffTo:
          - agent: "{{specialist_id}}"
            description: "Technical issues"
    store:
      receptionist_id: "response.raw.agentId"

  - name: "Test workflow"
    operation: run_agent
    params:
      agent: "{{receptionist_id}}"
      message: "My wifi isn't working"
    assertions:
      - type: success
      - type: field_exists
        field: "response.lifecycle.handoffs"

Templates

The adapter includes 19 example templates covering:

• Basic agent creation
• Tool execution (mock, template, module modes)
• Agent handoffs and delegation
• Lifecycle event tracking
• Guardrails and validation
• Model settings and configuration
• Structured outputs
• Built-in tools
• Full multi-agent workflows

Run templates with:

harness run templates/basic.yaml
harness run templates/with-handoffs.yaml
harness run templates/full-workflow.yaml

Development

Build the adapter:

npm run build

Run tests:

npm test

Type check:

npm run type-check

Requirements

• Node.js 18+
• OpenAI API key
• @openai/agents ^0.2.1
• openai ^4.0.0

License

MIT

Documentation

For complete documentation, see the Agent Harness documentation.

For OpenAI Agents SDK documentation, see @openai/agents.