axl-testing

Testing utilities for Axl agentic workflows. Provides deterministic mocks and assertions for unit testing workflows without hitting real LLM APIs.

Installation

npm install @axlsdk/testing --save-dev

API

MockProvider

Mock LLM provider with multiple response modes:

import { MockProvider } from '@axlsdk/testing';

// Sequence mode — return responses in order
const provider = MockProvider.sequence(['Hello!', 'World!']);

// Echo mode — return the user's prompt back
const provider = MockProvider.echo();

// JSON mode — return structured data
const provider = MockProvider.json({ answer: 42, confidence: 0.95 });

// Replay mode — replay from a recorded file
const provider = MockProvider.replay('./fixtures/conversation.json');

// Function mode — custom response logic
const provider = MockProvider.fn((messages, options) => {
  const lastMessage = messages[messages.length - 1];
  return `You said: ${lastMessage.content}`;
});

MockProvider also supports tool call simulation:

const provider = MockProvider.sequence([
  {
    content: '',
    tool_calls: [{
      id: 'call_1',
      type: 'function',
      function: { name: 'calculator', arguments: '{"expression":"2+2"}' },
    }],
  },
  'The answer is 4.',
]);

MockTool

Wrap a tool to intercept and record calls:

import { MockTool } from '@axlsdk/testing';
import { calculator } from './tools';

const mock = new MockTool(calculator);

// Use in agent
const myAgent = agent({ model: 'test:mock', system: 'Test', tools: [mock] });

// After execution, inspect calls
console.log(mock.calls); // [{ input: { expression: '2+2' }, output: { result: 4 } }]

AxlTestRuntime

Test runtime that mirrors WorkflowContext for deterministic testing:

import { AxlTestRuntime, MockProvider } from '@axlsdk/testing';

const provider = MockProvider.sequence(['42']);
const runtime = new AxlTestRuntime({ provider });

// Ask an agent
const result = await runtime.ask(myAgent, 'What is the answer?');
expect(result).toBe('42');

// Inspect recorded calls
expect(runtime.agentCalls()).toHaveLength(1);
expect(runtime.toolCalls()).toHaveLength(0);
expect(runtime.totalCost()).toBe(0);

// Test spawn
const results = await runtime.spawn(3, async (i) => runtime.ask(myAgent, `Q${i}`));
expect(results.filter(r => r.ok)).toHaveLength(3);

// Test vote
const winner = runtime.vote(results, { strategy: 'majority', key: 'answer' });

// Test budget
const budgeted = await runtime.budget({ cost: '$1.00' }, async () => {
  return runtime.ask(myAgent, 'Hello');
});
expect(budgeted.budgetExceeded).toBe(false);

Assertions

const runtime = new AxlTestRuntime({ provider });

// After running your workflow:
runtime.agentCalls();   // All recorded agent invocations
runtime.toolCalls();    // All recorded tool invocations
runtime.totalCost();    // Cumulative cost
runtime.steps();        // All recorded steps (agents + tools)
runtime.traces();       // All trace events

Example: Testing a Workflow

import { describe, it, expect } from 'vitest';
import { AxlTestRuntime, MockProvider } from '@axlsdk/testing';
import { myAgent } from '../agents';

describe('my workflow', () => {
  it('returns structured output', async () => {
    const provider = MockProvider.json({ answer: 42 });
    const runtime = new AxlTestRuntime({ provider });

    const result = await runtime.ask(myAgent, 'What is the answer?', {
      schema: z.object({ answer: z.number() }),
    });

    expect(result).toEqual({ answer: 42 });
    expect(runtime.agentCalls()).toHaveLength(1);
  });
});

License

Apache 2.0