@aimf/testing

Testing utilities for the AI-MCP Framework (AIMF).

Installation

pnpm add -D @aimf/testing vitest

Features

Custom Matchers

AIMF-specific Vitest matchers for asserting neural data:

import { expect } from "@aimf/testing";

// Validate NCP messages
expect(message).toBeValidNcpMessage();

// Check for neural hash
expect(data).toHaveNeuralHash();

// Check for verified data (has hash + embedding)
expect(data).toBeVerifiedData();

// Match against NCP schema
expect(data).toMatchNcpSchema(schema);

Mock Factories

NCP Mocks

import {
  createMockNcpClient,
  createMockNcpServer,
  createMockNcpRequest,
  createMockNcpTransport,
} from "@aimf/testing";

// Mock NCP client
const client = createMockNcpClient();
await client.callTool("my-tool", { input: "test" });

// Mock NCP server
const server = createMockNcpServer();
server.registerTool("my-tool", "description", async (input) => ({ result: "ok" }));

// Mock NCP request
const request = createMockNcpRequest("method/name", { param: "value" });

Storage Mocks

import {
  createMockNeuralStorage,
  createMockEmbeddingProvider,
  createMockVectorDB,
} from "@aimf/testing";

// Mock neural storage
const storage = createMockNeuralStorage();
const id = await storage.store({ data: "test" });
const result = await storage.retrieve(id);

// Mock embedding provider
const embedder = createMockEmbeddingProvider();
const embedding = await embedder.embed("text");

// Mock vector database
const db = createMockVectorDB();
await db.createCollection("my-collection", 1536);
await db.upsert("my-collection", "id", embedding, { payload: "data" });

Test Fixtures

Pre-built fixtures for common test data:

import {
  createUserFixture,
  createDataFixture,
  createToolFixture,
  createManifestFixture,
  createNcpRequestFixture,
} from "@aimf/testing";

// Create test data
const user = createUserFixture({ name: "Test User" });
const data = createDataFixture();
const tool = createToolFixture({ name: "my-tool" });
const manifest = createManifestFixture();
const request = createNcpRequestFixture("test/method");

Test Utilities

Helpful utilities for async testing:

import {
  wait,
  waitFor,
  createDeferred,
  captureError,
  mockFetch,
  createAsyncSpy,
  createMockEventEmitter,
} from "@aimf/testing";

// Wait for a duration
await wait(100);

// Wait for a condition
await waitFor(() => someCondition, { timeout: 5000 });

// Create a deferred promise
const deferred = createDeferred<string>();
deferred.resolve("value");
const result = await deferred.promise;

// Capture errors from async functions
const error = await captureError(async () => {
  throw new Error("test");
});

// Mock fetch globally
mockFetch(new Map([["https://api.example.com", { data: "response" }]]));

// Create async spy with call tracking
const spy = createAsyncSpy(async (x: number) => x * 2);
await spy(5);
expect(spy.calls).toEqual([[5]]);

Setup

The package includes a global setup file that:

1. Extends expect with custom matchers
2. Stubs global fetch
3. Clears mocks between tests
4. Restores mocks after tests

To use in your vitest config:

// vitest.config.ts
import { defineConfig } from "vitest/config";

export default defineConfig({
  test: {
    setupFiles: ["@aimf/testing/dist/setup.js"],
  },
});

Or use the root workspace config:

// vitest.config.ts
import { defineConfig } from "vitest/config";

export default defineConfig({
  test: {
    setupFiles: ["./packages/testing/src/setup.ts"],
  },
});

API Reference

Matchers

| Matcher | Description | |---------|-------------| | toBeVerifiedData() | Checks for hash and embedding properties | | toBeValidNcpMessage() | Validates JSON-RPC 2.0 message structure | | toHaveNeuralHash() | Checks for 64-character hex neuralHash property | | toMatchNcpSchema(schema) | Validates object against JSON schema |

NCP Mocks

| Function | Description | |----------|-------------| | createMockNcpClient() | Mock NCP client with all methods | | createMockNcpServer() | Mock NCP server with tool/resource registration | | createMockNcpTransport() | Mock transport layer | | createMockNcpRequest(method, params?) | Create mock request | | createMockNcpResponse(id, result) | Create mock response | | createMockNcpError(id, code, message) | Create mock error response |

Storage Mocks

| Function | Description | |----------|-------------| | createMockNeuralStorage() | Full neural storage mock | | createMockEmbeddingProvider() | Mock embedding generation | | createMockVectorDB() | Mock vector database | | createMockHashProvider() | Mock hash calculation |

Fixtures

| Function | Description | |----------|-------------| | createUserFixture(overrides?) | User data fixture | | createDataFixture(overrides?) | Neural data fixture | | createToolFixture(overrides?) | Tool definition fixture | | createResourceFixture(overrides?) | Resource definition fixture | | createPromptFixture(overrides?) | Prompt definition fixture | | createManifestFixture(overrides?) | Neural manifest fixture | | createNcpRequestFixture(method, params?) | NCP request fixture | | createNcpResponseFixture(id, result) | NCP response fixture |

Utilities

| Function | Description | |----------|-------------| | wait(ms) | Wait for specified milliseconds | | waitFor(condition, options?) | Wait for condition to be true | | createDeferred<T>() | Create a deferred promise | | captureError(fn) | Capture error from async function | | mockFetch(responses) | Mock global fetch | | createMockFetchResponse(data, options?) | Create mock Response object | | createAsyncSpy(impl?) | Create spy with call tracking | | createMockEventEmitter() | Create mock event emitter | | retryWithBackoff(fn, options?) | Retry with exponential backoff |

Integration Testing

The testing package provides comprehensive integration testing utilities for testing with real services.

Running Integration Tests

# Run integration tests (requires Docker)
pnpm test:integration

# Watch mode
pnpm test:integration:watch

Container Management

Automatically manage Docker containers for testing:

import { TestContainerManager } from "@aimf/testing";

// Get container manager instance
const containers = TestContainerManager.getInstance();

// Start all containers (Qdrant, Redis)
await containers.startAll();

// Get container info
const qdrant = containers.getContainer("qdrant");
console.log(qdrant?.url); // http://localhost:6333

// Stop all containers
await containers.stopAll();

Integration Test Context

Manage shared test resources:

import {
  IntegrationTestContext,
  createIntegrationContext,
  useIntegrationTest,
} from "@aimf/testing";

// Using the hook pattern
describe("My Integration Tests", () => {
  const { getContext } = useIntegrationTest();

  it("should use shared context", async () => {
    const ctx = getContext();
    await ctx.getDatabase("qdrant").seed([/* data */]);
  });
});

// Manual setup
const context = createIntegrationContext({
  qdrantUrl: "http://localhost:6333",
  redisUrl: "redis://localhost:6379",
});
await context.setup();
// ... run tests
await context.teardown();

NCP Integration Helpers

Test NCP clients and servers with real HTTP connections:

import {
  createTestNcpServer,
  createTestNcpClient,
  createNcpTestScenario,
} from "@aimf/testing";

// Create a test server
const server = await createTestNcpServer(3999);

// Register tools
server.registerTool("echo", async (args) => ({ echoed: args }));
server.registerResource("aimf://config", async () => ({ version: "1.0" }));

// Start the server
await server.start();

// Create a client
const client = await createTestNcpClient(server.url);

// Make calls
const result = await client.callTool("echo", { message: "hello" });

// Build test scenarios
const scenario = createNcpTestScenario("multi-step")
  .callTool("echo", { step: 1 })
  .assertResult((r) => r.echoed?.step === 1)
  .wait(100)
  .callTool("echo", { step: 2 });

await scenario.run(client);

// Cleanup
await client.close();
await server.stop();

Vector Database Helpers

Test vector database operations:

import {
  createVectorDBClient,
  createMockVectorDBClient,
  createTestVectors,
  seedVectorDB,
  cosineSimilarity,
} from "@aimf/testing";

// Create real client (falls back to mock if unavailable)
const client = createVectorDBClient({
  url: "http://localhost:6333",
  fallbackToMock: true,
});
await client.connect();

// Create collection
await client.createCollection("test_collection", 128);

// Generate test vectors
const vectors = createTestVectors(100, 128);
await seedVectorDB(client, "test_collection", vectors);

// Search
const results = await client.search(
  "test_collection",
  vectors[0].vector,
  5
);

// Calculate similarity
const similarity = cosineSimilarity(vectors[0].vector, vectors[1].vector);

// Cleanup
await client.deleteCollection("test_collection");
await client.disconnect();

Integration Test Configuration

Configure integration tests in vitest.integration.config.ts:

import { defineConfig } from "vitest/config";

export default defineConfig({
  test: {
    globals: true,
    include: ["src/integration/**/*.test.ts"],
    globalSetup: "./src/integration/global-setup.ts",
    globalTeardown: "./src/integration/global-teardown.ts",
    testTimeout: 30000,
    hookTimeout: 60000,
  },
});

License

MIT