atsdaemon

Companion daemon for Agent Testing Suite. Runs on your local machine, connects to the cloud server, manages Docker containers for agent test runs, and streams results back in real time.

Prerequisites

• Node.js 20+
• Docker installed and running (Docker Desktop or Docker Engine)
• The agent-runner:latest Docker image built locally (see setup instructions)

Installation

npm install -g atsdaemon

Usage

1. Open the Agent Testing Suite web UI
2. Create a test run and copy the pairing token
3. Run the daemon:

atsdaemon --token <YOUR_PAIRING_TOKEN>

The daemon will:

• Verify Docker is installed and running
• Connect to the server via WebSocket
• Wait for test commands
• Spin up Docker containers for each agent setup
• Stream container output back to the server for live metrics

Options

USAGE:
  atsdaemon --token <TOKEN> [OPTIONS]

REQUIRED:
  --token <TOKEN>     Pairing token from the web UI

OPTIONS:
  --server <URL>      Server URL (default: ws://localhost:3001)
  --help, -h          Show this help message
  --version, -v       Show version

API Keys

API keys can be provided in two ways:

1. Environment variables on the machine running the daemon:

export ANTHROPIC_API_KEY=sk-ant-...
export OPENAI_API_KEY=sk-...
export GOOGLE_API_KEY=AI...
export CURSOR_API_KEY=...
atsdaemon --token abc123

2. Via the web UI -- keys entered in the browser are forwarded to the daemon over the WebSocket connection and injected into containers. They are never persisted on the server.

Keys from the web UI take precedence over local environment variables.

What It Does

The daemon is a thin orchestration layer:

• Receives test configurations (setup files, model choice, instance count) from the server
• Creates isolated Docker containers using the agent-runner image
• Mounts setup files into each container
• Streams container stdout/stderr back to the server line-by-line
• Reports container lifecycle events (started, stopped, crashed, OOM)
• Enforces resource limits (4GB memory, 2 CPUs, 100 PIDs per container)
• Handles graceful shutdown on SIGINT/SIGTERM

Security

• Containers run with enforced resource limits (memory, CPU, PIDs)
• Setup files are written to a temp directory with path traversal protection
• API keys are only held in memory, never written to disk by the daemon
• Containers use an isolated Docker network (agent-network)
• The daemon connects outbound to the server (no inbound ports needed)

License

MIT