Subtext CLI

Version control and automated testing for AI prompts.

Installation

npm install -g subtext-cli

Or with yarn:

yarn global add subtext-cli

Quick Start

# Get info about a prompt
subtext info @raycast-eng/support-router

# Run tests
subtext test @raycast-eng/support-router

# Generate test cases from production data
subtext evals generate @raycast-eng/support-router --count 50

# Execute a prompt
subtext run @raycast-eng/support-router "I was charged twice"

Commands

Prompt Management

subtext info <@user/prompt>      # Show prompt details
subtext stats <@user/prompt>     # Show statistics
subtext run <@user/prompt> <input>  # Execute a prompt

Testing

subtext evals generate <@user/prompt> --count 50  # Generate test cases
subtext test <@user/prompt>                       # Run tests
subtext test <@user/prompt> --fail-under 90       # Fail if below 90%

A/B Testing

subtext experiment start <@user/prompt> --candidate 3.1.0 --traffic 10
subtext experiment status <@user/prompt>
subtext experiment promote <@user/prompt>
subtext experiment rollback <@user/prompt>

Alerts

subtext alerts list <@user/prompt>
subtext alerts add <@user/prompt> --condition "success_rate < 90" --notify "slack:#alerts"

Environment Variables

• SUBTEXT_URL - API endpoint (default: https://subtext-api-production.up.railway.app)
• SUBTEXT_API_KEY - Your API key (optional for public prompts)

Requirements

• Node.js 18 or higher

Links

• Documentation
• Web App
• GitHub

License

MIT