@relayplane/core-cli

CLI for the RelayPlane agent optimization layer. Provides commands for managing routing rules, viewing statistics, and running prompts.

Installation

npm install -g @relayplane/core-cli
# or
pnpm add -g @relayplane/core-cli

Commands

Stats

Show usage statistics:

rpc stats                    # Show stats for last 30 days
rpc stats --days 7           # Show stats for last 7 days
rpc stats --json             # Output as JSON

Example output:

📊 RelayPlane Stats
═══════════════════════════════════════
Total runs:        150
Success rate:      87%
Period:            Last 30 days

By Task Type:
  code_review          45 runs   91%  1234ms
  summarization        32 runs   88%  856ms
  code_generation      28 runs   85%  2100ms

Routing

Manage routing rules that determine which model handles each task type:

rpc routing list                           # List all routing rules
rpc routing get code_review                # Get rule for a task type
rpc routing set code_review openai:gpt-4   # Set routing rule
rpc routing delete code_review             # Reset to default

Example output:

🔀 Routing Rules
═══════════════════════════════════════
  code_generation      → openai:gpt-4              [user]
  code_review          → anthropic:claude-3-opus   [learned] 92%
  summarization        → local:llama3.2            [default]

Suggest

Manage routing improvement suggestions based on performance data:

rpc suggest                    # List pending suggestions
rpc suggest list               # Same as above
rpc suggest accept <id>        # Accept and apply a suggestion
rpc suggest reject <id>        # Reject a suggestion
rpc suggest generate           # Generate new suggestions

Example output:

💡 Pending Suggestions
═══════════════════════════════════════
  Found 2 suggestion(s):

  ID: abc123
  Task: code_review
  Current: local:llama3.2 → Suggested: anthropic:claude-3-opus
  Reason: 23% higher success rate observed
  Confidence: 87%

Benchmark

Run routing performance benchmarks:

rpc benchmark                              # Run full benchmark
rpc benchmark --iterations 5               # Run 5 iterations per task
rpc benchmark -t code_review summarization # Benchmark specific task types
rpc benchmark --json                       # Output as JSON

Example output:

⚡ Routing Performance Benchmark
═══════════════════════════════════════
ℹ Running 3 iteration(s) per task type...

✓ Completed 27 benchmark runs

code_review:
  anthropic:claude-3-opus      1.2ms   100%  (3 runs)

summarization:
  local:llama3.2               0.8ms   100%  (3 runs)

Summary:
  Total runs:         27
  Avg inference time: 1.0ms
  Routing accuracy:   100%

Run

Run a prompt through RelayPlane:

rpc run "Review this code for bugs"        # Run with auto-routing
rpc run "Hello" -t general                 # Override task type
rpc run "Test" -m openai:gpt-4             # Override model
rpc run "Test" --dry-run                   # Show routing without executing
rpc run "Test" -s "You are helpful"        # With system prompt
rpc run "Test" --json                      # Output as JSON

Example output:

🚀 Running Prompt
  "Review this code for bugs"

✓ Run completed successfully

Run ID:        abc123xyz
Task type:     code_review
Model:         anthropic:claude-3-opus
Duration:      1.2s
Tokens in:     45
Tokens out:    128

Output:
────────────────────────────────────────
The code looks good overall...
────────────────────────────────────────

JSON Output

All commands support --json for scripting:

# Get stats as JSON for processing
rpc stats --json | jq '.totalRuns'

# List rules and filter
rpc routing list --json | jq '.[] | select(.source == "learned")'

Task Types

RelayPlane recognizes these task types:

• code_generation - Writing new code
• code_review - Reviewing existing code
• summarization - Summarizing content
• analysis - Analyzing data or situations
• creative_writing - Creative content generation
• data_extraction - Extracting structured data
• translation - Language or format translation
• question_answering - Answering questions
• general - General-purpose tasks

Model Format

Models are specified in provider:model format:

• openai:gpt-4
• openai:gpt-3.5-turbo
• anthropic:claude-3-opus
• anthropic:claude-3-sonnet
• google:gemini-pro
• xai:grok-1
• local:llama3.2

Database

RelayPlane stores data in ~/.relayplane/data.db by default. Set a custom path with the RELAYPLANE_DB_PATH environment variable.

Related Packages

• @relayplane/core - Core agent optimization library
• @relayplane/sdk - Workflow SDK
• @relayplane/adapters - Model provider adapters
• @relayplane/cli - Cloud CLI (login, sync, workflows)

License

MIT