prvm — Prompt Version Manager

Track, test, and manage LLM prompt versions with confidence.

Features

• Version Control — Save and track multiple versions of your prompts
• Batch Testing — Run prompts against multiple inputs and compare results
• Cross-Provider Support — Works with Anthropic, OpenAI, and Google Gemini
• Template Variables — Use {{placeholders}} in prompts for dynamic content
• Import/Export — Share prompts as JSON files
• Configurable — Set defaults for provider, model, temperature, and max_tokens

Installation

npm install -g @fanu-el/prvm

Prerequisites

• Node.js 18+
• API key for your chosen provider (Anthropic, OpenAI, or Google Gemini)

Quick Start

1. Initialize

# Set up prvm in your project directory
prvm init

# Configure your default provider and model
prvm config --provider anthropic --model claude-sonnet-4-6 --temperature 0.7

2. Save a Prompt

Create a prompt file prompts/summarize.txt:

Please summarize the following text in {{style}} style:

{{content}}

Save it:

prvm save summarize prompts/summarize.txt -n "Initial version"

3. Test with Template Variables

Create an input file inputs.json:

{
  "style": "professional",
  "content": "The quick brown fox jumps over the lazy dog. This pangram contains every letter of the English alphabet at least once."
}

Run the test:

prvm test summarize -i inputs.json

4. Batch Eval

Create an eval file eval-inputs.json:

[
  { "style": "casual", "content": "First text to summarize..." },
  { "style": "formal", "content": "Second text to summarize..." },
  { "style": "technical", "content": "Third text to summarize..." }
]

Run batch eval:

prvm eval summarize eval-inputs.json

Commands

Prompt Management

| Command | Description | |---------|-------------| | prvm save <name> <file> | Save a new prompt or new version | | prvm list | List all saved prompts | | prvm show <name> [version] | Show prompt details | | prvm use <name> <version> | Set active version | | prvm rollback <name> | Revert to previous version | | prvm diff <name> | Compare versions | | prvm history <name> | Show version history | | prvm current <name> | Show active version content |

Testing & Evaluation

| Command | Description | |---------|-------------| | prvm test <name> [version] | Test prompt with input file | | prvm eval <name> <file> | Batch eval with multiple inputs |

Configuration

| Command | Description | |---------|-------------| | prvm config --show | View current config | | prvm config --provider <p> | Set default provider | | prvm config --model <m> | Set default model | | prvm config --temperature <t> | Set default temperature | | prvm config --max-tokens <n> | Set default max tokens | | prvm config --max-runs <n> | Set max test runs to keep |

Import/Export

| Command | Description | |---------|-------------| | prvm export <name> <outFile> | Export prompt to JSON | | prvm import <file> | Import prompt from JSON |

Utilities

| Command | Description | |---------|-------------| | prvm init | Initialize prvm in current directory | | prvm gitignore --add | Add .prompts to .gitignore |

Template Variables

Use {{variable}} syntax in your prompts to create dynamic templates:

Prompt file:

Act as a {{role}} and {{task}} the following:

{{content}}

Input JSON:

{
  "role": "senior editor",
  "task": "proofread and improve",
  "content": "Your text here..."
}

The variables will be substituted before sending to the LLM.

Configuration

Environment Variables

Set your API keys as environment variables:

# Anthropic
export ANTHROPIC_API_KEY=your-key

# OpenAI
export OPENAI_API_KEY=your-key

# Google Gemini
export GOOGLE_API_KEY=your-key

Config File

After prvm init, a .prompts/config.json file is created:

{
  "provider": "anthropic",
  "model": "claude-sonnet-4-6",
  "temperature": 0.7,
  "max_runs_per_version": 20,
  "max_tokens": 4096
}

Override Options

Most commands accept override options:

# Override temperature for a single test
prvm test my-prompt -i input.json --temperature 0.5

# Override model and max_tokens
prvm test my-prompt -i input.json --model gpt-4 --max-tokens 2048

# Override provider
prvm eval my-prompt inputs.json --provider openai

Examples

See the examples/ directory for complete workflows including content creation pipelines, A/B testing, and multi-provider comparisons.

A/B Test Prompt Versions

# Save two versions
prvm save summarizer v1-prompt.txt
prvm save summarizer v2-prompt.txt

# Test both against same inputs
prvm eval summarizer v1 inputs.json
prvm eval summarizer v2 inputs.json

# Compare results
prvm diff summarizer

Multi-Provider Testing

# Test with different providers
prvm test summarizer -i input.json --provider anthropic --model claude-sonnet-4-6
prvm test summarizer -i input.json --provider openai --model gpt-4
prvm test summarizer -i input.json --provider gemini --model gemini-pro

Share Prompts

# Export
prvm export summarizer summarizer.json

# Share the file, then import elsewhere
prvm import summarizer.json

File Structure

your-project/
├── .prompts/
│   ├── config.json          # Global configuration
│   ├── summarizer/
│   │   ├── meta.json        # Prompt metadata
│   │   ├── v1.json          # Version 1
│   │   └── v2.json          # Version 2
│   └── translator/
│       ├── meta.json
│       └── v1.json
├── prompts/                  # Your prompt files (optional)
│   ├── summarizer.txt
│   └── translator.txt
└── inputs/                   # Test input files (optional)
    ├── test1.json
    └── eval.json

Contributing

1. Fork the repository
2. Create a feature branch
3. Run tests: npm test
4. Submit a pull request

License

MIT