@microsoft/vally-cli

The vally command-line interface for Vally — the evaluation platform built for builders.

This package is a thin wrapper over @microsoft/vally that exposes the evaluation pipeline (lint, eval, grade, compare, export) as a CLI suitable for inner-loop development, CI, and outer-loop runs.

For end-user documentation, visit https://aka.ms/vally.

Installation

Install globally:

npm install -g @microsoft/vally-cli

Or run on demand without installing:

npx @microsoft/vally-cli <command>

Requires Node.js >=22.

Quick start

vally lint           # fast static checks (no execution)
vally eval           # run stimuli against an agent and grade the results
vally export         # convert eval specs to external tool formats

Commands

| Command | Description | | --------- | ------------------------------------------------------------- | | lint | Static checks for skills and eval specs (fast, no execution). | | eval | Run stimuli against an agent and grade the results. | | grade | Grade trajectories piped in from stdin using an eval spec. | | compare | Pairwise comparison of two eval runs. | | export | Export eval.yaml to an external tool format. |

Run any command with --help to see its full options:

vally lint --help
vally eval --help

License

MIT — see LICENSE. Copyright (c) Microsoft Corporation.