@davothebigafro/metrics

Built-in metric factories for Shipwright evaluations.

Install

pnpm add @davothebigafro/metrics @davothebigafro/eval --filter <your-package>

Inside this workspace, keep local dependencies on workspace:*. For unreleased local installs, use the tarball workflow in the root contributing guide.

Minimal Usage

import { evaluate } from '@davothebigafro/eval';
import { answerRelevancy, faithfulness } from '@davothebigafro/metrics';

const result = await evaluate({
  judge,
  testCases,
  metrics: [answerRelevancy({ threshold: 0.7 }), faithfulness({ threshold: 0.7 })],
});

The package includes RAG, agentic, safety, conversational, G-Eval, and registry helpers. Use buildMetric() when metric specs come from config.

Docs

• Evaluations
• Architecture
• Root README