@assay-ai/core

The evaluation engine powering Assay -- 18 metrics, 5 providers, zero any

Documentation · Metrics · API Reference

Installation

pnpm add @assay-ai/core     # pnpm
npm install @assay-ai/core   # npm
yarn add @assay-ai/core      # Yarn

Quick Start

import {
  AnswerRelevancyMetric,
  evaluate,
  FaithfulnessMetric,
  HallucinationMetric,
} from "@assay-ai/core";

const results = await evaluate(
  [
    {
      input: "What is the refund policy?",
      actualOutput: "You can request a full refund within 30 days.",
      retrievalContext: [
        "Refund Policy: Full refund within 30 days of purchase.",
      ],
      context: ["Our refund policy allows returns within 30 days."],
    },
  ],
  [
    new AnswerRelevancyMetric({ threshold: 0.7 }),
    new FaithfulnessMetric({ threshold: 0.7 }),
    new HallucinationMetric({ threshold: 0.3 }),
  ],
);

console.log(`Pass rate: ${results.summary.passRate.toFixed(1)}%`);

Part of the Assay monorepo