@postman/aiei-engine

Scoring engine for the AI Effectiveness Index.

Repo: github.com/postmanlabs/AIEI-Framework

Evaluate organizational AI/agent readiness and measure AI-assisted engineering outcomes across quality, lead time, reliability, cost, and security. This package provides the formulas, connectors, signal routing, and presentation layer that power the CLI, MCP server, and GitHub Action.

Installation

npm install @postman/aiei-engine

Quick Start

Score from a signal payload

import { scoreFromSignals } from '@postman/aiei-engine';

const assessment = scoreFromSignals({
  schemaVersion: '2.0.0',
  orgName: 'your-org',
  signals: {
    defect_rate_ai: { value: 0.07, source: 'measured' },
    defect_rate_nonai: { value: 0.12, source: 'measured' },
    ai_test_pass_rate: { value: 0.91, source: 'measured' },
    rework_rate_ai: { value: 0.16, source: 'measured' },
    cycle_time_ai: { value: 18, source: 'measured' },
    cycle_time_nonai: { value: 42, source: 'measured' },
    deploy_frequency: { value: 3.5, source: 'measured' },
    cfr_ai: { value: 0.08, source: 'measured' },
    slo_attainment: { value: 0.96, source: 'measured' },
    uptime_ai: { value: 0.995, source: 'measured' },
    total_ai_cost: { value: 7200, source: 'estimated' },
    hours_saved_monthly: { value: 126, source: 'estimated' }
  }
}, {
  weightProfile: 'reliability_first',
  hourlyRate: 175
});

console.log(assessment.result.composite);      // 0.68
console.log(assessment.result.dimensions);     // { Q: {...}, L: {...}, R: {...}, C: {...}, S: {...} }
console.log(assessment.presentation?.overall); // score, hours, annualized value, ROI

Run a full assessment with connectors

import { runAssessment } from '@postman/aiei-engine';

const result = await runAssessment({
  orgName: 'your-org',
  weightProfile: 'reliability_first',
  hourlyRate: 175,
  connectors: {
    github: { token: process.env.GITHUB_TOKEN, org: 'your-org', repos: ['api-service'] },
    postman: { apiKey: process.env.POSTMAN_API_KEY, workspaceId: 'ws-id' },
  },
});

console.log(result.composite);
console.log(result.presentation?.overall.value_recovered_annual);

Agent readiness gate

import { assessAgentReadiness, validateSignalPayload } from '@postman/aiei-engine';

const payload = { schemaVersion: '2.0.0', orgName: 'your-org', signals: {} };
const validation = validateSignalPayload(payload);
const readiness = assessAgentReadiness(payload.readinessSignals, payload.runtimeMeasures);
// readiness.gates: discover, understand, execute, recover, govern

Exports

| Path | What It Contains | |------|------------------| | @postman/aiei-engine | Full engine: scoreFromSignals, runAssessment, connectors, types | | @postman/aiei-engine/scoring | Dimension scorers: scoreQ, scoreL, scoreR, scoreC, scoreS | | @postman/aiei-engine/connectors | Connector registry: ALL_CONNECTORS, SIGNAL_ROUTES, buildCoverageMap | | @postman/aiei-engine/presentation | buildPresentation, buildConnectorStory, narrative types |

Key Types

import type {
  CompositeResult,       // Full assessment result with dimensions, weights, epistemic notes
  PresentationResult,    // Narratives, benchmarks, AX mapping, connector stories
  ConnectorStoryBlock,   // Per-dimension tool narrative with Postman tie-back
  EpistemicSummary,      // What the score knows and doesn't know
  AXPillarMapping,       // Maps dimensions to Access/Context/Tools/Orchestration
  DimensionNarrative,    // Per-dimension story, formula, factors, benchmarks, action
  EnvironmentConfig,     // .aiei.json config shape
} from '@postman/aiei-engine';

Bundled Files

| File | Purpose | |------|---------| | openapi.yaml | HTTP contract for the assessment API | | llms.txt | Agent-oriented package discovery index | | docs/TELEMETRY.md | Telemetry transparency documentation | | docs/DEVELOPER_INDEX.md | Entry points for humans and coding agents |

40 Connectors

GitHub, GitLab, GitHub Copilot, Postman, Datadog, Dynatrace, Grafana, PagerDuty, Cloudflare, New Relic, Splunk, ServiceNow, Vercel, Supabase, Linear, Jira, Jellyfish, GitHub Actions, OpenAI, Anthropic, Google AI, Portkey, Inngest, SonarQube, 42Crunch, Fern, AWS Bedrock, AWS CloudWatch, AWS CodePipeline, AWS CodeBuild, AWS X-Ray, AWS CodeGuru, AWS Cost Explorer, AWS Step Functions, Azure OpenAI, Azure Monitor, Azure DevOps, Microsoft Defender, OpenTelemetry, Wix.

License

MIT

Built by Postman. Formulas are open, auditable, and versioned.