delivery-intel

v1.8.1

Published

a month ago

Evidence-driven repo intelligence: DORA metrics, forensic signals, vulnerability scan, and delivery verdict for any GitHub repo. One command, no setup.

delivery-intel

The repo trust engine for GitHub.

Evidence-driven intelligence that tells you whether a repo is safe to adopt, healthy to change, and disciplined enough to trust in CI.

delivery-intel answers three questions — with evidence, not guesses:
Should I trust this repo? (trust score, 6 trust dimensions, 15 forensic signals)
Does it ship well and fail safely? (DORA metrics, vulnerability scan, change fail rate)
How does it compare? (side-by-side comparison of any two repos)
Every metric shows its source, sample size, and confidence level. No fake precision.

⚡ Quick Start

# Trust verdict — should I adopt this repo? (trust score, dimensions, forensics)
npx delivery-intel facebook/react --mode adopt

# Compare two repos side-by-side
npx delivery-intel --compare facebook/react vercel/next.js

# Full delivery health check (v2 engine)
npx delivery-intel facebook/react --v2

# Classic metrics dump (v1 engine)
npx delivery-intel facebook/react

┌──────────────────────────────────────────────────────────────┐
│  facebook/react                                              │
│  Grade A-   Score 91/100   Confidence high                  │
│  Trend ↑ +4 pts                                             │
└──────────────────────────────────────────────────────────────┘

✓   NO POLICY VIOLATIONS
──────────────────────────────────────────────────────────────
  ✓ All delivery health policies are passing.

METRICS  (source · sample · confidence)
──────────────────────────────────────────────────────────────
  ◆ Deploy Frequency       12.4/wk  [high]  deployment_statuses
  ◆ Change Lead Time       3.2h     [high]  commit_to_deploy (84 PRs)
  ● Recovery Time          0.8h     [med]   deployment_failures
  ◆ Change Fail Rate       4.8%     [high]  deployment_statuses
  ● Pipeline Failures      8.2%     [high]  workflow_runs (52 runs)

SIGNAL QUALITY
──────────────────────────────────────────────────────────────
  ◈ Production signal strength: high
  ✓ Scanned manifests: npm (package.json), Go (go.mod)

SECURITY
──────────────────────────────────────────────────────────────
  ✓ No known vulnerabilities found in scanned manifests

Works with full URLs too: npx delivery-intel https://github.com/vercel/next.js --v2

Why DORA metrics matter

Google's 2024 DORA Report (32,000+ respondents, 10 years of data):

| Metric | Elite teams | Low performers | Gap | |--------|------------|----------------|-----| | Deploy Frequency | On-demand (multiple/day) | < once per 6 months | 973× | | Lead Time for Changes | < 1 hour | 1 – 6 months | 6,570× | | Change Failure Rate | 0–15% | 46–60% | — | | Time to Restore | < 1 hour | 1 week – 1 month | 6,570× |

Elite teams are 2× more likely to meet reliability targets and 1.8× more likely to meet business goals (DORA 2024).

Lead Time ★ Elite 3.2 hours median (0.1 days)

Change Failure Rate ● High 4.8% (2 failed / 42 total runs)

◈ Vulnerability Scan (OSV.dev) ──────────────────────────────────────────────────────── ✓ No known vulnerabilities found

◈ Suggestions ──────────────────────────────────────────────────────── ✓ Looking good, no critical issues detected


> Works with full URLs too: `npx delivery-intel https://github.com/vercel/next.js`

---

## What It Measures

| Metric | What it tells you | Source | Confidence |
|--------|-------------------|--------|------------|
| **Deploy Frequency** | How often code ships to production | Deployment statuses → deploys → Actions → PRs | Waterfall |
| **Change Lead Time** | Commit → production (PR flow as fallback) | Deployments + commits | High / Low |
| **Change Fail Rate** | % of deployments that failed or rolled back | Deployment statuses + heuristics | High / Low |
| **Recovery Time** | Time from failure to successful recovery | Deployment failures + workflow runs | High / Low |
| **Pipeline Failure Rate** | CI workflow run reliability (distinct from CFR) | Workflow Runs API | High |
| **Deployment Rework Rate** | Rollback/revert/hotfix rate (inferred) | PR titles + labels + deployment refs | Low (flagged) |
| **Vulnerabilities** | Known CVEs across 7 manifest ecosystems | [OSV.dev](https://osv.dev) batch API | — |
| **Delivery Score** | Confidence-weighted composite (0–100) | All of the above | Aggregate |

> Every metric exposes: data source · sample size · lookback window · confidence level.
> Low-confidence signals contribute less to the composite score — no fake precision.

Supported manifest ecosystems: **npm** · **pip** · **Go modules** · **Poetry** · **pnpm** · **Cargo** · **RubyGems**

---

## 🔍 Trust Engine

The trust engine (`--mode adopt`) computes a **Trust Score (0–100)** across 6 weighted dimensions, powered by 15 forensic signal detectors:

| Dimension | Weight | Key signals |
|-----------|--------|-------------|
| **Maintenance freshness** | 1.0 | Days since last commit/release, commit cadence |
| **CI reliability** | 0.9 | Pipeline failure rate, CI flakiness across workflows |
| **Vulnerability exposure** | 0.9 | Critical/high CVE density from OSV.dev scan |
| **Release hygiene** | 0.8 | Semver compliance, conventional commits, changelog |
| **Contributor concentration** | 0.8 | Bus factor — single-maintainer risk |
| **Change safety** | 0.7 | Rollback signals, rework density, review latency |

Trust levels: **high** (≥75) · **moderate** (≥45) · **low** (<45) · **insufficient-evidence** (too few signals)

### Compare mode

```bash
npx delivery-intel --compare facebook/react vercel/next.js
npx delivery-intel --compare facebook/react vercel/next.js --json

Runs the trust engine on both repos and produces a structured comparison: per-dimension deltas, metric tier diffs, unique forensic signals, and a winner verdict.

Fix Packs with Trust Gain

Every fix pack now includes a trust gain estimate — how much each fix would improve the trust score:

{
  "id": "fix-missing-dependabot",
  "title": "Enable Dependabot",
  "trustGain": 15,          // estimated trust score improvement
  "rationale": "Automated dependency updates reduce vulnerability exposure..."
}

Usage

CLI (zero install)

# Verdict: grade, confidence, policy violations, evidence chain (v2 engine)
npx delivery-intel facebook/react --v2

# Classic metrics output (v1 engine — backwards compatible)
npx delivery-intel facebook/react

# Compare last 30 days vs prior 30 days
npx delivery-intel vercel/next.js --trend

# Include workflow strain analysis
npx delivery-intel vercel/next.js --risk

# AI-powered executive narrative (requires LLM key — falls back to template)
npx delivery-intel vercel/next.js --narrative

# JSON output (v2 includes full evidence chain + policy result)
npx delivery-intel vercel/next.js --v2 --json

# Save report to file
npx delivery-intel vercel/next.js --v2 --json --output report.json

All flags

| Flag | Description | |------|-------------| | --v2 | Evidence-driven engine with grade, confidence, forensics, and policy | | --mode <mode> | Analysis mode: oss, adopt, pr, exec, platform (implies --v2) | | --pr-comment | Write a PR guardrail comment to delivery-intel-pr-comment.md (use with --v2) | | --fail-below N | Exit code 2 if delivery score is below N | | --block | Enable blocking violations (use with --v2 --fail-below) | | --compare | Compare two repos side-by-side: --compare repoA repoB (implies --v2, defaults to adopt mode) | | --json | Output raw JSON instead of the formatted terminal report | | --output <file> | Write JSON to a file (can combine with --json) | | --trend | Show 30-day vs prior-30-day deltas for all metrics | | --risk | Include workflow strain analysis (velocity + stability signal) | | --narrative | Generate an executive summary (LLM or template fallback) | | --token <token> | GitHub token — prefer gh auth login instead | | --no-spinner | Disable the scanning animation (useful in CI logs) | | --version | Print version | | --help | Show help |

Web Dashboard

git clone https://github.com/ParthibanRajasekaran/delivery-intel.git
cd delivery-intel
npm install
npm run dev
# → http://localhost:3000

Paste a repo URL and get an animated dashboard with score ring, DORA cards, charts, vulnerability table, and suggestions.

Docker

# Dashboard
docker compose up dashboard

# CLI
REPO=facebook/react docker compose run --rm cli

📦 JSON Output Schema

Pass --json (or --json --output report.json) to get machine-readable output.

{
  "repo": { "owner": "vercel", "repo": "next.js" },
  "fetchedAt": "2026-04-17T12:00:00.000Z",
  "overallScore": 87,                          // 0–100
  "doraMetrics": {
    "deploymentFrequency": {
      "deploymentsPerWeek": 12.4,
      "rating": "Elite",                       // Elite | High | Medium | Low
      "source": "merged_prs_fallback"          // deployments_api | merged_prs_fallback
    },
    "leadTimeForChanges": {
      "medianHours": 3.2,
      "rating": "Elite"
    },
    "changeFailureRate": {
      "percentage": 4.8,
      "failedRuns": 2,
      "totalRuns": 42,
      "rating": "High"
    }
  },
  "vulnerabilities": [
    {
      "packageName": "lodash",
      "currentVersion": "4.17.15",
      "vulnId": "GHSA-xxxx-xxxx-xxxx",
      "summary": "Prototype pollution",
      "severity": "high",                      // critical | high | medium | low
      "aliases": ["CVE-2021-23337"],
      "fixedVersion": "4.17.21"
    }
  ],
  "suggestions": [
    {
      "category": "reliability",               // performance | reliability | security
      "severity": "high",                      // high | medium | low
      "title": "High Pipeline Failure Rate",
      "description": "...",
      "actionItems": ["..."]
    }
  ],
  "dailyDeployments": [0, 1, 2, 3, 1, 2, 3], // last 7 days, index 0 = 6 days ago
  // present only with --trend
  "trend": {
    "windowDays": 30,
    "current":  { "deploymentsPerWeek": 12.4, "leadTimeHours": 3.2, "changeFailureRate": 4.8, "score": 87 },
    "prior":    { "deploymentsPerWeek": 9.1,  "leadTimeHours": 5.6, "changeFailureRate": 6.2, "score": 78 },
    "deltas":   { "deploymentsPerWeek": 3.3,  "leadTimeHours": -2.4, "changeFailureRate": -1.4, "score": 9 }
  },
  // present only with --risk
  "riskScore": {
    "score": 42,
    "level": "moderate",                       // low | moderate | high | critical
    "cycleTimeDelta": 0.12,
    "failureRateDelta": -0.05,
    "sentimentMultiplier": 1.0,
    "summary": "..."
  }
}

🔐 Authentication

| Method | Setup | Best for | |--------|-------|----------| | None | Just run it | Public repos (60 req/hr) | | gh auth login | brew install gh && gh auth login | Daily use, token stays in OS keychain ✨ | | GITHUB_TOKEN | export GITHUB_TOKEN=ghp_... | CI environments | | --token | --token ghp_... | Quick one-off (avoid in CI) |

Token resolution order: --token flag → GITHUB_TOKEN env → gh auth token

Private repos require a token with repo scope. For CI, use ${{ secrets.GITHUB_TOKEN }}. It's auto-scoped and expires per job.

CI Integration

GitHub Actions Marketplace action

The easiest way — use the action directly:

# .github/workflows/delivery-intel.yml
name: Delivery Health Check

on:
  push:
    branches: [main]
  pull_request:

jobs:
  analyze:
    runs-on: ubuntu-latest
    steps:
      - uses: ParthibanRajasekaran/[email protected]
        with:
          fail-below: '40'   # fail the job if score drops below 40

Outputs available after the step: score, deploy-frequency, lead-time, change-failure-rate, mean-time-to-restore.

Evidence-backed PR guardrail (v2 engine)

Posts a policy-aware comment on every PR — no block unless a real threshold is breached:

    steps:
      - name: Delivery Health Check
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
        run: |
          npx delivery-intel@latest ${{ github.repository }} \
            --v2 \
            --pr-comment \
            --fail-below 40 \
            --no-spinner

      - name: Post PR Comment
        if: always() && github.event_name == 'pull_request'
        env:
          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
        run: |
          gh pr comment ${{ github.event.pull_request.number }} \
            --body-file delivery-intel-pr-comment.md \
            --repo ${{ github.repository }}

The comment leads with a verdict (✅ No block / ⚠️ Warning / 🚫 BLOCKED), followed by a collapsible metrics table with source and confidence for every data point.

npx (custom pipeline)

    steps:
      - name: Run delivery-intel
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
        run: npx delivery-intel@latest ${{ github.repository }} --v2 --json --output report.json

      - name: Check health score
        run: |
          SCORE=$(jq '.scores.delivery.score' report.json)
          echo "Health score: $SCORE / 100"
          if (( $(echo "$SCORE < 40" | bc -l) )); then
            echo "::error::Score $SCORE is below threshold (40)"
            exit 1
          fi

🏅 Badge

Once you have the dashboard deployed, you can show a live delivery score in any README:

[![Delivery Score](https://your-deployment-url/api/badge?repo=owner/repo)](https://github.com/ParthibanRajasekaran/delivery-intel)

The GET /api/badge?repo=owner/repo endpoint returns a Shields.io endpoint-badge payload. Score maps to color: < 20 red → < 40 orange → < 60 yellow → < 80 green → ≥ 80 bright green. Results cached 5 minutes.

Self-hosting: deploy the dashboard (npm run build && npm start or docker compose up dashboard) and replace your-deployment-url.

🏗️ Architecture

┌──────────────────────────────────────────────────────────────┐
│                      delivery-intel                          │
├──────────────┬───────────────┬───────────────────────────────┤
│   CLI        │   Dashboard   │   CI Workflow / Action        │
│  (npx)       │  (Next.js)    │   (action.yml)                │
├──────────────┴───────────────┴───────────────────────────────┤
│    Renderers (terminal, JSON, PR comment, compare output)    │
├──────────────────────────────────────────────────────────────┤
│  Trust Engine  ·  Compare Engine  ·  Verdict Engine          │
│  (6 dimensions, 15 signals, trust score 0–100)               │
├──────────┬────────────┬──────────────┬───────────────────────┤
│ Scoring  │ Policies   │  Fix Packs   │  Recommendations      │
│ (conf-   │ (gates,    │  (trust gain │  (ranked actions)     │
│  weighted)│  blocking) │   estimates) │                       │
├──────────┴────────────┴──────────────┴───────────────────────┤
│  DORA Metrics (6) · Confidence · Evidence Sources · Caveats  │
├──────────────────────────────────────────────────────────────┤
│       Normaliser (raw API → typed EvidenceEvent stream)       │
├──────────┬────────────┬──────────────────────────────────────┤
│ GitHub   │ OSV.dev    │  Extractors (7 ecosystems)           │
│ REST API │ Vuln API   │  npm·pip·Go·Poetry·pnpm·Cargo·Gems  │
└──────────┴────────────┴──────────────────────────────────────┘

🛠 Tech Stack

AIDT — Canary Trust Scoring

AI tools accelerate delivery, but they introduce a Velocity Paradox: the same tooling that lets a developer ship more code per day also reduces the natural review friction that catches subtle regressions. A PR co-authored by Copilot or Claude may contain highly regular, syntactically correct code that passes linting and unit tests but encodes wrong assumptions — and because the code looks polished, reviewers spend less time on it. AIDT (AI-Integrated Delivery Telemetry) makes this risk visible by combining an AI authorship signal with a review density measurement and mapping both to a recommended canary deployment window.

CLI usage

# Standalone — analyse a single PR
npx delivery-intel owner/repo --aidt --pr 142

# Combined with v2 engine — full delivery health + canary trust
npx delivery-intel owner/repo --v2 --aidt --pr 142

# JSON output
npx delivery-intel owner/repo --aidt --pr 142 --format json

Set COPILOT_ORG_TOKEN in your environment to enable Heuristic 3 (Copilot acceptance rate). Without it the heuristic is skipped and its weight is redistributed proportionally across the other two signals.

Output shape

| Field | Type | Description | |---|---|---| | pr_number | number | The PR that was analysed | | ai_authorship_pct | number (0–1) | Combined AI authorship probability | | review_time_per_loc | number | Seconds of review time per line changed | | baseline_density | number | Rolling 90-day median s/loc (or fallback) | | risk_weight | number (0–1, 3dp) | Composite risk weight | | canary_duration_minutes | 30 \| 60 \| 240 | Recommended canary window | | risk_tier | "low" \| "medium" \| "high" | Human-readable tier | | authorship_signals.description_match | boolean | H1: AI phrase found in PR title/body | | authorship_signals.diff_entropy_score | number | H2: diff regularity score (high = AI-like) | | authorship_signals.copilot_acceptance_rate | number \| null | H3: org-level Copilot acceptance rate (null if unavailable) | | baseline_source | "computed" \| "fallback" | Whether baseline was calculated or defaulted | | recommendation | string | Human-readable deployment guidance |

BASELINE_DENSITY

BASELINE_DENSITY is the rolling 90-day median of review_time_seconds / lines_changed computed from merged PRs in the target repo that contain no AI authorship signals (description patterns, Copilot attribution, etc.). It answers the question: how much review attention does a human-authored change in this repo typically receive per line?

If fewer than 10 qualifying PRs exist in the 90-day window, the module falls back to 6.0 seconds/line — a conservative estimate derived from typical open-source PR review patterns.

Calibration note

The thresholds used in AIDT (risk weight 0.70 and 0.45, Heuristic 3 weight 20%, review density baseline fallback 6.0 s/loc) are calibration starting points derived from a single production environment. You should recalibrate these thresholds against your own team's incident data before treating canary durations as hard gates.

Ethics note

This module produces team-level trending signals. Do not use review density or authorship scores as individual engineer performance metrics.

🤝 Contributing

Contributions welcome! See CONTRIBUTING.md for dev setup, coding standards, and workflow.

git clone https://github.com/ParthibanRajasekaran/delivery-intel.git
cd delivery-intel
npm install
npm run validate   # lint + typecheck + test in one shot

📄 License

MIT. Use it however you want.

Published

Vulnerabilities

Links

Maintainers

Keywords

Readme

delivery-intel

⚡ Quick Start

Why DORA metrics matter

Fix Packs with Trust Gain

Usage

CLI (zero install)

All flags

Web Dashboard

Docker

📦 JSON Output Schema

🔐 Authentication

CI Integration

GitHub Actions Marketplace action

Evidence-backed PR guardrail (v2 engine)

npx (custom pipeline)

🏅 Badge

🏗️ Architecture

🛠 Tech Stack

AIDT — Canary Trust Scoring

CLI usage

Output shape

BASELINE_DENSITY

Calibration note

Ethics note

🤝 Contributing

📄 License