Agent Reliability Kit

Verify, harden, and ship AI-agent-assisted codebases in one command.

Agent Reliability Kit scans a repository the way a careful maintainer would before letting AI coding agents work there: agent instructions, verification commands, README quality, secret hygiene, GitHub Actions safety, MCP/tooling risk, n8n workflow exports, team policy, and release readiness.

The flagship path is simple: keep agent-secret-guard as the sharp security wedge, and use agent-reliability-kit as the one command center for agent-era repository reliability.

Quick Start

Pre-release status: the public repository and docs are live, and the npm package is not published yet. Run from source for now; the npx command below becomes the public install path after npm publication.

• Source: https://github.com/aolingge/agent-reliability-kit
• Docs: https://aolingge.github.io/agent-reliability-kit/

npm install
npm run build
node dist/cli.js scan . --out .agent-reliability --format markdown,json,html

After npm publication:

npx agent-reliability-kit scan .

Optional focused checks:

ark team-audit . --out .agent-reliability/team
ark mcp-registry . --registry .agent-reliability/mcp-registry.json
ark n8n-scan . --out .agent-reliability/n8n
ark n8n-backup . --backup-dir .agent-reliability/n8n-backup
ark cost-report . --trace .agent-reliability/traces --budget-usd 10

The scan writes:

• .agent-reliability/report.md
• .agent-reliability/report.json
• .agent-reliability/report.html

The quick start runs entirely on your machine. Do not include real secrets, private logs, cookies, browser profiles, or private URLs in examples, fixtures, bug reports, or shared scan output.

Why It Exists

AI coding agents fail most often on the unglamorous parts: missing repo rules, unclear commands, conflicting instruction files, unsafe CI defaults, accidental secret exposure, and README promises nobody has replayed. This project turns those weak signals into one shareable report.

What It Checks

| Area | What gets verified | | --- | --- | | Agent instructions | AGENTS.md, CLAUDE.md, GEMINI.md, CODEX.md, Copilot instructions | | Commands | test, build, lint, typecheck, check scripts across common stacks | | README | install path, quick start, visual proof, license, contribution path | | Secrets | token-like values, tracked .env files, redacted evidence | | GitHub Actions | validation commands, explicit permissions, risky triggers, pipe-to-shell | | AI tooling | MCP command configs and prompt-injection-like instruction files | | MCP registry | private allowlist, trust score, approved commands/URLs, risk owner | | n8n | public webhooks, command nodes, risky code nodes, workflow secrets, redacted backups | | Team layer | scan history, policy gates, audit report, dry-run Slack payload | | Cost guard | local trace token/cost summary and budget alerts |

CLI

agent-reliability-kit scan [path]
agent-reliability-kit doctor [path]
agent-reliability-kit init [path]
agent-reliability-kit team-audit [path]
agent-reliability-kit mcp-registry [path]
agent-reliability-kit n8n-scan [path]
agent-reliability-kit n8n-backup [path]
agent-reliability-kit cost-report [path]

Examples:

ark scan . --min-score 85
ark scan . --format sarif --stdout > agent-reliability.sarif
ark doctor .
ark init .
ark team-audit .
ark mcp-registry .
ark n8n-scan .
ark cost-report . --budget-usd 10

Machine-readable stdout stays clean for CI:

ark scan . --format sarif --stdout > agent-reliability.sarif

Report Preview

The HTML report is designed for maintainers, contributors, and launch pages. It gives a score, severity counts, repository signals, and next actions for each finding.

Product Modules

• Team audit layer: scan history, policy checks, audit report, and local Slack payload.
• Private MCP registry: team allowlist with trust score, approved commands/URLs, permissions, owner, and reason.
• n8n safety and backup: risky workflow scanning and redacted Git-friendly backups.
• AI cost guard: local trace cost summaries and budget alerts.
• Commercial support path: open-source boundary and future paid team features.
• Consolidation roadmap: how small tools roll into the flagship CLI.

Comparisons

• agent-secret-guard vs gitleaks
• Agent Reliability Kit vs generic linters

Launch Kit

The repository includes a pre-release launch kit so maintainers can prepare a public launch without inventing copy or sharing private data at the last minute.

• Launch plan
• Channel copy
• Demo script
• Press kit
• Community responses
• Channel rules
• Distribution checklist
• Demo GIF script
• Product Hunt draft
• DEV article draft

Visual assets are available in assets/, including social-preview.png for GitHub/social cards and product-hunt-thumbnail.png for square launch surfaces.

Product Principles

• Local-first: source code and findings stay on your machine.
• No secret echo: token-like evidence is redacted before it appears in reports.
• Private-data safe: reports, examples, and issues must not include real secrets, private logs, cookies, browser profiles, or private URLs.
• Agent-neutral: useful for Codex, Claude Code, Cursor, Gemini CLI, OpenCode, and similar tools.
• CI-friendly: Markdown, JSON, SARIF, and GitHub Actions annotations are first-class outputs.
• Maintainer-friendly: findings explain why they matter and what to do next.

Development

npm install
npm run check
npm run build
npm run smoke

Repository layout:

src/
  cli.ts
  core/
  scanners/
  report/
  init/
tests/
  fixtures/
docs/
assets/

Roadmap

• v0.1: CLI scan, doctor, init, Markdown/JSON/HTML/SARIF reports.
• v0.2: team audit, private MCP registry, n8n safety/backup, and local cost guard.
• v0.3: GitHub Action wrapper, dogfood gallery, and agent-secret-guard rule-pack consolidation.
• v0.4: hosted team dashboard prototype, org policy packs, and private MCP approval workflow.
• v0.5: pr verify, trace run, and compatibility matrix for Codex, Claude Code, Cursor, Gemini CLI, and OpenCode.

Security

Do not include real secrets in issues, examples, or fixtures. See SECURITY.md for reporting guidance.

Contributing

Small, well-tested contributions are welcome. Start with CONTRIBUTING.md, run npm run check, and include the scanner output when changing rules.

License

MIT