skill-portfolio

Review Agent Skills as a collection, not one file at a time.

An agent can see dozens of skills. That does not mean every skill belongs in the repository in front of you.

As a catalog grows, the useful question changes from “Can I install this?” to “Which instructions does this project actually justify?” skill-portfolio gives that decision a repeatable starting point. It scans repository paths and supported manifests plus one or more collections of SKILL.md files, builds a focused shortlist, and shows the evidence behind each shortlisted choice.

A portfolio is judged as a whole. Two individually useful skills can be redundant together, and a well-written skill can still be the wrong fit for a particular codebase. This tool reviews the set: project fit, marginal coverage, likely routing overlap, opposing global instructions, and the estimated discovery cost of the final shortlist.

Once installed, analysis stays on your machine. The analyzer has no runtime dependencies, makes no model calls, uses no network access, and never executes code from a skill.

Quick start

Review a local skill collection against the current repository:

npx skill-portfolio analyze \
  --repo . \
  --source ../Major-AI-Skills/skills

Tell the planner what work you have in mind when the repository alone does not tell the whole story:

npx skill-portfolio analyze \
  --repo . \
  --source ../Major-AI-Skills/skills \
  --goal "add browser tests and improve the Docker release workflow" \
  --max-skills 8 \
  --budget 1200

The default report is written for a person. Use JSON when another tool or CI job needs every evidence record:

npx skill-portfolio analyze \
  --repo . \
  --source ../Major-AI-Skills/skills \
  --json \
  --out skill-portfolio.json

What makes the shortlist

The planner collects an evidence ledger before it ranks anything. Strong evidence carries more weight than a coincidental filename:

| Evidence | Example | Weight | | --- | --- | ---: | | Declared dependency | @playwright/test | 8 | | Your stated goal | “add browser tests” | 6 | | Recognized configuration | playwright.config.ts | 5 | | Package script | test:e2e | 4 | | Python, Rust, or Go manifest | pyproject.toml | 3 | | Detected project language | 42 TypeScript files | 2 | | Ordinary repository path | browser-smoke.ts | 1 |

Each candidate is matched against that ledger using its name and description. A name match receives the full evidence weight. Description-only matches are discounted and need at least two distinct terms, which prevents generic prose such as “run scripts and tests” from outranking a skill whose identity directly matches playwright.config.ts. Candidates below the evidence threshold are set aside.

The remaining candidates are considered in evidence order. A skill only enters the shortlist when it adds coverage that an earlier choice did not already provide. The planner also respects the discovery-token budget, maximum skill count, and description-overlap threshold.

This is deliberately inspectable lexical analysis. It is not presented as semantic understanding.

How to read the report

A shortlist item explains three things:

playwright — strong fit from @playwright/test and playwright.config.ts.
Adds playwright and browser coverage; about 31 discovery tokens.

• Fit describes the strength of the repository or goal evidence.
• Adds shows the marginal evidence this skill contributes to the portfolio.
• Discovery tokens estimate the cost of exposing its name and description to an agent.

“Pairs to review” contains two different signals:

• A lexical overlap means two names or descriptions share enough language that an agent may have trouble choosing between them. The lower-ranked candidate is left out.
• A possible opposing instruction means the analyzer found opposite phrases about the same narrow policy, such as “always run tests” and “never run tests.” It includes evidence in JSON and should still be checked by a person.

Neither finding is a safety verdict.

Why it exists

Catalogs help people discover skills. Installers copy them into place. Validators inspect individual files. skill-portfolio handles the composition review between those steps:

• Which skills have concrete evidence in this repository?
• Which candidates add something the shortlist does not already cover?
• Which pairs deserve review before their instructions are loaded together?
• How much discovery context will the shortlist approximately consume?

The package does not install, update, or execute skills. Keeping that boundary makes the result safer to inspect and easier to reproduce.

Project evidence

Version 0.2 recognizes:

• dependencies and scripts from package.json files across a repository or monorepo
• JavaScript, TypeScript, Python, Rust, Go, Ruby, PHP, Java, Kotlin, Swift, C#, C++, Vue, and Svelte files
• Playwright, Vitest, Jest, Docker, Compose, Next.js, Vite, Tailwind, Prisma, GitHub Actions, and Kubernetes configuration
• pyproject.toml, requirements files, Cargo.toml, and go.mod
• a repeatable --goal supplied by the person making the portfolio decision

Common agent directories such as .agents, .claude, .codex, and .cursor are excluded from project evidence. Every --source path is excluded too, even when the catalog lives inside the repository. An available skill therefore cannot recommend itself merely because its files exist.

CLI reference

skill-portfolio analyze [options]

--repo <path>              Repository whose files and manifests provide evidence
--source <path>            SKILL.md file or directory to review; repeatable
--goal <text>              Work the portfolio should support; repeatable
--budget <tokens>          Estimated discovery-token ceiling (default: 3000)
--max-skills <count>       Maximum number of shortlisted skills (default: 12)
--min-score <score>        Minimum weighted evidence score (default: 3)
--overlap-threshold <0-1>  Lexical overlap threshold for names and descriptions (default: 0.45)
--fail-on-conflict         Exit 3 when shortlisted skills contain opposing rules
--json                     Print the complete evidence report as JSON
--out <path>               Save the report instead of printing it
--version                  Print the installed version

If no --source is provided, the CLI reviews .agents/skills inside the target repository. That directory is still excluded from repository evidence.

JavaScript API

import {
  analyzePortfolio,
  discoverSkills,
  inspectProject,
  renderMarkdown
} from "skill-portfolio";

const skills = await discoverSkills(["../Major-AI-Skills/skills"]);
const project = await inspectProject(".", {
  goals: ["ship browser tests"]
});
const report = analyzePortfolio(skills, project, {
  budget: 1200,
  maxSkills: 8
});

console.log(renderMarkdown(report));

The JSON report uses a versioned shape and stable decision codes. See the report schema and the methodology.

Deliberate boundaries

skill-portfolio is a pre-install decision aid. It does not:

• claim that a shortlisted skill is correct, safe, or effective;
• interpret arbitrary source code or infer semantic equivalence;
• execute scripts or resolve runtime requirements declared by a skill;
• install, remove, or update anything;
• measure tokens with a provider-specific tokenizer;
• test the behavior of an agent after the skills are loaded.

The token figures are character-based planning estimates. Similarity and policy findings are review prompts, not final judgments.

Ecosystem path

• Major AI Skills provides a broad catalog of candidates.
• skill-portfolio explains which candidates a particular repository can justify together.
• llm-contract tests whether the behavior the team approves remains acceptable.

In short: discover capabilities → compose a focused set → verify behavior.

Development

npm test
npm run check
npm run pack:check

Issues and focused pull requests are welcome. The default analysis path must remain deterministic, offline, and unable to execute inspected content.

License

MIT