🛠 Skill Manager

概述

Skill Manager (skmgr) 是一个 npm-like 的 Agent Skill 管理平台，提供三大核心能力：

| 能力 | 说明 | 学术来源 | |------|------|---------| | 生命周期管理 | draft → validating → verified → published → installed → broken → deprecated → retired | MUSE-Autoskill, SkillOps, SkillsVote | | 多维评估 | Agent-as-a-Judge 轨迹评估 + CLEAR 五维评分 + ABC 有效性审计 | Agent-as-a-Judge (ICML 2025), CLEAR | | 自进化 | GRASP 回归门控 + HDSO 假设验证 + SkillOpt 文本优化 + 协同进化 | GRASP, SkillOpt, HDSO, CoEvoSkills |

架构

┌─────────────── TypeScript ───────────────┐  ┌───── Python ──────┐
│  CLI · Registry · MCP · Export · Store   │  │  Eval · Evolution  │
│              (port 3000)                  │  │    (port 3001)     │
└──────────────────┬───────────────────────┘  └────────┬──────────┘
                   │                                    │
                   ▼                                    ▼
            ┌─────────────┐                    ┌──────────────┐
            │ PostgreSQL  │                    │  ChromaDB     │
            └─────────────┘                    └──────────────┘

快速开始

安装

# Clone
git clone https://github.com/user/skill-manager.git
cd skill-manager

# 安装依赖
npm install

# 启动基础设施
docker-compose up -d postgres chromadb

# 初始化数据库
psql postgresql://skmgr:skmgr_dev@localhost:5432/skmgr < scripts/init-db.sql

# 启动开发环境
npm run dev

CLI 一览

# ── 生命周期 ──
skmgr create <name>              # 创建新 skill
skmgr validate <name>            # 校验 contract + format
skmgr publish <name>             # 发布到 registry
skmgr deprecate <name>           # 弃用
skmgr retire <name>              # 退役

# ── 安装管理 ──
skmgr install <name>             # 安装 skill
skmgr uninstall <name>           # 卸载
skmgr list                       # 列出已安装
skmgr search <query>             # 搜索 registry

# ── 评估 ──
skmgr eval <name>                # 完整评估（5-judge panel）
skmgr eval <name> --quick        # 快速评估
skmgr eval <name> --compare      # 版本对比

# ── 进化 ──
skmgr evolve <name>              # 触发自进化（semi-auto）
skmgr evolve <name> --auto       # 全自动进化
skmgr evolve <name> --dry-run    # 仅生成建议

# ── 其他 ──
skmgr doctor                     # 全量 drift 检测
skmgr export <name>              # 导出标准 SKILL.md
skmgr dev                        # 启动本地开发服务器

评估示例

$ skmgr eval dev-lifecycle --full

🧪 Evaluating "dev-lifecycle" — FULL MODE

Phase 1/4: Benchmark ready: 10 tasks (3 normal, 3 ambiguous, 2 degraded, 2 adversarial)
Phase 2/4: Execution complete: 30 trajectories recorded
Phase 3/4: Judge Panel
  ✓ Outcome Judge          score: 82  PASS
  ✓ Trajectory Judge       score: 71  PASS
  ✓ Safety Judge           score: 90  PASS
  ✓ Adversarial Judge      score: 85  PASS (2 refute attempts failed)
  ⚠ Capability Judge       score: 68  L3 warn: adaptability issue

┌─────────────────────────────────────┐
│  CLEAR Score                        │
├─────────────────────────────────────┤
│  Cost:       85  (token efficiency) │
│  Latency:    72  (step efficiency)  │
│  Efficacy:   78  (task success)     │
│  Assurance:  90  (safety compliance)│
│  Reliability:65  (consistency)      │
├─────────────────────────────────────┤
│  OVERALL:    78.0 / 100             │
└─────────────────────────────────────┘

💡 Run "skmgr evolve dev-lifecycle" to auto-fix the L3 issue

进化示例

$ skmgr evolve dev-lifecycle

🧬 Evolving "dev-lifecycle" — SEMI-AUTO MODE

Phase 1/5: Diagnosis complete
  Primary issue: L3 Adaptability — fails when tools are degraded

Phase 2/5: 3 hypotheses generated
  H1: Add fallback behavior (+8%)  H2: Retry logic (+5%)  H3: Pre-check tools (+3%)

Phase 3/5: H1 validated
  Control: efficacy=78 → Treatment: efficacy=86  Δ=+8
  Regression check: PASS (0 new failures)

Phase 4/5: Co-evolutionary verification: converged in 2 rounds

Phase 5/5: Evolution proposal created — pending review
  Review: skmgr evolve-log dev-lifecycle
  Apply:  skmgr evolve-approve <runId>  # writes source SKILL.md and creates .bak backup

调用者集成

方式 1: CLI

skmgr install <skill-name>

方式 2: REST API

curl http://localhost:3000/api/v1/skills/dev-lifecycle
curl -X POST http://localhost:3001/api/v1/eval -d '{"skill_name":"dev-lifecycle"}'

方式 3: MCP Server

Agent 可直接通过 MCP 调用：

{
  "tool": "skmgr_search",
  "arguments": { "query": "code review", "category": "dev" }
}

方式 4: SKILL.md Export

skmgr export dev-lifecycle --format=skillmd
# 生成标准 SKILL.md，兼容 Claude Code / Codex / OpenClaw 等 27+ 平台

项目结构

skill-manager/
├── packages/
│   ├── cli/              # skmgr CLI (Commander.js)
│   ├── registry/         # REST API (Hono)
│   ├── mcp-server/       # MCP Server
│   ├── export/           # Export engine (SKILL.md, JSON, MCP tool)
│   └── local-store/      # Local storage + lifecycle manager
├── engines/
│   ├── eval/             # Python Eval Engine (FastAPI + ChromaDB)
│   └── evolution/        # Python Evolution Engine
├── shared/types/         # Shared TypeScript type definitions
├── scripts/              # DB init, build scripts
├── docs/                 # Design docs
├── docker-compose.yml    # Infrastructure (PostgreSQL + ChromaDB)
└── package.json          # Monorepo root (turborepo)

学术基础

本项目基于以下 2025-2026 年前沿论文：

综述

• Evaluation and Benchmarking of LLM Agents (KDD 2025)
• Survey on Evaluation of LLM-based Agents (2025)
• Evolutionary Perspectives on AI Agent Evaluation (2025)

方法论

• Agent-as-a-Judge (ICML 2025)
• CLEAR: Multi-Dimensional Framework (2025)
• ABC: Agentic Benchmark Checklist (2026)
• Human-on-the-Bridge (2026)

生命周期

• MUSE-Autoskill (2026)
• SkillOps (NeurIPS 2026)
• SkillsVote (2026)
• Skill Drift as Contract Violation (2026)
• Library Drift (2026)

自进化

• GRASP (2026)
• SkillOpt (2026)
• HDSO (2026)
• CoEvoSkills (2026)
• SkillSmith (2026)

License

MIT