Skills Indexer

AI-powered GitHub skills indexer with hierarchical taxonomy organization.

Companion tool, not part of the published npm package. This directory is a separate project (skill-tree-indexer) shipped in the repo for reference. The skill-tree npm package publishes only its own dist/ and does not include this tool or its dependencies.

Overview

The Skills Indexer discovers, scrapes, and indexes Agent Skills from GitHub repositories. It uses Claude AI to intelligently classify skills into a hierarchical taxonomy, enabling semantic search and discovery.

⚠️ Security & permissions

The indexer can spawn Claude Code agents to analyze scraped repositories (src/agents/spawner.ts). By default these agents run with an auto-approve permission mode, meaning the spawned agent may execute tools without prompting while reasoning over the contents of untrusted, third-party repositories.

Before pointing the indexer at repositories you do not control:

• Understand that scraped repo content is untrusted input that flows into agent prompts — treat it as a prompt-injection surface.
• Prefer running the indexer in a sandboxed/ephemeral environment.
• Review src/agents/spawner.ts and tighten the permission mode if you are scraping at scale or from arbitrary sources.
• Keep your GITHUB_TOKEN / ANTHROPIC_API_KEY scoped to the minimum needed; never commit a real .env (only .env.example is tracked).

Features

• GitHub Scraping: Discovers skills from awesome-lists and repositories
• AI Classification: Uses Claude to categorize skills into a hierarchical tree
• Multi-path Placement: Skills can belong to multiple taxonomy categories
• Relationship Detection: Identifies dependencies, alternatives, and extensions
• Async Architecture: Designed for scale with concurrent processing

Quick Start

# Install dependencies
npm install

# Set up environment
cp .env.example .env
# Edit .env with your GitHub token and Anthropic API key

# Run database migrations
npm run db:migrate

# Scrape skills from default sources
npx skillindexer scrape --discover

# Index unclassified skills
npx skillindexer index

# Browse the taxonomy
npx skillindexer tree

Commands

# Scraping
skillindexer scrape <repo-url>     # Scrape specific repository
skillindexer scrape --discover     # Scrape from configured sources

# Indexing
skillindexer index                 # Index all unclassified skills
skillindexer index --skill <id>    # Re-index specific skill

# Browsing
skillindexer tree                  # Show full taxonomy tree
skillindexer tree <path>           # Show subtree (e.g., "Development/Python")
skillindexer list                  # List all skills
skillindexer show <skill-id>       # Show skill details

# Search
skillindexer search <query>        # Keyword search

# Maintenance
skillindexer stats                 # Show statistics
skillindexer export --json         # Export database to JSON

Configuration

Environment Variables

| Variable | Required | Default | Description | |----------|----------|---------|-------------| | GITHUB_TOKEN | Yes | - | GitHub personal access token | | ANTHROPIC_API_KEY | Yes | - | Anthropic API key | | SKILLINDEXER_DB_PATH | No | ./data/skills.db | SQLite database path | | SKILLINDEXER_CACHE_DIR | No | ./data/cache | Cache directory | | SKILLINDEXER_LOG_LEVEL | No | info | Log level (debug/info/warn/error) | | SKILLINDEXER_CLAUDE_MODEL | No | claude-sonnet-4-20250514 | Claude model to use |

Optional PostgreSQL support

SQLite is the default and does not require a PostgreSQL client package. To use database.type: 'postgresql', install the optional peer dependency:

npm install pg

The PostgreSQL server must also provide the vector extension. The separate pgvector npm package is not used.

Architecture

See docs/DESIGN.md for detailed architecture documentation.

┌─────────────┐    ┌─────────────┐    ┌─────────────┐
│    CLI      │───▶│   Scraper   │───▶│   Indexer   │
└─────────────┘    └─────────────┘    └─────────────┘
       │                 │                   │
       └────────────────────────────────────┘
                        │
                        ▼
               ┌─────────────────┐
               │  Database       │
               │ SQLite / opt. PG│
               └─────────────────┘

Default Sources

Skills are scraped from:

1. VoltAgent/awesome-agent-skills - ~172+ curated skills
2. VoltAgent/awesome-openclaw-skills - ~700+ community skills

Development

# Watch mode for development
npm run dev

# Run tests
npm test

# Type check
npm run lint

# Database management
npm run db:generate  # Generate migrations
npm run db:migrate   # Run migrations
npm run db:studio    # Open Drizzle Studio

License

MIT