Code Extractor MCP

A Model Context Protocol (MCP) server that extracts code and content from GitHub repositories and Google Colab notebooks for LLM ingestion.

🚀 Quick Start

Installation

npm install -g @siddhantxh/code-extractor-mcp

Setup in AI Tools

Cursor IDE

1. Open Cursor Settings → AI → Model Context Protocol
2. Add new server:

{
  "name": "code-extractor",
  "command": "npx",
  "args": ["-y", "@siddhantxh/code-extractor-mcp"],
  "env": {
    "GITHUB_TOKEN": "your_github_token_here"
  }
}

Claude Desktop

Add to your claude_desktop_config.json:

{
  "mcpServers": {
    "code-extractor": {
      "command": "npx",
      "args": ["-y", "@siddhantxh/code-extractor-mcp"],
      "env": {
        "GITHUB_TOKEN": "your_github_token_here"
      }
    }
  }
}

🔧 Available Tools

1. tree - Repository Structure

Get the file tree structure of any GitHub repository or Colab notebook.

Example:

Use the code-extractor tree tool with https://github.com/vercel/next.js

2. fetchAllContent - Extract All Files

Download and format all files from a repository with intelligent filtering.

Example:

Use code-extractor fetchAllContent for https://github.com/microsoft/vscode excluding "**/*.test.ts,**/node_modules/**"

3. fetchAllContentButExclude - Smart Filtering

Extract files while excluding specific extensions or patterns.

Example:

Use code-extractor fetchAllContentButExclude for https://github.com/facebook/react excluding extensions: "test.js,spec.ts,.md"

4. specificContent - Individual Files

Get content from a specific file or notebook cell.

Example:

Use code-extractor specificContent for https://github.com/vercel/next.js with filePath: "packages/next/src/server/next.ts"

🌐 Supported URLs

• GitHub Repositories: https://github.com/owner/repo
• Specific Branches: https://github.com/owner/repo/tree/branch-name
• Individual Files: https://github.com/owner/repo/blob/main/file.js
• Colab Notebooks: https://colab.research.google.com/github/owner/repo/blob/main/notebook.ipynb

🔑 Authentication

Required Environment Variable

• GITHUB_TOKEN - Your GitHub Personal Access Token

Get a GitHub Token

1. Go to GitHub Settings → Developer settings → Personal access tokens
2. Click "Generate new token (classic)"
3. Select scopes: public_repo (and repo for private repositories)
4. Copy the token and add it to your MCP configuration

✨ Features

• 🌳 Intelligent Tree View - Clean repository structure visualization
• 📄 Smart Content Extraction - Respects .gitignore and custom patterns
• 🎯 Flexible Filtering - Exclude by file type, size, or custom patterns
• 📊 Token Estimation - Built-in token counting for LLM context planning
• 🔍 Binary File Detection - Automatically skips images, videos, and other binary files
• 📝 Notebook Support - Extract individual cells or full Jupyter notebooks

🛠️ Usage Examples

Analyze Repository Structure

Show me the structure of the React repository using the tree tool

Extract TypeScript Files Only

Get all TypeScript files from https://github.com/microsoft/vscode but exclude test files

Get Specific Component

Show me the content of src/components/Button.tsx from https://github.com/user/repo

Extract Colab Notebook

Extract the content from this Colab notebook: https://colab.research.google.com/github/tensorflow/docs/blob/master/site/en/tutorials/quickstart/beginner.ipynb

📊 Rate Limits & Best Practices

• GitHub API: 5,000 requests/hour with token, 60/hour without
• File Size: Default 64KB limit per file (configurable)
• Repository Size: Optimized for repos under 1,000 files
• Binary Files: Automatically filtered out (images, videos, etc.)

🐛 Troubleshooting

"0 tools enabled" in Cursor

1. Restart Cursor completely
2. Verify your GitHub token is valid
3. Check that the package is installed: npm list -g @siddhantxh/code-extractor-mcp

"Tool not found" errors

1. Ensure you're using the exact tool names: tree, fetchAllContent, etc.
2. Check your MCP configuration syntax
3. Verify the package is globally accessible: which code-extractor-mcp

Rate limit errors

1. Make sure you're using a valid GitHub token
2. Consider using exclude patterns to reduce API calls
3. For large repositories, use the sizeLimitKB parameter

📄 License

MIT License - see LICENSE file for details.

🤝 Contributing

1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Test with real repositories
5. Submit a pull request

🔗 Links

• NPM Package
• GitHub Repository
• Model Context Protocol
• Cursor IDE
• Claude Desktop