n8n-nodes-gitlab-code-splitter

This is an n8n community node package that provides integration with the GitLab Code Splitter API. It includes both direct processing nodes and job-based workflow nodes for splitting GitLab and local repository code into manageable chunks for AI processing and analysis.

Features

Direct Processing Nodes (Legacy)

• System Operations: Health checks and supported language queries
• Code Splitting: Split individual code content into manageable chunks
• Repository Processing: Comprehensive GitLab repository code splitting with various filtering options
• Architectural Layer Support: Split code by architectural layers (domain, application, adapters, ports, tests, command)
• Custom Path Filtering: Include/exclude specific paths and file extensions

Job-Based Workflow Nodes (Recommended)

• Background Processing: Handle large repositories without timeout issues
• Progress Tracking: Monitor job progress and completion status
• Paginated Results: Retrieve chunks in manageable pages
• GitLab & Local Support: Process both GitLab repositories and local file systems
• Automatic Storage: Upload results to GCP Storage
• Error Handling: Robust error handling and retry mechanisms

Installation

Follow the installation guide in the n8n community nodes documentation.

npm install n8n-nodes-gitlab-code-splitter

Available Nodes

Job-Based Workflow Nodes (Recommended)

1. Job-Based GitLab Splitter - Create background jobs for GitLab repositories
2. Job-Based Local Repo Splitter - Create background jobs for local repositories
3. Job Status Checker - Monitor job progress and wait for completion
4. Chunk Retriever - Retrieve processed chunks with pagination

Direct Processing Nodes (Legacy)

1. GitLab Code Splitter - Direct processing of GitLab repositories

Configuration

Credentials

Code Splitter API (For Job-Based Nodes)

• API URL: The base URL of your Code Splitter API server (e.g., http://localhost:3000)
• API Key: Your API key for authentication

GitLab Code Splitter API (For Legacy Nodes)

• API URL: The base URL of your GitLab Code Splitter API instance (default: http://localhost:3000)
• API Key: Authentication key sent in X-API-Key header for all API requests

Operations

System Operations

• Health Check: Verify API availability
• Get Languages: Retrieve supported programming languages

Code Operations

• Split: Split individual code content with customizable options

Repository Operations

• Split All Files: Process all repository files
• Split by Layer: Process specific architectural layers
• Split by Path: Process files in custom paths

Parameters

GitLab Configuration

• GitLab URL: GitLab instance URL (validates URL format)
• Project ID: GitLab project identifier (group/project format)
• GitLab Token: Personal access token (validates token format: glpat-, gldt-, or gloas-)
• Branch: Git branch to process

File Filtering

• File Extensions: Specify file types to process
• Include Paths: Directory paths to include
• Exclude Paths: Directory paths to exclude (default: .git, vendor, node_modules)
• Max File Size: Maximum file size limit in bytes

Split Options

• Max Tokens: Maximum tokens per chunk
• Overlap: Token overlap between chunks
• Min Chunk Size: Minimum chunk size in tokens
• Preserve Newlines: Whether to preserve newline characters

Usage Examples

Job-Based Workflow (Recommended)

GitLab Repository Processing

1. Job-Based GitLab Splitter
   ├── Project ID: 12345678
   ├── Token: glpat-xxxxxxxxxxxxxxxxxxxx
   ├── Branch: main
   └── Returns: { jobId: "abc-123", status: "pending" }

2. Job Status Checker
   ├── Job ID: abc-123 (from step 1)
   ├── Operation: Wait for Completion
   └── Returns: { status: "completed", totalChunks: 1500 }

3. Chunk Retriever
   ├── Job ID: abc-123 (from step 1)
   ├── Operation: Get All Chunks
   └── Returns: All processed chunks (paginated)

Local Repository Processing

1. Job-Based Local Repo Splitter
   ├── Repository Path: /path/to/local/repo
   ├── File Extensions: [".js", ".ts", ".py"]
   └── Returns: { jobId: "def-456", status: "pending" }

2. Job Status Checker
   ├── Job ID: def-456 (from step 1)
   └── Wait for completion

3. Chunk Retriever
   ├── Job ID: def-456 (from step 1)
   └── Get processed chunks

Direct Processing (Legacy)

Health Check

{
  "resource": "system",
  "operation": "healthCheck"
}

Split Domain Layer

{
  "resource": "repository",
  "operation": "splitByLayer",
  "layerType": "domain",
  "gitlabUrl": "https://gitlab.com",
  "projectId": "mygroup/myproject",
  "token": "glpat-xxxxxxxxxxxxxxxxxxxx",
  "branch": "main"
}

Split Custom Code

{
  "resource": "code",
  "operation": "split",
  "code": "package main\n\nimport \"fmt\"\n\nfunc main() {\n\tfmt.Println(\"Hello, World!\")\n}",
  "language": "go",
  "filePath": "example.go"
}

Security Features

• URL Validation: Ensures valid API URLs
• Token Format Validation: Validates GitLab token formats
• Enhanced API Key Validation: Comprehensive API key validation including:
  • Length constraints (10-200 characters)
  • Character validation (alphanumeric, hyphens, underscores only)
  • Placeholder/common invalid value detection
  • Required X-API-Key header for all requests
• Secure Credential Storage: Uses n8n's secure credential system

Development

npm install
npm run build
npm run lint

License

MIT

Support

For issues and questions, please visit the GitHub repository.