🚀 Cadenlane CLI - Multi-Instance Image Download System

A production-ready, cross-platform TypeScript CLI application that processes image downloads from a Supabase PGMQ queue using event-driven architecture. Supports multiple distributed instances across Windows, macOS, and Linux with comprehensive monitoring, health tracking, and real-time observability.

✅ 310+ Tests Passing | ⚡ Event-Driven | 🌐 Cross-Platform | 🔄 Multi-Instance | 🏗️ Clean Architecture | 🔒 Type-Safe | 📊 Full Observability | 🤖 CI/CD Automated

🚀 Quick Navigation

| Task | Section | |------|---------| | Install the CLI | Installation | | Setup configuration | Global Configuration Setup | | Run the application | Running the Application | | Run with PM2 | Run with PM2 (Production) | | Auto-start on boot | Auto-Start on System Boot | | View all features | Features | | Read documentation | Documentation |

📦 Installation

Install globally via npm:

npm install -g cadenlane-cli

Or use directly with npx:

npx cadenlane-cli start

🔧 Global Configuration Setup

After installation, create a global configuration file:

# Create config directory
mkdir -p ~/.cadenlane-cli

# Create your .env file
cat > ~/.cadenlane-cli/.env << EOF
SUPABASE_URL=https://your-project.supabase.co
SUPABASE_SERVICE_ROLE_KEY=your-key-here
QUEUE_NAME=image_downloads
EOF

The CLI automatically searches for .env files in this order:

1. ~/.cadenlane-cli/.env (Recommended for global installation)
2. ./.env (Current working directory)
3. <installation-dir>/.env (npm installation directory)

See Configuration Guide for more details.

🚀 Running the Application

Option 1: Run Directly (Simple)

Run the CLI directly from any directory:

# Start the worker
cadenlane-cli start

# With custom timeout
cadenlane-cli start --timeout 10

# Check version
cadenlane-cli --version

# View help
cadenlane-cli --help

The CLI will automatically load configuration from ~/.cadenlane-cli/.env.

Option 2: Run with PM2 (Production)

For production deployments with auto-restart and monitoring:

Quick Start

macOS/Linux:

# 1. Create logs directory
mkdir -p ~/.cadenlane-cli/logs

# 2. Start with PM2 using bundled config
pm2 start $(npm root -g)/cadenlane-cli/pm2.config.js

# 3. View status
pm2 status

# 4. View logs
pm2 logs cadenlane-worker

Windows (PowerShell):

# 1. Create logs directory
New-Item -ItemType Directory -Force -Path "$env:USERPROFILE\.cadenlane-cli\logs"

# 2. Get npm root and start with PM2
$npmRoot = npm root -g
pm2 start "$npmRoot\cadenlane-cli\pm2.config.js"

# 3. View status
pm2 status

# 4. View logs
pm2 logs cadenlane-worker

Windows (Command Prompt):

REM 1. Create logs directory
mkdir "%USERPROFILE%\.cadenlane-cli\logs"

REM 2. Start with PM2 (replace path with your npm root)
pm2 start "C:\Users\%USERNAME%\AppData\Roaming\npm\node_modules\cadenlane-cli\pm2.config.js"

REM 3. View status
pm2 status

REM 4. View logs
pm2 logs cadenlane-worker

Auto-Start on System Boot

macOS/Linux:

# Save PM2 process list
pm2 save

# Generate and configure startup script
pm2 startup

# Follow the instructions shown (may require sudo)

Windows:

# Install pm2-windows-startup globally
npm install -g pm2-windows-startup

# Configure startup
pm2-startup install

# Start your worker
pm2 start $(npm root -g)/cadenlane-cli/pm2.config.js

# Save PM2 process list
pm2 save

Now the worker will automatically start when Windows boots!

Useful PM2 Commands

# View status
pm2 status

# View logs (live)
pm2 logs cadenlane-worker

# Monitor dashboard
pm2 monit

# Restart
pm2 restart cadenlane-worker

# Stop
pm2 stop cadenlane-worker

# Remove from PM2
pm2 delete cadenlane-worker

See PM2 Guide for complete instructions and advanced configuration.

✨ Features

🌐 Cross-Platform Support

• Windows, macOS, Linux: Single codebase runs on all platforms
• Platform-Aware Paths: Automatic validation for drive letters (Windows) and Unix paths
• Path Resolution: Home directory expansion (~, %USERPROFILE%), environment variables
• Absolute Paths Only: Clear, unambiguous file locations
• Reserved Name Detection: Windows reserved names (CON, PRN, etc.) automatically blocked

🔄 Multi-Instance Architecture

• Distributed Processing: Run multiple CLI instances across different computers
• Instance Registration: Automatic registration with unique identity
• Health Monitoring: Heartbeat system tracks instance health (5-minute intervals)
• Workload Tracking: Per-instance statistics (jobs processed, failures, bytes downloaded)
• Configuration Hierarchy: Global config with instance-specific overrides
• Platform Distribution: Track workload across Windows, Mac, and Linux instances

Core Functionality

• 🖼️ Image Download: Downloads images from any public URL to specified folders
• 📬 Queue-Based Processing: Uses Supabase PGMQ for reliable, scalable message handling
• ⚡ Event-Driven: Blocking reads eliminate polling - instant message processing
• 🚀 Fail-Fast Design: No retry mechanism - immediate, clear failure handling
• 📊 Complete Observability: All logs and metrics stored in Supabase
• 🎯 Single Processing: Each message processed exactly once with ACID guarantees

Architecture & Quality

• 🏗️ Clean Architecture: SOLID principles with clear layer separation
• 💉 Dependency Injection: TSyringe IoC container for testability
• 🧪 Test-Driven Development: 278+ tests with 85%+ coverage
• 🔒 Type-Safe: Full TypeScript with strict mode
• 🛡️ Security: Path traversal protection, input validation, RLS policies

Admin UI

• 🎨 Web Dashboard: Real-time monitoring and management interface
• 📈 Live Statistics: Track completed, processing, and failed jobs
• 📝 Detailed Logs: View execution logs with expandable data and stack traces
• 🔄 Auto-Refresh: Real-time updates every 3 seconds
• ✉️ Queue Management: Send test messages directly from the UI

📚 Documentation

This README provides a quick overview. Explore the comprehensive documentation in the docs/ folder:

🚀 Getting Started

• 📖 Setup Guide - Complete installation and configuration
• ⚡ Quick Start - Get running in 5 minutes (see below)
• 🎨 Admin UI Guide - Web dashboard usage

🏗️ Architecture & Design

• 🏗️ Architecture - System design and patterns
• 🧩 Components - Detailed component documentation
• 📐 Architecture Decisions - ADRs and design rationale
• ⚙️ Configuration - Configuration management guide

🌐 Cross-Platform & Multi-Instance

• 🌐 Cross-Platform Guide - Windows, macOS, Linux support
• 🔄 Multi-Instance Architecture - Distributed CLI instances
• 📊 Configuration Hierarchy - Global vs instance-specific config

🧪 Testing & Quality

• 🧪 Testing Guide - Complete testing documentation (278+ tests)
• 🔬 E2E Testing - End-to-end test guide
• ✅ Test Coverage - Detailed coverage breakdown

🔧 Operations & Troubleshooting

• 🔧 Troubleshooting - Common issues and solutions
• 🚀 PM2 Process Manager - Production deployment with PM2
• 📋 PRD - Full product requirements document
• 🚀 Deployment - Production deployment guide

📦 Publishing & CI/CD

• 📦 Publishing Guide - Manual npm publishing guide
• 🤖 CI/CD Setup - Automated GitHub Actions workflows
• 🔄 Auto-Versioning - Semantic versioning automation
• 🚀 Supabase Deployment - Automated Supabase migrations and functions deployment

📖 Quick Navigation

| I want to... | See documentation | |--------------|-------------------| | Install the CLI | Installation | | Get started quickly | Quick Start | | Understand the architecture | Architecture | | Run on Windows/Mac/Linux | Cross-Platform Guide | | Run multiple instances | Multi-Instance Architecture | | Write and run tests | Testing Guide | | Configure the application | Configuration | | Deploy to production | Deployment Guide | | Publish to npm | Publishing Guide | | Setup CI/CD automation | CI/CD Setup | | Deploy Supabase changes | Supabase Deployment | | Troubleshoot issues | Troubleshooting | | Use the Admin UI | Admin UI Guide |

⚡ Quick Start

Prerequisites

• Node.js 18+
• Docker (for local Supabase)
• Supabase CLI (npm install -g supabase)

1. Clone and Install

git clone <repository-url>
cd cadenlane-cli
npm install

2. Start Local Supabase

supabase start

This will output connection details including:

• API URL: http://127.0.0.1:54321
• Database URL: postgresql://postgres:[email protected]:54322/postgres
• Studio URL: http://127.0.0.1:54323
• Service Role Key: (copy this for .env)

3. Configure Environment

Copy .env.example to .env and update with your local Supabase credentials:

cp .env.example .env

Update .env with the service role key from supabase start output.

4. Run the Worker

# Development mode (event-driven, instant processing)
npm run dev

# Or with custom timeout
npm run dev -- --timeout 10

# Production mode
npm run build
npm start

The worker now uses event-driven blocking reads - it waits for messages instead of polling, providing instant processing with zero CPU waste.

5. Access the Admin UI

Start the Admin UI Edge Function:

supabase functions serve admin-ui --no-verify-jwt

Then open: http://127.0.0.1:54321/functions/v1/admin-ui

✨ You're ready! Send messages via the UI and watch them process in real-time.

🗄️ Database Schema

The application uses five main components:

📬 PGMQ Queue (image_downloads)

• Message queue for reliable job distribution
• ACID guarantees with PostgreSQL
• Visibility timeout for processing isolation

📊 Messages Table

Complete audit trail for all jobs:

• Status Flow: received → processing → completed/failed
• Timing: Received, started, completed timestamps
• Metrics: File size, download duration
• Errors: Message, category, stack trace
• Instance Tracking: Which instance processed each message

📝 Job Logs Table

Detailed execution logs:

• Levels: INFO, WARN, ERROR, DEBUG
• Structured Data: JSONB for complex data
• Stack Traces: Full error context
• Linked: Foreign key to messages table

🖥️ CLI Instances Table (New)

Tracks distributed CLI instances:

• Identity: Instance name, hostname, platform
• Paths: Instance-specific download paths
• Health: Heartbeat timestamps, status
• Statistics: Jobs processed, failures, bytes downloaded

⚙️ App Config Table (Enhanced)

Configuration management:

• Global Config: Applies to all instances
• Instance Config: Per-instance overrides
• Priority: Instance-specific > Global > Environment > Default

See ARCHITECTURE.md and MULTI_INSTANCE.md for detailed schema and relationships.

Project Structure

src/
├── cli/                    # CLI Layer
├── application/            # Application Layer (Services, Handlers, Managers)
├── domain/                 # Domain Layer (Entities, DTOs, Interfaces, Validators)
├── infrastructure/         # Infrastructure Layer (Repositories, Adapters)
└── shared/                 # Shared utilities

tests/
├── unit/                   # Unit tests
├── integration/            # Integration tests
├── e2e/                    # End-to-end tests
├── fixtures/               # Test data
└── mocks/                  # Mock implementations

Development

Running Tests

# All tests
npm test

# Unit tests only
npm run test:unit

# Integration tests
npm run test:integration

# E2E tests
npm run test:e2e

# Watch mode
npm run test:watch

# Coverage report
npm run test:coverage

Code Quality

# Lint
npm run lint

# Format
npm run format

# Type check
npm run type-check

Usage

Sending Messages to Queue

Via Admin UI (Recommended):

1. Open http://127.0.0.1:54321/functions/v1/admin-ui
2. Fill in the form and click "Send to Queue"

Via curl:

# Using the Admin UI API endpoint
curl -X POST http://127.0.0.1:54321/functions/v1/admin-ui/api/send-message \
  -H "Content-Type: application/json" \
  -d '{
    "image_url": "https://picsum.photos/200/300",
    "target_folder": "/downloads"
  }'

Via Supabase Client:

const { data, error } = await supabase.rpc('pgmq_send_message', {
  queue_name: 'image_downloads',
  msg: {
    image_url: 'https://example.com/image.png',
    target_folder: '/path/to/downloads'
  }
});

Via SQL:

SELECT pgmq_send_message(
  'image_downloads',
  '{"image_url": "https://example.com/image.png", "target_folder": "/downloads"}'::jsonb
);

Monitoring

View logs and message status in Supabase Studio:

• Open http://127.0.0.1:54323
• Navigate to Table Editor
• Check messages and job_logs tables

Querying Messages

-- Recent messages
SELECT * FROM messages ORDER BY received_at DESC LIMIT 10;

-- Failed messages
SELECT * FROM messages WHERE status = 'failed';

-- Messages with logs
SELECT m.*, COUNT(l.id) as log_count
FROM messages m
LEFT JOIN job_logs l ON m.id = l.message_id
GROUP BY m.id;

Configuration

Environment variables (see .env.example):

| Variable | Required | Default | Description | |----------|----------|---------|-------------| | SUPABASE_URL | Yes | - | Supabase project URL | | SUPABASE_SERVICE_ROLE_KEY | Yes | - | Service role key for auth | | CLI_INSTANCE_NAME | No | Auto-generated | Unique instance identifier | | DOWNLOAD_DEFAULT_PATH | No | Platform default | Default download directory (absolute path) | | DOWNLOAD_FALLBACK_PATH | No | - | Fallback download directory | | QUEUE_NAME | No | image_downloads | PGMQ queue name | | MESSAGES_TABLE | No | messages | Messages table name | | LOGS_TABLE | No | job_logs | Logs table name | | DOWNLOAD_TIMEOUT_MS | No | 30000 | Download timeout | | MAX_FILE_SIZE_MB | No | 50 | Maximum file size | | LOG_LEVEL | No | INFO | Logging level |

Configuration Priority:

1. Database (instance-specific)
2. Database (global)
3. Environment variables
4. Platform defaults

See CONFIGURATION.md and MULTI_INSTANCE.md for detailed configuration guide.

Error Handling

The application follows a fail-fast approach:

• No retries: Messages are processed once
• Immediate deletion: Messages deleted after processing (success or failure)
• Clear categorization: Errors categorized as network, filesystem, validation, or unknown
• Comprehensive logging: All errors logged to Supabase with stack traces

🏗️ Architecture

The application follows Clean Architecture principles with clear layer separation:

┌─────────────────────────────────────────┐
│           CLI Layer                     │
│  (Commands, User Interface)             │
└──────────────┬──────────────────────────┘
               │
┌──────────────▼──────────────────────────┐
│      Application Layer                  │
│  (Services, Use Cases, Handlers)        │
└──────────────┬──────────────────────────┘
               │
┌──────────────▼──────────────────────────┐
│         Domain Layer                    │
│  (Entities, DTOs, Interfaces,           │
│   Validators, Business Rules)           │
└──────────────┬──────────────────────────┘
               │
┌──────────────▼──────────────────────────┐
│     Infrastructure Layer                │
│  (Repositories, Adapters, DB, HTTP)     │
└─────────────────────────────────────────┘

Key Principles

• SOLID: All five principles applied consistently
• Dependency Inversion: Interfaces define contracts, implementations are injected
• Separation of Concerns: Each layer has a single, well-defined responsibility
• Test-Driven Development: Tests written before implementation
• Fail-Fast: No retries, immediate error handling with clear categorization

See ARCHITECTURE.md for detailed design decisions and patterns.

🧪 Testing

Comprehensive test suite with 310+ passing tests:

# All tests
npm test

# Unit tests (266 tests)
npm run test:unit

# E2E tests (44 tests)
npm run test:e2e

# Coverage report
npm run test:coverage

# Watch mode
npm run test:watch

Test Coverage:

• ✅ 266 Unit Tests: All passing
  • Cross-Platform: 63 tests (PathResolverService, PathValidator)
  • Multi-Instance: 38 tests (CliInstanceEntity, CliInstanceRepository)
  • Core Application: 165 tests (Services, Repositories, Validators, Entities, DTOs)
• ✅ 44 E2E Tests: All passing
  • Multi-Instance workflows (16 tests)
  • Application workflows (7 tests)
  • Full system workflows (6 tests)
  • Configuration workflows (15 tests)
• ✅ Coverage: 85%+ across all layers
• ✅ Platform Mocking: Test Windows, Mac, Linux on any platform
• ✅ Integration: Real Supabase + PGMQ + Worker

See TESTING.md for complete testing guide with detailed coverage breakdown.

🛠️ Development

Code Quality

# Lint
npm run lint

# Format
npm run format

# Type check
npm run type-check

# Build
npm run build

Project Structure

cadenlane-cli/
├── src/
│   ├── cli/                 # CLI commands and interface
│   ├── application/         # Business logic and services
│   ├── domain/              # Core domain models and rules
│   └── infrastructure/      # External integrations
├── tests/
│   ├── unit/                # Unit tests
│   ├── e2e/                 # End-to-end tests
│   └── helpers/             # Test utilities
├── supabase/
│   ├── migrations/          # Database migrations
│   └── functions/           # Edge Functions (Admin UI)
└── docs/                    # Documentation

See COMPONENTS.md for detailed component documentation.

🔧 Troubleshooting

Common issues and solutions:

Worker Not Processing Messages

• ✅ Check worker is running: ps aux | grep "node dist"
• ✅ Verify queue has messages: SELECT * FROM pgmq.q_image_downloads;
• ✅ Check logs: SELECT * FROM job_logs ORDER BY created_at DESC;

Admin UI Not Loading

• ✅ Verify Edge Function is running: supabase functions serve admin-ui
• ✅ Check URL: http://127.0.0.1:54321/functions/v1/admin-ui
• ✅ Check browser console for errors

Downloads Failing

• ✅ Verify URL is accessible: curl -I <image-url>
• ✅ Check error category in messages.error_category
• ✅ Review stack trace in job_logs

See TROUBLESHOOTING.md for comprehensive troubleshooting guide.

📦 Deployment

Local Development

supabase start
npm run dev

Production

# Link to production project
supabase link --project-ref <your-project-ref>

# Push migrations
supabase db push

# Deploy Edge Functions
supabase functions deploy admin-ui

# Build and run worker (use process manager like PM2)
npm run build
pm2 start dist/src/index.js --name cadenlane-cli -- start

# Or with custom timeout
pm2 start dist/src/index.js --name cadenlane-cli -- start --timeout 10

🤝 Contributing

1. Follow TDD: Write tests first
2. Maintain 90%+ coverage
3. Follow SOLID principles
4. Update documentation
5. Run all tests before committing

📄 License

MIT License - see LICENSE file for details

🆘 Support

• 📚 Documentation: Explore the comprehensive docs/ folder
• 📦 npm Package: https://www.npmjs.com/package/cadenlane-cli
• 🧪 Testing: See TESTING.md for test guide
• 🌐 Cross-Platform: Check CROSS_PLATFORM.md for platform support
• 🔄 Multi-Instance: See MULTI_INSTANCE.md for distributed setup
• 🤖 CI/CD: See CICD_SETUP.md for automation setup
• 📊 Logs: Review Supabase Studio or Admin UI
• 🔧 Issues: Check TROUBLESHOOTING.md
• 📋 PRD: See PRD-CLI-BASE.md for requirements

🎯 Key Highlights

• ✅ 310+ Tests: Comprehensive test coverage with unit and E2E tests
• ⚡ Event-Driven: Blocking reads for instant processing, zero polling overhead
• 🌐 Cross-Platform: Single codebase for Windows, macOS, and Linux
• 🔄 Multi-Instance: Run distributed CLI instances across multiple computers
• 📊 Health Monitoring: Real-time instance health tracking with heartbeats
• ⚙️ Flexible Config: Global config with instance-specific overrides
• 🏗️ Clean Architecture: SOLID principles with clear layer separation
• 🔒 Type-Safe: Full TypeScript with strict mode
• 📈 Observable: Complete logging and metrics in Supabase
• 🤖 CI/CD Automated: Auto-publish to npm with semantic versioning
• 📦 npm Published: Install globally with npm install -g cadenlane-cli

Built with ❤️ using TypeScript, Supabase, and Clean Architecture principles

Event-Driven Architecture | Windows, macOS, and Linux | Multi-Instance | 310+ Tests Passing | CI/CD Automated