ICGYAD - I Can Get You A Date

Automated appointment booking engine for high-demand government services in Uruguay

🎯 Overview

ICGYAD is a sophisticated automation system designed to detect and book appointments for government services with limited availability and brief release windows. The system monitors appointment availability, solves security challenges using AI, and automatically notifies users or books appointments when slots become available.

🔐 NEW: API Key Authentication System

ICGYAD now includes a comprehensive API key authentication system for secure access control:

• API Key Management: Create, list, and revoke API keys
• Strategy Permissions: Control which strategies each API key can access
• Public/Private Strategies: Configure strategies as public or require authentication
• Usage Tracking: Monitor API key usage and access patterns
• Enterprise Ready: Designed for integration into third-party APIs and services

Initial Use Case

• Service: Inscripción de partidas extranjeras - Registro Civil Uruguay
• Platform: SAE (Sistema de Agenda Electrónica)
• Target: Sede Uruguay 933 appointments that are released during night hours (typically 20:45-00:00)

🚀 Features

• 🔐 API Key Authentication: Secure access control with comprehensive permission management
• Multi-Strategy Engine: Extensible framework for different government services
• Intelligent Detection: Multiple availability detection methods (DOM, XHR, hidden inputs, API interception)
• AI-Powered Security: Automated resolution of security questions using LLM
• Smart Notifications: Email alerts with detailed availability information + immediate notifications
• Auto-Booking: Optional automatic appointment confirmation
• Robust Scheduling: Configurable time windows and intervals
• Comprehensive Logging: Full audit trail with screenshots and performance metrics
• Stealth Mode: Anti-detection measures for reliable operation
• 📦 Library Integration: Can be used as NPM package for API integration

🆕 Recent Updates (v2.1 - Enhanced Retry Loop & CLI Parameters)

New CLI Parameters & Testing Features

• ⏱️ --delay Parameter: Configure retry delay in seconds (default: 300)
  • Fast testing: --delay 30 for 30-second intervals
  • Custom intervals: --delay 120 for 2-minute delays
  • Respects server load while allowing flexible timing
• 📧 --notify-always Flag: Test notification system without waiting for real slots
  • Sends email notifications even when no appointments available
  • Perfect for testing email delivery and notification configuration
  • Validates notification pipeline functionality

Enhanced Uruguay Strategy (v2.1)

• 🔄 Improved Retry Loop: Internal retry logic with configurable delays
  • Robust cycle: Security Question → Calendar → Availability Check → Volver → Delay
  • Consistent availability detection between calendar navigation and polling
  • Proper "Volver" button navigation using specific href patterns
  • Comprehensive error handling and logging for debugging
• 🔍 Stable Form Selectors: Using name attributes for maximum reliability
  • select[name="titular_pais"], input[name="titular_nombres"], input[name="titular_apellidos"]
  • select[name="TipoDocumento"], input[name="NroDocumento"]:not([disabled])
• 🔐 Security Question Detection: label[id^="lblpreg_"] → input[name="pregunta"]
• 📍 Location & Schedule: Role-based selectors for "Box 1 y 2" and "Cualquier horario"
• ⚡ Real-time API Interception: agenda_sae_api_disponibilidades endpoint monitoring
• 🚨 Immediate Notifications: Instant alerts upon availability detection (hasSlots === true)
• 🎯 Enhanced Availability Detection:
  • input[name="sin_fechas_disponibles"] (no slots signal)
  • input[name="fechas_disponibles"] (available dates)
  • input[name="horas_disponibles_chk"] (time slot radios)
• ✅ Booking Confirmation: Extract num_serie_reserva, cod_cancelacion_reserva, cod_trazabilidad_reserva

📋 Requirements

• Node.js: 18.0.0 or higher
• Database: MongoDB (local or cloud)
• Email Service: SendGrid, AWS SES, or SMTP
• AI Service: OpenAI, Google Gemini, or Anthropic Claude

🛠 Installation

1. Clone the repository

git clone <repository-url>
cd icgyad

2. Install dependencies

npm install

3. Install Playwright browsers

npx playwright install chromium

4. Configure environment

cp .env.example .env
# Edit .env with your configuration

5. Build the application

npm run build

⚙️ Configuration

Required Environment Variables

Database

DATABASE_URL=mongodb://localhost:27017/icgyad

AI Service (choose one)

GENAI_PROVIDER=openai
OPENAI_API_KEY=your_openai_api_key

Email Service (choose one)

# SendGrid
MAIL_PROVIDER=sendgrid
SENDGRID_API_KEY=your_sendgrid_api_key
[email protected]

# AWS SES
MAIL_PROVIDER=ses
AWS_ACCESS_KEY_ID=your_aws_key
AWS_SECRET_ACCESS_KEY=your_aws_secret
AWS_REGION=us-east-1
[email protected]

# SMTP
MAIL_PROVIDER=smtp
SMTP_HOST=smtp.gmail.com
SMTP_PORT=587
[email protected]
SMTP_PASS=your_app_password
[email protected]

Optional Configuration

# Browser settings
BROWSER_HEADLESS=true
BROWSER_TIMEOUT=30000
MAX_ATTEMPTS_PER_NIGHT=50

# Feature flags
ENABLE_AUTO_BOOKING=false
ENABLE_SCREENSHOTS=true
ENABLE_HAR_CAPTURE=true

# Notification settings
NOTIFY_ON=availability  # availability | booking | both

# API Configuration
ICGYAD_API_URL=http://localhost:3000  # Base URL for API key validation endpoint

📖 Usage

🔐 API Key Authentication

Note: API key management (creation, listing, revocation) is handled directly in the database by administrators. The CLI only validates API keys through the REST API endpoint.

Using API Keys with Strategies

To run a strategy with API key authentication:

# Run a private strategy with API key
npm run start:prod run UruguayInscripcionDePartidasExtrangerasRegistroCivil \
  --api-key "your-api-key-here" \
  --user-data ./user-data.json

# Run a public strategy (no API key required)
npm run start:prod run PublicStrategyId \
  --user-data ./user-data.json

API Key Validation Flow

1. Private Strategies: Require a valid API key that has access to the specific strategy
2. Public Strategies: Can be executed without an API key
3. Validation: API keys are validated through the REST API endpoint GET /:apiKey/scopes

CLI Commands

1. List Available Strategies

npm run start:prod list

2. Generate User Data Template

# For all strategies
npm run start:prod template

# For specific strategy
npm run start:prod template UruguayInscripcionDePartidasExtrangerasRegistroCivil

3. Run Strategy (Public - No API Key Required)

# If strategy is configured as public
npm run start:prod run UruguayInscripcionDePartidasExtrangerasRegistroCivil \
  --user-data ./user-data.json \
  --dry-run \
  --headful

4. Run Strategy (With API Key Authentication)

# For private strategies or API key validation
npm run start:prod run UruguayInscripcionDePartidasExtrangerasRegistroCivil \
  --api-key "icgyad_abc123def456..." \
  --user-data ./user-data.json \
  --dry-run \
  --headful

5. Run Strategy (Scheduled with API Key)

npm run start:prod run UruguayInscripcionDePartidasExtrangerasRegistroCivil \
  --api-key "icgyad_abc123def456..." \
  --window "20:45-00:00" \
  --interval 5 \
  --user-data ./user-data.json \
  --auto-book

5. Run Strategy (Custom Retry Delay)

# Fast retry for testing (30 seconds between attempts)
npm run start:prod run UruguayInscripcionDePartidasExtrangerasRegistroCivil \
  --user-data ./user-data.json \
  --delay 30 \
  --headful

# Custom retry delay (2 minutes between attempts)
npm run start:prod run UruguayInscripcionDePartidasExtrangerasRegistroCivil \
  --user-data ./user-data.json \
  --delay 120 \
  --auto-book

6. Test Notifications (Development)

# Test notification system - sends email even when no slots found
npm run start:prod run UruguayInscripcionDePartidasExtrangerasRegistroCivil \
  --user-data ./user-data.json \
  --delay 30 \
  --notify-always \
  --headful

7. Check Run Status

npm run start:prod status <run-id>

User Data Format

Create a user-data.json file with your information:

{
  "titular_pais": "UY",
  "titular_nombres": "Juan Carlos",
  "titular_apellidos": "García López",
  "tipo_partida": "PN",
  "TipoDocumento": "CI",
  "NroDocumento": "12345678",
  "apellidos": "García López", 
  "nombres": "Juan Carlos",
  "Mail": "[email protected]",
  "Telefono": "099123456",
  "gestoria": false,
  "email": "[email protected]"
}

📦 Library Integration (for API Development)

You can import ICGYAD as a library to integrate appointment booking into your own applications:

import { StrategyRunnerService, AuthService, AppModule } from 'icgyad';
import { NestFactory } from '@nestjs/core';

// Initialize the application
const app = await NestFactory.createApplicationContext(AppModule);
const strategyRunner = app.get(StrategyRunnerService);
const authService = app.get(AuthService);

// Create API key for your application
const { apiKey, keyId } = await authService.createApiKey({
  name: 'My App Integration',
  issuer: 'myapp',
  allowedStrategies: ['UruguayInscripcionDePartidasExtrangerasRegistroCivil']
});

// Run strategy with API key authentication
const runId = await strategyRunner.runStrategy({
  strategyId: 'UruguayInscripcionDePartidasExtrangerasRegistroCivil',
  apiKey: apiKey,
  window: { start: '20:45', end: '00:00', intervalMin: 5 },
  userData: {
    titular_pais: 'UY',
    titular_nombres: 'Juan Carlos',
    // ... other required fields
  },
  options: {
    dryRun: false,
    autoBook: true,
    notifyOn: 'availability'
  }
});

console.log(`Strategy execution started: ${runId}`);

🔧 Command Line Options

| Option | Description | Example | Default | |--------|-------------|---------|---------| | --api-key | 🔐 NEW API key for authentication | --api-key "icgyad_abc123..." | None (checks if strategy is public) | | --window | Time window for checking (HH:mm-HH:mm) | --window "20:45-00:00" | One-time execution | | --interval | Check interval in minutes | --interval 5 | 5 minutes | | --dry-run | Test mode (no actual booking) | --dry-run | false | | --headful | Show browser window | --headful | false | | --max-attempts | Maximum attempts per night | --max-attempts 50 | 10 | | --auto-book | Enable automatic booking | --auto-book | false | | --delay | Retry delay in seconds | --delay 30 | 300 (5 minutes) | | --notify-always | Send test notifications on no slots | --notify-always | false | | --user-data | Path to user data file | --user-data ./data.json | Strategy-specific file | | --config | Path to configuration file | --config ./config.json | - | | --api-key | 🔐 API key for authentication | --api-key "your-key" | Required for private strategies |

🆕 New Parameters (v2.1)

--delay <seconds>

• Purpose: Configure retry delay between attempts in the Uruguay strategy
• Usage: --delay 30 (30 seconds), --delay 300 (5 minutes)
• Benefits:
  • ⚡ Fast testing: Use --delay 30 for quick development testing
  • 🔄 Custom intervals: Adjust retry frequency based on your needs
  • ⚖️ Server respect: Avoid overwhelming the target server

--notify-always

• Purpose: Send email notifications even when no slots are found (for testing notification system)
• Usage: --notify-always (flag, no value needed)
• Benefits:
  • 📧 Test email delivery without waiting for real availability
  • 🔧 Debug notification service configuration
  • ✅ Verify email templates and formatting
  • 🚀 Quick validation of notification pipeline

📊 Monitoring

Logs and Screenshots

• Logs: ./logs/ directory with structured JSON logs
• Screenshots: ./logs/screenshots/<run-id>/ for debugging
• HAR Files: ./logs/traces/ for network analysis

Database Collections

• runs: Execution records
• attempts: Step-by-step attempt logs
• availabilities: Detected appointment slots
• bookings: Successful reservations
• notifications: Email delivery records

🔐 Security Considerations

API Key Security

1. 🔑 API Key Storage: Store API keys securely, never commit to version control
2. 🔄 Key Rotation: Regularly rotate API keys and revoke unused ones
3. 📊 Usage Monitoring: Monitor API key usage patterns for anomalies
4. 🛡️ Permissions: Follow principle of least privilege - only grant necessary strategy access
5. ⏰ Expiration: Set appropriate expiration dates for API keys

General Security

1. Environment Variables: Never commit sensitive data to version control
2. Rate Limiting: Respects reasonable intervals to avoid detection
3. User Agents: Rotates browser fingerprints
4. Terms of Service: Review and comply with website terms
5. Legal Compliance: Ensure compliance with local automation laws

API Key Best Practices

# ✅ DO: Store API keys in environment variables
export ICGYAD_API_KEY="icgyad_abc123def456..."

# ✅ DO: Use different API keys for different environments
export ICGYAD_API_KEY_DEV="icgyad_dev_..."
export ICGYAD_API_KEY_PROD="icgyad_prod_..."

# ❌ DON'T: Hardcode API keys in source code
const apiKey = "icgyad_abc123..."; // NEVER DO THIS

# ✅ DO: Revoke compromised keys immediately
npm run start:prod revoke-key --key-id "compromised_key_id"

🎨 Architecture

CLI Interface / Library API
    ↓
🔐 API Key Authentication
    ↓
Strategy Runner
    ↓
┌─────────────────────────────────────────────────────┐
│                Strategy Engine                       │
├─────────────────────────────────────────────────────┤
│ • AuthService (API Key Management & Validation)    │
│ • PlaywrightService (Browser Automation)           │
│ • GenAISolverService (Security Questions)          │  
│ • AvailabilityDetector (Multi-method Detection)    │
│ • NotifierService (Email Notifications)            │
└─────────────────────────────────────────────────────┘
    ↓
Database (MongoDB)
├── Strategies (with permissions)
├── API Keys (with usage tracking)
├── Runs & Attempts
└── Notifications & Bookings

🔄 Adding New Strategies

1. Create Strategy Class

@Strategy({
  id: 'YourNewStrategy',
  name: 'Your Service Name',
  version: '1.0.0',
  enabled: true
})
export class YourNewStrategy implements IStrategy {
  // Implement required methods
}

2. Register in AppModule

this.strategyFactory.register(YourNewStrategy, metadata);

3. Configure Detection Rules

private readonly detectionConfig: DetectionConfig = {
  noSlotsText: ['No appointments available'],
  availabilitySelectors: ['.available-slot'],
  // ... other selectors
};

🚀 Deployment

Local Development

npm run start:dev

Production

# Build
npm run build

# Start
npm run start:prod

# Or with PM2
pm2 start ecosystem.config.js

Docker

# Build image
docker build -t icgyad .

# Run container
docker run -d \
  --name icgyad \
  -v $(pwd)/.env:/app/.env \
  -v $(pwd)/logs:/app/logs \
  icgyad

Cloud Deployment

Google Cloud Run

# Deploy as a job
gcloud run jobs create icgyad \
  --image gcr.io/PROJECT/icgyad \
  --region us-central1 \
  --set-env-vars DATABASE_URL=... 

AWS Lambda (with Container)

# Package and deploy
aws lambda create-function \
  --function-name icgyad \
  --package-type Image \
  --code ImageUri=ACCOUNT.dkr.ecr.region.amazonaws.com/icgyad:latest

🧪 Testing

Unit Tests

npm test

End-to-End Tests

npm run test:e2e

Manual Testing

# Test with dry-run mode (default 5-minute delays)
npm run start:prod run UruguayInscripcionDePartidasExtrangerasRegistroCivil \
  --dry-run \
  --headful \
  --user-data ./test-data.json

# Fast testing with 30-second retry delays
npm run start:prod run UruguayInscripcionDePartidasExtrangerasRegistroCivil \
  --dry-run \
  --headful \
  --delay 30 \
  --user-data ./test-data.json

# Test notification system (sends emails even on no slots)
npm run start:prod run UruguayInscripcionDePartidasExtrangerasRegistroCivil \
  --dry-run \
  --headful \
  --delay 30 \
  --notify-always \
  --user-data ./test-data.json

🔍 Troubleshooting

Common Issues

"Strategy not found"

• Check strategy ID spelling
• Run npm run start:prod list to see available strategies

"Invalid user data"

• Generate template: npm run start:prod template <strategy-id>
• Verify all required fields are filled

"Browser timeout"

• Increase BROWSER_TIMEOUT in environment
• Check internet connection and website availability

"Security question failed"

• Verify AI service API key is correct
• Check AI service credits/quota

"Email notification failed"

• Verify email service configuration
• Check spam folder for test emails

Debug Mode

# Run with debug logging
LOG_LEVEL=debug npm run start:prod run ...

Logs Analysis

# View structured logs
tail -f logs/app.log | jq '.'

# Filter by run ID
grep "run-id-here" logs/app.log | jq '.'

📚 API Reference

Strategy Interface

interface IStrategy {
  bootstrap(ctx: ExecutionContext): Promise<void>;
  fillForms(ctx: ExecutionContext, userData: Record<string, unknown>): Promise<void>;
  solveSecurityCheck(ctx: ExecutionContext, solver: SecurityQuestionSolver): Promise<void>;
  reachCalendar(ctx: ExecutionContext): Promise<void>;
  pollAvailability(ctx: ExecutionContext): Promise<AvailabilityResult>;
  book(ctx: ExecutionContext, slot: Slot, userData: Record<string, unknown>): Promise<BookResult>;
  notify(ctx: ExecutionContext, event: NotificationEvent): Promise<void>;
}

🤝 Contributing

1. Fork the repository
2. Create a feature branch
3. Add tests for new functionality
4. Ensure all tests pass
5. Submit a pull request

📄 License

MIT License - see LICENSE file for details

⚠️ Disclaimer

This software is provided for educational and personal use only. Users are responsible for:

• Complying with website terms of service
• Respecting rate limits and robot policies
• Following local laws regarding automation
• Using the system ethically and responsibly

The authors are not responsible for any misuse or consequences arising from the use of this software.

📞 Support

• Issues: Use GitHub Issues for bug reports
• Discussions: Use GitHub Discussions for questions
• Email: For security issues only

Made with ❤️ for the Uruguay community