🛰️ Orbital

The ultimate distributed task scheduler for Node.js

🚀 Quick Start

Installation

# Using pnpm (recommended)
pnpm add @mini256/orbital

# Using npm
npm install @mini256/orbital

# Using yarn
yarn add @mini256/orbital

Basic Usage

import { Orbital } from '@mini256/orbital';

// Initialize the scheduler
const scheduler = new Orbital({
  redis: 'redis://localhost:6379',
  database: 'mysql://localhost/orbital'
});

// Define a task handler
scheduler.define('email.send', async (job) => {
  const { to, subject, body } = job.data;
  await sendEmail(to, subject, body);
  return { sent: true, timestamp: new Date() };
});

// Enqueue a task
const jobId = await scheduler.enqueue('email.send', {
  to: '[email protected]',
  subject: 'Hello from Orbital',
  body: 'This is a distributed task!'
});

// Schedule a recurring job
scheduler.schedule('daily.report', '0 0 * * *', async () => {
  await generateDailyReport();
});

// Start workers
await scheduler.start();

✨ Features

| Feature | Description | |---------|-------------| | 🚀 High Performance | 50k+ TPS, Redis-powered queue | | 🔒 Distributed | Multi-worker cluster, no single point of failure | | 💾 Persistent | Redis + TiDB dual-write architecture | | ⏰ Scheduling | Cron, delayed, priority queues | | 🔍 Observable | Prometheus metrics, Grafana dashboards | | 📘 TypeScript | Full type definitions, strict mode | | 🔌 Plugin System | Custom workers and backends | | 📊 Metrics | Built-in Prometheus exporters |

📚 Documentation

• Getting Started - Installation and basic setup
• Core Concepts - Architecture and design
• API Reference - Complete API documentation
• Deployment Guide - Production deployment
• Worker Guide - Building custom workers

🎯 Use Cases

• Data Pipelines - ETL workflows, data synchronization (like OSS Insight)
• Background Jobs - Email sending, image processing, notifications
• Cron Scheduling - Periodic tasks, reports, cleanup jobs
• Workflow Orchestration - Multi-step business processes
• Distributed Processing - Parallel task execution across workers

🏗️ Architecture

┌─────────────┐     ┌─────────────┐
│   Client    │────▶│   Redis     │
│             │     │  (Queue)    │
└─────────────┘     └──────┬──────┘
                           │
                    ┌──────▼──────┐
                    │   Worker    │
                    │   Cluster   │
                    └──────┬──────┘
                           │
                     ┌─────▼──────┐
                     │    TiDB    │
                     │ (Persistent)│
                     └────────────┘

Components

• Client - Enqueues tasks and schedules jobs
• Redis - High-speed task queue and distributed locking
• Workers - Process tasks from the queue
• TiDB - Persistent storage for task history and state

🛠️ Development

Prerequisites

• Node.js >= 18.0.0
• pnpm >= 9.0.0
• Redis >= 6.0
• MySQL/TiDB >= 8.0

Setup

# Clone the repository
git clone https://github.com/pingcap/ossinsight.git
cd orbital

# Install dependencies
pnpm install

# Copy environment configuration
cp .env.example .env

# Set up the database
pnpm prisma migrate dev

# Run development server
pnpm dev

# Run tests
pnpm test

# Build for production
pnpm build

Commands

| Command | Description | |---------|-------------| | pnpm dev | Start development server with hot reload | | pnpm build | Compile TypeScript to JavaScript | | pnpm test | Run test suite | | pnpm test:watch | Run tests in watch mode | | pnpm test:coverage | Run tests with coverage report | | pnpm lint | Run oxlint | | pnpm fmt | Format code with oxfmt | | pnpm check | Run all checks (format, lint, type) | | pnpm worker | Start a worker process |

📦 Package Exports

// Main entry point
import { Orbital } from '@mini256/orbital';

// Core modules
import { Queue, Worker, Scheduler, Task } from '@mini256/orbital/core';

// Adapters
import { RedisAdapter, TiDBAdapter } from '@mini256/orbital/adapters';

// Workers
import { DataImportWorker } from '@mini256/orbital/workers';

🔧 Configuration

Environment Variables

| Variable | Description | Default | |----------|-------------|---------| | REDIS_URL | Redis connection string | redis://localhost:6379 | | DATABASE_URL | MySQL/TiDB connection string | mysql://localhost/orbital | | WORKER_CONCURRENCY | Number of concurrent jobs | 10 | | LOG_LEVEL | Logging level | info | | METRICS_PORT | Prometheus metrics port | 9090 |

See .env.example for full configuration options.

📊 Performance

Benchmarks (Redis 7.0, Node.js 20, M1 Mac):

| Operation | Throughput | Latency (p99) | |-----------|------------|---------------| | Enqueue | 55,000 ops/s | 2ms | | Dequeue | 52,000 ops/s | 3ms | | Complete | 48,000 ops/s | 4ms | | Schedule | 10,000 ops/s | 5ms |

🤝 Contributing

Contributions are welcome! Please follow these steps:

1. Fork the repository
2. Create a feature branch (git checkout -b feature/amazing-feature)
3. Commit your changes (git commit -m 'Add amazing feature')
4. Push to the branch (git push origin feature/amazing-feature)
5. Open a Pull Request

Please read our Contributing Guide for details.

📄 License

Apache-2.0 © OSS Insight Team

🙏 Acknowledgments

• Built with Vite+ - The unified toolchain
• Powered by Redis - In-memory data structure store
• Uses Prisma - Next-generation ORM
• Metrics by Prometheus - Monitoring system

Made with ❤️ by the OSS Insight Team

Report Issue · Request Feature