@pidgeyjs/redis

Redis adapter for Pidgey (powered by BullMQ) - a high-performance Redis-based job queue.

Installation

pnpm add @pidgeyjs/redis bullmq ioredis

Features

• ✅ Composite job IDs for reliable queue tracking
• ✅ Lazy connection (connects on first use)
• ✅ Compatible with BullMQ dashboard tools (arena, bull-board)
• ✅ High throughput (~16,000+ jobs/sec per worker)

Usage

Basic Setup

import { PidgeyClient } from '@pidgeyjs/core';
import { RedisQueueAdapter, RedisWorkerAdapter } from '@pidgeyjs/redis';

// Create client with Redis adapter
const client = new PidgeyClient(
  new RedisQueueAdapter({
    host: 'localhost',
    port: 6379,
  })
);

await client.connect();

// Define and register jobs
const emailJob = client.defineJob({
  name: 'send-email',
  handler: async (data: { to: string; subject: string }) => {
    console.log(`Sending email to ${data.to}`);
    return { sent: true };
  },
  config: {
    retries: 3,
    timeout: 30000, // 30 seconds
  },
});

// Enqueue a job
await emailJob.enqueue({
  to: '[email protected]',
  subject: 'Welcome!',
});

// Start worker
const worker = new RedisWorkerAdapter({
  host: 'localhost',
  port: 6379,
});

await worker.start(
  {
    'send-email': emailJob.handler,
  },
  {
    concurrency: 10,
  }
);

// Graceful shutdown
await worker.stop();
await client.disconnect();

With Delayed Jobs

// Enqueue job to run in 5 minutes
await emailJob.enqueue({ to: '[email protected]', subject: 'Reminder' }, { delay: 5 * 60 * 1000 });

Multiple Queues

const emailJob = client.defineJob({
  name: 'send-email',
  handler: emailHandler,
  config: { queue: 'emails' },
});

const reportJob = client.defineJob({
  name: 'generate-report',
  handler: reportHandler,
  config: { queue: 'reports' },
});

// Worker will process both queues
await worker.start(
  {
    emails: emailJob.handler,
    reports: reportJob.handler,
  },
  { concurrency: 5 }
);

Implementation Details

Powered by BullMQ

This adapter uses BullMQ under the hood, giving you access to Redis-level performance with Pidgey's simple API.

Composite Job IDs

Job IDs use the format {queueName}:{jobId} (e.g., emails:abc123). This ensures reliable job-to-queue association and makes debugging easier.

Lazy Connection

The adapter connects to Redis automatically when first needed - no upfront connection test is performed. This follows BullMQ's design pattern.

Metadata Handling

Internal Pidgey metadata (like __pidgey_timeout) is stored in job data but is automatically stripped before being passed to your job handlers.

Differences from Database Adapters

Unlike @pidgeyjs/postgres and @pidgeyjs/sqlite, the Redis adapter:

• No Migrations: Redis is schema-less, so no migration system is needed
• Job Management: Redis handles job claiming, completion, and retries internally
• Worker Pattern: Uses Redis's Worker API instead of database polling
• No Direct Job Methods: Methods like claimJobs(), completeJob(), etc. are not exposed (handled by Redis internally)
• Composite IDs: Job IDs include the queue name (queue:id) for proper tracking

Configuration

Redis Options

The adapter accepts standard ioredis connection options:

new RedisQueueAdapter({
  host: 'localhost',
  port: 6379,
  password: 'your-password',
  db: 0,
  maxRetriesPerRequest: null, // Required for BullMQ
  tls: {
    // TLS options for secure connections
  },
});

Worker Options

await worker.start(handlers, {
  concurrency: 10, // Number of jobs to process in parallel
  queues: ['queue1', 'queue2'], // Optional: specific queues to process
});

Testing

Integration tests require Docker for Redis testcontainers:

# Start Docker, then run tests
pnpm test

License

MIT