💠 Soft Queue - Simple Job Queue System (MongoDB + Node.js)

A lightweight job queue system using MongoDB and Mongoose. Useful for handling background processing like sending notifications, emails, etc., without needing Redis or RabbitMQ.

📦 Features

• 🆕 Create named queues dynamically backed by MongoDB
• 🧠 Register custom consumers (job processors) for each queue
• 🚀 Start job processing on-demand per queue
• 📬 Add (enqueue) jobs with flexible options like delays or priorities
• 📡 Check and fetch queues by name with metadata
• ✅ Track job status using unique job IDs
• 📊 Count jobs in a queue using filters (e.g., by status)
• 📦 Retrieve all jobs from a specific queue
• 🧹 Clear individual queues or purge all queues
• 🗂 List all queues along with their jobs
• ❌ Remove specific queues and their jobs permanently
• ⛔ Deactivate queues to pause processing
• 🔁 Activate queues to resume processing
• ⚙️ MongoDB-powered: persistent, scalable, and fault-tolerant
• 🔌 Lightweight & plug-and-play with minimal setup

⚙️ Setup

1. Install Dependencies

npm install mongoose uuid

2. Start MongoDB

Make sure MongoDB is running on your system:

mongod

3. Install soft-queue

npm install soft-queue

🚀 Usage

✅ index.js

const Queue = require("./Queue");
const { init } = require("./db/dbConfig");

const main = async () => {
    await init({
        uri: 'mongodb://localhost:27017/soft-jobQueueDB',
        options: {
            useNewUrlParser: true,
            useUnifiedTopology: true,
        }
    });

    // Create a new queue
    const queue = await Queue.createQueue('notifications');
    console.log({ queue });

    // Register a consumer for the 'notifications' queue
    Queue.registerConsumer('notifications', async (job) => {
        console.log(`🔔 Sending notification: ${job.message}`);
        await new Promise(resolve => setTimeout(resolve, 500)); // simulate delay
        console.log(`✅ Notification sent: ${job.message}`);
    });

    // Add jobs to the queue
    await Queue.enqueue('notifications', { message: 'Welcome to our service!' });
    await Queue.enqueue('notifications', { message: 'Your account has been created.' });
    await Queue.enqueue('notifications', { message: 'Your password has been changed.' });

    // Count pending jobs
    const count = await Queue.getJobCount('notifications');
    console.log({ count });

    // Start processing jobs in the queue
    Queue.startProcessing('notifications');
};

main();

📂 API Reference

createQueue(queueName, options)

Create a new queue.

registerConsumer(queueName, async (job) => { ... })

Registers a function to handle jobs in a queue.

enqueue(queueName, jobData, options)

Adds a new job to a queue.

startProcessing(queueName)

Begins processing the queue using the registered consumer.

getJobStatus(queueName, jobId)

Get status of a specific job.

getJobCount(queueName, filters)

Count jobs in a queue with optional filters.

getJobs(queueName)

Retrieve all jobs in a queue.

clearQueue(queueName)

Delete all jobs in a specific queue.

clearAllQueues()

Remove all queues and their jobs.

getAllQueues()

Get all queues with their jobs.

removeQueue(queueName)

Permanently delete a queue.

deactivateQueue(queueName)

Pause job processing for a queue.

activateQueue(queueName)

Resume job processing for a queue

🧪 Sample Output

{ queue: { queueId: ..., queueName: 'notifications', ... } }
🔔 Sending notification: Welcome to our service!
✅ Notification sent: Welcome to our service!
🔔 Sending notification: Your account has been created.
✅ Notification sent: Your account has been created.
🔔 Sending notification: Your password has been changed.
✅ Notification sent: Your password has been changed.
{ count: 3 }