clever-queue

Purpose

Queuing system for promises that handle concurring, throttling, weighting and prioritizing in a clever fashion ?

Zero dependancies.

Features

• [X] Promises
• [X] Concurrency running
• [X] Priority & BestEffort queues
• [X] Weighted Queue
• [ ] Throttling

Architecture - clever-queue implements 4 classes :

• engine : this is the starting point of clever-queue. You must instantiate at least one engine in your program. You may instantiate multiple engines if needed.
• task : this is what we would like to execute. It may be an async or a sync function
• runner : It is in charge of the execution of tasks. It runs tasks, one by one. You must have at least one runner in an engine (one best-effort runner is automatically created on engine instantiation). If you need concurrency running, you may instantiate as many runners as needed. You may configure each runner in a way that it will only execute tasks coming from queues that have an equal or higher priority than runner priority.
• queue : It is in charge of keeping tasks in a deterministic order (FIFO : First In, First Out by default), waiting for an available runner. You must have at least one queue in an engine. You may instantiate as many queues you need and manage priorities and weights between queues.

import * as cleverQueue from "clever-queue"; // import the library - no dependancies

const engine = cleverQueue.createEngine(); // create and start (by default) engine with a single best effort runner (by default)
const queue = engine.createQueue(); // create your first queue, with standard priority (by default) and default weight (by default) 

const myAsyncTaskToExecute: cleverQueue.tasks.FunctionToExecute = async function (message: string, timeout: number) {
    // do your stuff here - In this case, just wait for the timeout and return the message
    await new Promise((resolve) => setTimeout(resolve, timeout));
    return message;
};

(async () => {
    const result = await queue.createTaskAndEnqueue(() => myAsyncTaskToExecute("myValue", 1000));
    console.log(result);
    engine.stop(); // stop the engine when you have finished
})();

Examples

1 x Queue / 1 x Runner / 4 x Tasks

  const engine = cleverQueue.createEngine();
  const queueA = engine.createQueue();

1 x Queue / 2 x Runners / 4 x Tasks

  const engine = cleverQueue.createEngine();
  engine.createRunner({ priority: cleverQueue.queues.Priorities.BestEffort });
  const queueA = engine.createQueue();

2 x Queues / 1 x Runner / 8 x Tasks

  const engine = cleverQueue.createEngine();
  const queueA = engine.createQueue();
  const queueB = engine.createQueue();