Trison

A TypeScript based Synchronous multilevel queue.

Table of Contents

• Usecase
• Installing
• STTunnel
• ConditionalTunnel
• Messages
• Running Asynchronous Tasks
• Generics
• License
• API

Usecase

Have you come across a use case where you have to run some tasks in parallel, and some in sequence with respect to others. One of the cases can be in a chat app. In any chatting app, you send messages to multiple people and for a particular person, messages must be in the same order. So what we want is a queue for every person. The solution seems very simple but maintaining many queues is difficult and this is where Trison will help it will create queues for you and process tasks automatically in sequence without any extra code. One more advantage of using Trison is less 3rd party dependencies, Trison only requires 3 dependencies, and these may also be removed in a future versions.

Installing

Using npm:

$ npm install trison

Docs

In Trison you have to first initialize a multilevel Queue, and sub-queues are called Tunnels and there are two types of Tunnels. Type parameter tells queue which type of data can be processed in this queue

  import Queue from "trison"; 
  let newMultiLevelQueue = new Queue<object>(
    true // Optional  autoCreateTunnels; this will create a tunnel if tunnel with a particular id is not present
    true // Optional  createWorkerForAutoCreatedTunnels: whether to create workers for automatic created tunnels
    true // Optional  autoCreateProcessorFunction: default processor function for autocreated tunnels
  ); // initialize Queue, all paramters are optional

STTunnel

STTunnels are identified by their unique string id, which can be provided by the user or can be automatically created using UUID. There are various ways to create a STTunnel

Using Queue object

  import Queue from "trison";
  import ReadOnlyMessage from "trison/Messages/ReadOnlyMessage";
  let newMultiLevelQueue = new Queue(); // initialize Queue 
  //  type ProcessorFunction = (readOnlyMessage: ReadOnlyMessage) => Promise<ReadOnlyMessage>;
  let processorFunction = async (message: ReadOnlyMessage<object>) => {
    let extractedData = message.getData();
    extractedData["processed"] = true;
    return new ReadOnlyMessage<object>(message);
  }
  let tunnelCreated: Tunnel<object> = newMultiLevelQueue.createSTTunnelWithId(
      processorFunction, 
      "uuid", // unique id of tunnel, must be unique accross STTunnels
      false // withWorker: if true, it will create a worker that will start processing message automatically
  );

With a preprocessor function (This processor function will run before inserting message into a tunnel)

  ...
  // type PreProcessorFunction = (readOnlyMessage: ReadOnlyMessage) => ReadOnlyMessage
  let preProcessorFunction = (message: ReadOnlyMessage<object>) => {
    let extractedData: object = message.getData();
    extractedData["preProcessed"] = true;
    return new ReadOnlyMessage<object>(message);
  }
  let tunnel: Tunnel = newMultiLevelQueue.createSTTunnelWithPreProcessor(
      processorFunction,
      "uuid",
      preProcessorFunction, 
      false
  );

Creating STTunnel directly from class constructor

    import STTunnel from "trison/tunnels/STTunnel";
    let sTTunnel = new STTunnel<object>(
        processorFunction,
        "uuid",
        undefined, // optinal preprocessor function
        true // withWorker
    )

While creating a tunnel you have to provide ProcessorFunction, which will process every message pushed in the tunnel.

ConditionalTunnel

A conditional tunnel as the name implies the message will be pushed in the tunnel only if a condition is met. For this condition, you have to provide a matcher function that will take the message and return true or false. A current message is checked iteratively and search will stop at first tunnel which return true

Using Queue

  ...
  // type MatcherFunction = (readOnyMessage: ReadOnlyMessage) => boolean;
  let matcherFunction1 = (message: ReadOnlyMessage<object>) => {
    let data = message.getData();
    return data && data["tunnel"] && data["tunnel"] === "tunnel1";
  }
  let tunnel: Tunnel<object> = newMultiLevelQueue.createConditionalTunnelWithPreProcessor(
      matcherFunction1, // this function will be used to match the message with tunnel
      processorFunction,
      preProcessorFunction,
      false //  withWorker
  );

Creating ConditionalTunnels directly from class constructor

 import ConditionalTunnel from "trison/tunnels/ConditionalTunnel";
 let conditionalTunnel: ConditionalTunnel<object> = new ConditionalTunnel<object>(
        processorFunction,
        matchFunction,
        tunnelId,
        undefined, // optional preProcessor
        withWorker // optioal
 );

Messages

There are two types of messages, one is Message and the other ReadOnlyMessage. User can only create Message but ReadOnlyMessage can be extracted from it. Every message has a unique id and assigned automatically when you create a new message. Id will change if you clone Message class but will not change if you create a ReadOnlyMessage from id. ReadOnlyMessage Id will not change on cloning.

    import Message from "trison/Messages/Message";
    let data = {
        userId: "lk3kj3kj3kj3k3jk3j",
        text: "Hello Testing"
    }
    // type CallbackFunction = (message: ReadOnlyMessage) => any;
    let writeableMessage: Message<object> = new Message<object>(
          {...data}, // data to be passed to the processor function
          () => {}, // CallbackFunction, this function will be called when this message is processed
          2 // priority of message, currenlty it is not being used 
      );

Inserting message in a tunnel

Message with same id can exist in same tunnel. So you can process a message again and again (a Feature that i am thinking of)

  ...
  // for STTunnel and ConditionTunnel
  // you cannot assign ID to ConditionalTunnel but you can get Id using the getter function 
  newMultiLevelQueue.offerMessageForTunnelId(
    message, // Message
    "uuid" // unique Id
   );
   
    // will match message with conditional tunnels only, using matcher function
   newMultiLevelQueue.offer(message)

One more way to add messages is directly using Tunnel object. On creating a tunnel from any method, Tunnel object is returned and you can use that object directly to push messages into it.

 import STTunnel from "trison/tunnels/STTunnel";
 let sTTunnel = new STTunnel<object>(
    processorFunction,
    "uuid",
    undefined, // optional preprocessor function
    true // withWorker
 );
 sTTunnel.addMessage(message)
 let tunnel: Tunnel<object> = newMultiLevelQueue.createConditionalTunnelWithPreProcessor(
    matcherFunction1, // this function will be used to match the message with tunnel
    processorFunction,
    preProcessorFunction,
    false //  withWorker
 );
 tunnel.addMessage(message)

Poll message from tunnel

  ...
  let polledMessage: ReadOnlyMessage<object> = newMultiLevelQueue.poll(tunnel.getTunnelId());
  

Workers

Trison provide workers which can automatically start processing messages. To use workers, you only have to set withWorker parameter true while creating a tunnel. If you don't use a worker you have to process messages yourself by polling messages

Running Asynchronous Tasks

Return type of processor function is of type Promise<ReadOnlyMessage>, so If you want to perform some async task in processor function, and you want task to complete before processing next message, you have to resolve the promise accordingly. You can also use async/await syntax as shown below.

   let processorFunction = async (message: ReadOnlyMessage<object>) => {
    let extractedData = message.getData();
    let data = await axios({
      method: 'post',
      url: '/user/12345',
      data: {...extractedData}
    });
    return new ReadOnlyMessage<object>(message);
  }

Generics

In trison you can have your custom data type and use that in the queue. Keep in mind that you can only process messages of the type specified during queue creation. To process multiple type of data in single queue use new Queue<any>

    class MyClass{
        userId: string = "32832094823904"
        text: string = "hello"
        processed: boolean = false
    }
    let writeableMessage = new Message<MyClass>(
        new MyClass(),
        callbackFunction,
        2
    );

    let processorFunction = async (message: ReadOnlyMessage<MyClass>) => {
        let extractedData = message.getData();
        extractedData.processed = true;
        return new ReadOnlyMessage<MyClass>(message);

    }
    
    let multiLevelQueue = new Queue<MyClass>();

    let tunnelCreated = multiLevelQueue.createSTTunnelWithId(
        processorFunction,
        myUUID,
        false
    );
    
    let readOnlyMessage = multiLevelQueue.offerMessage(
        writeableMessage,
        tunnelCreated
    );

License

[MIT]

API

cooming soon !