monday-api-queue-sdk

A module for executing and consuming monday.com API requests at scale using queue-based processing.

Overview

This SDK provides a robust solution for handling high-volume monday.com API requests through a queue-based architecture. It handles API rate limiting, automatic retries with exponential backoff, and callback notifications when requests complete.

Key Features

• Queue-based processing for scalable API request handling
• Built-in retry logic with exponential backoff
• Automatic handling of monday.com rate limits (complexity budget exhaustion, concurrency limits)
• Support for AWS SQS as the message queue
• S3 integration for storing API response payloads
• Callback notifications via HTTP endpoints
• Both long-polling consumers and trigger-based (Lambda) consumers

Installation

npm install @mondaydotcomorg/monday-api-queue-sdk

Architecture

The SDK uses a two-queue architecture:

┌─────────────┐     ┌─────────────────────┐     ┌───────────────┐
│  API Queue  │────▶│  ApiRequestProcessor │────▶│ Callback Queue│
└─────────────┘     └─────────────────────┘     └───────────────┘
                              │                         │
                              ▼                         ▼
                         ┌─────────┐         ┌──────────────────────┐
                         │   S3    │         │ CallbackQueueProcessor│
                         └─────────┘         └──────────────────────┘
                                                        │
                                                        ▼
                                                ┌──────────────┐
                                                │ Your Webhook │
                                                └──────────────┘

1. API Queue: Receives monday.com API requests
2. ApiRequestQueueProcessor: Executes the API call, uploads response to S3, pushes to callback queue
3. Callback Queue: Holds completed requests ready for delivery
4. CallbackQueueProcessor: Delivers results to your webhook endpoint

Quick Start

Using the Factory (Recommended)

The QueueFactory provides a convenient way to create pre-configured processors. It internally creates the necessary producers for retries and pushing results to the callback queue:

import {
  QueueFactory,
  QueueConfigTypes,
  FileStorageTypes,
  TriggerConsumer,
} from "@mondaydotcomorg/monday-api-queue-sdk";

const apiProcessor = QueueFactory.createApiProcessor({
  apiQueueSettings: {
    type: QueueConfigTypes.SQS,
    queueUrl: "https://sqs.us-east-1.amazonaws.com/123456789/api-queue",
    region: "us-east-1",
  },
  callbackQueueSettings: {
    type: QueueConfigTypes.SQS,
    queueUrl: "https://sqs.us-east-1.amazonaws.com/123456789/callback-queue",
    region: "us-east-1",
  },
  fileStorageSettings: {
    type: FileStorageTypes.S3,
    bucketName: "my-api-responses-bucket",
  },
  maxRetries: 3,
  shouldRetryOnGeneralError: true,
});

const callbackProcessor = QueueFactory.createCallbackProcessor({
  queueSettings: {
    type: QueueConfigTypes.SQS,
    queueUrl: "https://sqs.us-east-1.amazonaws.com/123456789/callback-queue",
    region: "us-east-1",
  },
  maxRetries: 3,
  shouldRetryOnGeneralError: true,
});

Lambda / Trigger-Based Consumer

For AWS Lambda or other serverless environments:

import {
  TriggerConsumer,
  parseSqsRecords,
  ApiRequestQueueJob,
} from "@mondaydotcomorg/monday-api-queue-sdk";

export const handler = async (event: SQSEvent) => {
  const consumer = new TriggerConsumer(apiProcessor);
  const jobs = parseSqsRecords<ApiRequestQueueJob>(event.Records);
  
  const result = await consumer.consumeBatch(jobs);
  
  console.log(`Processed: ${result.successful.length} successful, ${result.failed.length} failed`);
};

Enqueueing API Requests

The QueueFactory creates producers internally for retries and callback delivery, but you'll need to create your own SqsProducer to initially enqueue jobs to the API queue:

import { SqsProducer, ApiRequestQueueJob } from "@mondaydotcomorg/monday-api-queue-sdk";

const producer = new SqsProducer({
  queueUrl: "https://sqs.us-east-1.amazonaws.com/123456789/api-queue",
  region: "us-east-1",
});

const job: ApiRequestQueueJob = {
  token: "your-monday-api-token",
  query: `query { boards(limit: 10) { id name } }`,
  variables: {},
  data: {},
};

const result = await producer.enqueue(job);
if (result.success) {
  console.log("Job enqueued with ID:", result.id);
}

API Reference

Queue Jobs

ApiRequestQueueJob

Job structure for API requests:

interface ApiRequestQueueJob {
  token: string;
  query: string;
  variables?: Record<string, unknown>;
  id?: string | number | null;
  attempts?: number;
  data: unknown;
}

CallbackQueueJob

Job structure for callbacks:

interface CallbackQueueJob {
  callbackUrl: string;
  payload: unknown;
  id?: string | number | null;
  attempts?: number;
  data: unknown;
}

Processors

ApiRequestQueueProcessor

Processes monday.com API requests:

• Executes GraphQL queries using the provided token
• Handles rate limiting (COMPLEXITY_BUDGET_EXHAUSTED, MAX_CONCURRENCY_EXCEEDED)
• Uploads responses to S3
• Pushes results to the callback queue
• Automatic retry with exponential backoff

CallbackQueueProcessor

Delivers results to webhook endpoints:

• Makes HTTP POST requests to callback URLs
• Retries on 5xx errors
• Configurable retry behavior

Consumers

TriggerConsumer

For serverless/Lambda environments:

const consumer = new TriggerConsumer(processor);
const result = await consumer.consumeBatch(jobs);

SqsConsumer

Abstract base class for long-polling SQS consumers. Extend this for continuous processing:

class MyConsumer extends SqsConsumer {
  constructor() {
    super(processor, {
      queueUrl: "https://sqs...",
      region: "us-east-1",
    });
  }
}

const consumer = new MyConsumer();
await consumer.start();

Producers

SqsProducer

Enqueue jobs to SQS:

const producer = new SqsProducer({
  queueUrl: "https://sqs.us-east-1.amazonaws.com/123456789/queue",
  region: "us-east-1",
});

await producer.enqueue(job, { delay: 30 });

Utilities

parseSqsRecords

Parse SQS event records into typed jobs:

import { parseSqsRecords, ApiRequestQueueJob } from "@mondaydotcomorg/monday-api-queue-sdk";

const jobs = parseSqsRecords<ApiRequestQueueJob>(event.Records);

Configuration

Queue Settings

interface SQSVendorQueueConfig {
  type: QueueConfigTypes.SQS;
  queueUrl: string;
  region: string;
}

File Storage Settings

interface S3FileStorageConfig {
  type: FileStorageTypes.S3;
  bucketName: string;
}

Processor Options

| Option | Type | Default | Description | |--------|------|---------|-------------| | maxRetries | number | 3 | Maximum retry attempts | | shouldRetryOnGeneralError | boolean | true | Retry on non-monday.com errors |

Error Handling

The SDK includes specialized error types:

MondayError

Thrown when the monday.com API returns errors:

import { MondayError, MondayErrorCodes } from "@mondaydotcomorg/monday-api-queue-sdk";

if (error instanceof MondayError) {
  const hasRateLimit = error.mondayErrors.some(
    (e) => e.extensions?.code === MondayErrorCodes.COMPLEXITY_BUDGET_EXHAUSTED
  );
}

JobError

Thrown during job processing with retry information:

import { JobError } from "@mondaydotcomorg/monday-api-queue-sdk";

if (error instanceof JobError) {
  console.log("Should retry:", error.shouldRetry);
  console.log("Retry delay:", error.options?.delay);
}

AWS Requirements

IAM Permissions

The SDK requires the following AWS permissions:

SQS:

• sqs:SendMessage
• sqs:ReceiveMessage
• sqs:DeleteMessage
• sqs:GetQueueAttributes

S3:

• s3:PutObject

Queue Configuration

Recommended SQS queue settings:

• Visibility Timeout: 60 seconds (adjust based on API response times)
• Message Retention: 4 days
• Dead Letter Queue: Recommended for failed messages

Development

npm install

npm run build

npm run test

npm run test:coverage

npm run lint

npm run typecheck

License

MIT