monday-api-queue-sdk

A module for executing and consuming monday.com API requests at scale using queue-based processing.

Overview

This SDK provides a robust solution for handling high-volume monday.com API requests through a queue-based architecture. It handles API rate limiting, automatic retries with exponential backoff, and callback notifications when requests complete.

Key Features

• Queue-based processing for scalable API request handling
• Built-in retry logic with exponential backoff
• Automatic handling of monday.com rate limits (complexity budget exhaustion, concurrency limits)
• Support for AWS SQS as the message queue
• S3 integration for storing API response payloads
• Callback notifications via HTTP endpoints
• Both long-polling consumers and trigger-based (Lambda) consumers

Installation

npm install @mondaydotcomorg/monday-api-queue-sdk

Architecture

The SDK uses a two-queue architecture:

┌─────────────┐     ┌─────────────────────┐     ┌───────────────┐
│  API Queue  │────▶│  ApiRequestProcessor │────▶│ Callback Queue│
└─────────────┘     └─────────────────────┘     └───────────────┘
                              │                         │
                              ▼                         ▼
                         ┌─────────┐         ┌──────────────────────┐
                         │   S3    │         │ CallbackQueueProcessor│
                         └─────────┘         └──────────────────────┘
                                                        │
                                                        ▼
                                                ┌──────────────┐
                                                │ Your Webhook │
                                                └──────────────┘

1. API Queue: Receives monday.com API requests
2. ApiRequestQueueProcessor: Executes the API call, uploads response to S3, pushes to callback queue
3. Callback Queue: Holds completed requests ready for delivery
4. CallbackQueueProcessor: Delivers results to your webhook endpoint

Quick Start

Using the Factory (Recommended)

The QueueFactory provides a convenient way to create pre-configured processors. It internally wires the necessary producers for retries and callback queue delivery.

import {
  QueueFactory,
  QueueConfigTypes,
  FileStorageTypes,
} from "@mondaydotcomorg/monday-api-queue-sdk";

const apiProcessor = QueueFactory.createApiProcessor({
  apiQueueSettings: {
    type: QueueConfigTypes.SQS,
    queueUrl: "https://sqs.us-east-1.amazonaws.com/123456789/api-queue",
    region: "us-east-1",
  },
  callbackQueueSettings: {
    type: QueueConfigTypes.SQS,
    queueUrl: "https://sqs.us-east-1.amazonaws.com/123456789/callback-queue",
    region: "us-east-1",
  },
  fileStorageSettings: {
    type: FileStorageTypes.S3,
    bucketName: "my-api-responses-bucket",
  },
  maxRetries: 3,
  shouldRetryOnGeneralError: true,
});

const callbackProcessor = QueueFactory.createCallbackProcessor({
  queueSettings: {
    type: QueueConfigTypes.SQS,
    queueUrl: "https://sqs.us-east-1.amazonaws.com/123456789/callback-queue",
    region: "us-east-1",
  },
  maxRetries: 3,
  shouldRetryOnGeneralError: true,
});

Lambda / Trigger-Based Consumer

For AWS Lambda or other serverless environments:

import {
  TriggerConsumer,
  parseSqsRecords,
  ApiRequestQueueJob,
} from "@mondaydotcomorg/monday-api-queue-sdk";
import { SQSEvent } from "aws-lambda";

export const handler = async (event: SQSEvent) => {
  const consumer = new TriggerConsumer(apiProcessor);
  const jobs = parseSqsRecords<ApiRequestQueueJob>(event.Records);

  const result = await consumer.consumeBatch(jobs);

  console.log(
    `Processed: ${result.successful.length} successful, ` +
    `${result.requeued.length} requeued, ${result.failed.length} failed`
  );
};

Enqueueing API Requests

Create your own SqsProducer to enqueue jobs to the API queue:

import { SqsProducer, ApiRequestQueueJob } from "@mondaydotcomorg/monday-api-queue-sdk";

const producer = new SqsProducer({
  queueUrl: "https://sqs.us-east-1.amazonaws.com/123456789/api-queue",
  region: "us-east-1",
});

const job: ApiRequestQueueJob = {
  token: "your-monday-api-token",
  query: `query { boards(limit: 10) { id name } }`,
  variables: { limit: 10 },
  callbackUrl: "https://your-service.example.com/webhook",
  metadata: { requestId: "abc-123" }, // forwarded as-is in the callback payload
};

const result = await producer.enqueue(job);
if (result.success) {
  console.log("Job enqueued with ID:", result.id);
}

API Reference

Queue Jobs

ApiRequestQueueJob

Job structure for API requests:

interface ApiRequestQueueJob {
  token: string;                         // monday.com API token
  query: string;                         // GraphQL query string
  variables?: Record<string, unknown>;   // GraphQL variables
  callbackUrl?: string;                  // Webhook URL for result delivery
  metadata?: Record<string, unknown>;    // Arbitrary data forwarded in callback payload
  headers?: Record<string, string>;      // HTTP headers forwarded to the callback endpoint
  id?: string | number | null;           // Job identifier (set automatically from SQS MessageId)
  attempts?: number;                     // Retry attempt count (managed by the processor)
}

CallbackQueueJob

Job structure for callbacks:

interface CallbackQueueJob {
  callbackUrl: string;              // Webhook URL to POST results to
  payload: unknown;                 // Callback payload (includes S3 reference, status, query context)
  headers?: Record<string, string>; // HTTP headers forwarded to the callback endpoint
  id?: string | number | null;
  attempts?: number;
}

Callback Payload

When CallbackQueueProcessor delivers a result to your webhook, the POST body has the following shape:

interface CallbackPayload {
  result: { fileUrl: string };           // S3 URL of the full API response
  status: "success" | "partial";        // "partial" when some errors occurred alongside data
  token: string;                         // Original API token
  query: string;                         // Original GraphQL query
  variables?: Record<string, unknown>;   // Original GraphQL variables
  metadata?: Record<string, unknown>;    // Forwarded from the original ApiRequestQueueJob
}

Partial results

When a monday.com API call returns errors alongside partial data (e.g. one field rate-limited while others succeed), the processor forwards the partial result to your webhook rather than dropping it. In this case status is "partial" and result.fileUrl points to the S3 object containing the partial response, including the errors array.

Your webhook should handle both statuses:

app.post("/webhook", (req, res) => {
  const { status, result, metadata } = req.body;

  if (status === "partial") {
    // Partial data was returned — inspect result.fileUrl for the errors array
    console.warn("Partial result", metadata?.requestId);
  }

  // Fetch the full response from S3 using result.fileUrl
});

Processors

ApiRequestQueueProcessor

Processes monday.com API requests:

• Executes GraphQL queries against the monday.com API
• Handles rate limiting (COMPLEXITY_BUDGET_EXHAUSTED, MAX_CONCURRENCY_EXCEEDED) with bypassRetryLimit — these retries are not capped by maxRetries
• Forwards partial results (non-retryable errors with data present) to the callback queue rather than retrying
• Ensures CursorException errors reach the DLQ so cursor-recovery logic can run
• Uploads full responses to S3, then pushes a reference to the callback queue
• Exponential backoff on retryable errors; uses retry_in_seconds from the API when available

CallbackQueueProcessor

Delivers results to webhook endpoints:

• Makes HTTP POST requests to callback URLs
• Retries on 5xx server errors, 429 (Too Many Requests), and 423 (Locked) with exponential backoff
• Does not retry on 4xx client errors (except 429/423)
• Retries on network errors (no HTTP status)

Consumers

TriggerConsumer

For serverless/Lambda environments where messages are pushed to your handler:

const consumer = new TriggerConsumer(processor);
const result = await consumer.consumeBatch(jobs);
// result: { successful: string[], requeued: string[], failed: string[] }

SqsConsumer

Abstract base class for long-polling SQS consumers. Extend this for continuous processing:

class MyApiConsumer extends SqsConsumer {
  constructor() {
    super(processor, {
      queueUrl: "https://sqs.us-east-1.amazonaws.com/123456789/api-queue",
      region: "us-east-1",
    });
  }
}

const consumer = new MyApiConsumer();
await consumer.start();
// ...
await consumer.stop();

Producers

SqsProducer

Enqueue jobs to SQS. Supports an optional delay (in seconds, clamped to the SQS maximum of 900):

const producer = new SqsProducer({
  queueUrl: "https://sqs.us-east-1.amazonaws.com/123456789/queue",
  region: "us-east-1",
});

await producer.enqueue(job, { delay: 30 }); // deliver after 30 seconds

Utilities

parseSqsRecords

Parse SQS event records into typed jobs. The parsed body fields are spread to the top level and id is set from the SQS MessageId:

import { parseSqsRecords, ApiRequestQueueJob } from "@mondaydotcomorg/monday-api-queue-sdk";

const jobs = parseSqsRecords<ApiRequestQueueJob>(event.Records);

Configuration

Queue Settings

interface SQSVendorQueueConfig {
  type: QueueConfigTypes.SQS;
  queueUrl: string;
  region: string;
}

File Storage Settings

interface S3FileStorageConfig {
  type: FileStorageTypes.S3;
  bucketName: string;
}

Processor Options

| Option | Type | Default | Description | |--------|------|---------|-------------| | maxRetries | number | 3 | Maximum retry attempts for capped errors | | shouldRetryOnGeneralError | boolean | true | Retry on non-monday.com errors (network timeouts, etc.) |

Note: Rate-limit signals (COMPLEXITY_BUDGET_EXHAUSTED, MAX_CONCURRENCY_EXCEEDED, 429, 423) bypass maxRetries and retry indefinitely until the rate limit clears.

Error Handling

The SDK includes specialized error types:

MondayError

Wraps errors returned by the monday.com API:

import { MondayError, MondayErrorCodes } from "@mondaydotcomorg/monday-api-queue-sdk";

if (error instanceof MondayError) {
  const hasRateLimit = error.mondayErrors?.some(
    (e) => e.extensions?.code === MondayErrorCodes.COMPLEXITY_BUDGET_EXHAUSTED
  );
  console.log("Partial data:", error.partialData);
}

JobError

Thrown internally during job processing to signal retry behavior. You typically won't need to construct this directly, but you can inspect it from a custom processor:

import { JobError } from "@mondaydotcomorg/monday-api-queue-sdk";

if (error instanceof JobError) {
  console.log("Should retry:", error.shouldRetry);
  console.log("Retry delay (seconds):", error.options?.delay);
  console.log("Bypass max retries:", error.bypassRetryLimit);
}

Extending the SDK

All core components are designed for extension. The base classes (BaseCallMondayApiTask, BaseFileUploadTask, BaseCallEndpointTask, BaseQueueProcessor) can be subclassed to swap in custom implementations — for example, a non-S3 file storage backend or a mock API task for testing.

import { BaseCallMondayApiTask, CallEndpointResult } from "@mondaydotcomorg/monday-api-queue-sdk";

class MockCallMondayApiTask extends BaseCallMondayApiTask {
  async executeMondayRequest(token, query, variables): Promise<CallEndpointResult> {
    // custom implementation
  }
}

AWS Requirements

IAM Permissions

SQS:

• sqs:SendMessage
• sqs:ReceiveMessage
• sqs:DeleteMessage
• sqs:GetQueueAttributes

S3:

• s3:PutObject

Queue Configuration

Recommended SQS settings:

| Setting | Recommended value | Reason | |---------|------------------|--------| | Visibility Timeout | 60 s | Must exceed maximum API response time | | Message Retention | 4 days | Allows time for manual intervention on failures | | Dead Letter Queue | Required | Captures jobs that exhaust receive count | | Receive Count (DLQ redrive) | 5–10 | Allows natural retry exhaustion before DLQ routing |

Important: CursorException errors from paginated queries are intentionally routed to the DLQ. Your DLQ consumer should detect these and re-issue the query with a fresh cursor.

Development

npm install

npm run build

npm run test

npm run test:coverage

npm run typecheck

npm run lint

License

MIT