MCG HTTP Plugin

A comprehensive HTTP client plugin for Managed Cyclic Graph (MCG) that enables nodes to make HTTP requests with advanced features like retries, timeouts, transformations, and more.

Features

• ✅ All HTTP Methods: GET, POST, PUT, DELETE, PATCH, HEAD, OPTIONS
• ✅ Template Variables: Dynamic URL resolution using context data (${ctx.data.userId})
• ✅ Query Parameters: Automatic URL query string building
• ✅ Custom Headers: Full control over request headers including authentication
• ✅ Request/Response Transformations: Transform data before sending and after receiving
• ✅ Retry Logic: Configurable retries with exponential backoff
• ✅ Timeout Support: Prevent hanging requests with configurable timeouts
• ✅ Response Validation: Validate responses before accepting them
• ✅ Smart Body Parsing: Automatic content-type detection (JSON, text, blob)
• ✅ Error Handling: Comprehensive error tracking and metadata
• ✅ Context Integration: Store responses in custom paths within the graph context

Installation

npm install @quarry-systems/mcg-http

Basic Usage

Basic Example

Use createHttpAction to add HTTP requests to your graph nodes:

import { ManagedCyclicGraph } from '@quarry-systems/managed-cyclic-graph';
import { createHttpAction, httpGet } from '@quarry-systems/mcg-http';

const graph = new ManagedCyclicGraph()
  .node('fetchData', {
    label: 'Fetch Data',
    execute: [
      createHttpAction('fetchData', httpGet('https://api.example.com/data'))
    ]
  })
  
  .node('end', { label: 'End', isEndpoint: true })
  
  .edge('fetchData', 'end', 'any')
  .start('fetchData')
  .build();

How it works:

• createHttpAction(nodeId, config) creates an Action that performs the HTTP request
• The nodeId determines where the response is stored: data.http.{nodeId}
• The action reuses the plugin's core httpnode handler internally
• You can mix HTTP actions with other actions in the same node's execute array

1. Register the Plugin

import { ManagedCyclicGraph } from '@quarry-systems/managed-cyclic-graph';
import { mcgHttpPlugin } from '@quarry-systems/mcg-http';

const graph = new ManagedCyclicGraph('myGraph')
  .use(mcgHttpPlugin)
  // ... rest of your graph configuration

2. Create an HTTP Node

graph.node('fetchUser', {
  label: 'Fetch User Data',
  meta: {
    http: {
      url: 'https://api.example.com/users/123',
      method: 'GET'
    }
  }
});

3. Access the Response

The response is automatically stored in the context at data.http.{nodeId}.response:

// Response structure:
{
  status: 200,
  ok: true,
  statusText: 'OK',
  headers: { ... },
  body: { ... } // Parsed response body
}

Advanced Usage

POST Request with Body

import { httpPost } from '@quarry-systems/mcg-http';

graph.node('createUser', {
  label: 'Create New User',
  meta: {
    http: httpPost('https://api.example.com/users', {
      name: 'John Doe',
      email: '[email protected]'
    })
  }
});

Using Context Data in Requests

graph.node('fetchUserOrders', {
  label: 'Fetch User Orders',
  meta: {
    http: {
      url: 'https://api.example.com/users/${data.userId}/orders',
      method: 'GET',
      queryParams: {
        limit: 10,
        status: 'active'
      }
    }
  }
});

Authentication

import { httpGet, withAuth } from '@quarry-systems/mcg-http';

graph.node('fetchProtectedData', {
  label: 'Fetch Protected Data',
  meta: {
    http: withAuth(
      httpGet('https://api.example.com/protected'),
      'your-token-here',
      'Bearer'
    )
  }
});

Retry Logic

import { httpGet, withRetry } from '@quarry-systems/mcg-http';

graph.node('fetchWithRetry', {
  label: 'Fetch with Retry',
  meta: {
    http: withRetry(
      httpGet('https://api.example.com/data'),
      3,      // Number of retries
      1000    // Delay between retries (ms)
    )
  }
});

Request Transformation

graph.node('transformedRequest', {
  label: 'Transform Request',
  meta: {
    http: {
      url: 'https://api.example.com/data',
      method: 'POST',
      bodyFromPath: 'data.rawInput',
      transformRequest: (body, ctx) => {
        // Transform the request body before sending
        return {
          ...body,
          timestamp: Date.now(),
          userId: ctx.data.userId
        };
      }
    }
  }
});

Response Validation

graph.node('validatedResponse', {
  label: 'Validated Response',
  meta: {
    http: {
      url: 'https://api.example.com/data',
      method: 'GET',
      validateResponse: (response, ctx) => {
        if (!response.ok) {
          return `Request failed with status ${response.status}`;
        }
        if (!response.body || typeof response.body !== 'object') {
          return 'Invalid response format';
        }
        return true; // Validation passed
      }
    }
  }
});

Custom Response Storage

graph.node('customStorage', {
  label: 'Custom Response Storage',
  meta: {
    http: {
      url: 'https://api.example.com/user',
      method: 'GET',
      responseStorePath: 'data.currentUser', // Store at custom path
      transformResponse: (body, ctx) => {
        // Extract only what you need
        return {
          id: body.id,
          name: body.name,
          email: body.email
        };
      }
    }
  }
});

Configuration Options

HttpRequestConfig

| Option | Type | Default | Description | |--------|------|---------|-------------| | url | string | required | Target URL (supports template variables) | | method | HttpMethod | 'GET' | HTTP method | | headers | Record<string, string> | {} | Custom headers | | queryParams | Record<string, string\|number\|boolean> | {} | Query parameters | | body | unknown | undefined | Request body | | bodyFromPath | string | undefined | Path in context to extract body from | | timeoutMs | number | 30000 | Request timeout in milliseconds | | retries | number | 0 | Number of retry attempts | | retryDelayMs | number | 1000 | Delay between retries | | retryOnStatus | number[] | [408, 429, 500, 502, 503, 504] | Status codes to retry on | | transformRequest | function | undefined | Transform request before sending | | transformResponse | function | undefined | Transform response after receiving | | validateResponse | function | undefined | Validate response | | responseStorePath | string | data.http.{nodeId}.response | Custom storage path | | credentials | 'omit'\|'same-origin'\|'include' | undefined | Include credentials | | redirect | 'follow'\|'error'\|'manual' | undefined | Redirect behavior |

Helper Functions

Request Builders

• httpGet(url, options?) - Create GET request
• httpPost(url, body, options?) - Create POST request
• httpPut(url, body, options?) - Create PUT request
• httpDelete(url, options?) - Create DELETE request
• httpPatch(url, body, options?) - Create PATCH request

Configuration Enhancers

• withAuth(config, token, type?) - Add authentication header
• withHeaders(config, headers) - Add custom headers
• withRetry(config, retries?, retryDelayMs?) - Add retry logic
• withTimeout(config, timeoutMs) - Set timeout

Response Metadata

Each HTTP request stores metadata at data.http.{nodeId}.meta:

{
  attempts: 1,        // Number of attempts made
  duration: 234,      // Request duration in ms
  success: true       // Whether request succeeded
}

Error Handling

Errors are stored at data.http.{nodeId}.error:

{
  message: 'Request timeout after 30000ms',
  code: 'REQUEST_FAILED',
  status?: 500,       // HTTP status if available
  response?: {...}    // Response object if available
}

Complete Example

import { ManagedCyclicGraph } from '@quarry-systems/managed-cyclic-graph';
import { mcgHttpPlugin, httpGet, httpPost, withAuth, withRetry } from '@quarry-systems/mcg-http';

const graph = new ManagedCyclicGraph<{ apiToken: string }, { userId: string }>('userWorkflow')
  .use(mcgHttpPlugin)
  
  // Fetch user data
  .node('fetchUser', {
    label: 'Fetch User',
    meta: {
      http: withAuth(
        httpGet('https://api.example.com/users/${injected.userId}'),
        '${global.apiToken}'
      )
    }
  })
  
  // Update user preferences
  .node('updatePreferences', {
    label: 'Update Preferences',
    meta: {
      http: withRetry(
        withAuth(
          httpPost(
            'https://api.example.com/users/${injected.userId}/preferences',
            { theme: 'dark', notifications: true }
          ),
          '${global.apiToken}'
        ),
        3,
        2000
      )
    }
  })
  
  // Send notification
  .node('sendNotification', {
    label: 'Send Notification',
    meta: {
      http: {
        url: 'https://api.example.com/notifications',
        method: 'POST',
        headers: {
          'Authorization': 'Bearer ${global.apiToken}',
          'X-User-Id': '${injected.userId}'
        },
        bodyFromPath: 'data.http.fetchUser.response.body',
        transformRequest: (userData, ctx) => ({
          userId: ctx.injected.userId,
          message: `Welcome back, ${userData.name}!`,
          timestamp: Date.now()
        }),
        validateResponse: (response) => {
          return response.ok || `Notification failed: ${response.status}`;
        }
      }
    }
  })
  
  .edge('fetchUser', 'updatePreferences', 'any')
  .edge('updatePreferences', 'sendNotification', 'any')
  .start('fetchUser')
  .build();

TypeScript Support

The plugin is fully typed with TypeScript:

import type { 
  HttpRequestConfig, 
  HttpResponse, 
  HttpError,
  HttpMethod 
} from '@quarry-systems/mcg-http';

License

MIT