@drew-foxall/upstash-workflow-world

An edge-runtime compatible World implementation for Workflow DevKit using Upstash Redis.

Works on: Cloudflare Workers, Vercel Edge, Deno Deploy, and any JavaScript runtime with fetch().

Attribution

This package is derived from workflow-worlds by Dustin Townsend. The original project provides the foundational architecture and patterns for implementing World interfaces for the Workflow DevKit.

Features

• Edge-Runtime Compatible: Uses HTTP-based Redis operations (no TCP connections required)
• Full World Interface: Implements Storage, Queue, and Streamer capabilities
• Self-Invoking Queue: Uses Redis Streams with HTTP triggers for reliable job processing
• Polling-Based Streaming: Real-time output streaming without persistent connections
• Automatic Retries: Built-in retry logic with exponential backoff
• Idempotency: Prevents duplicate job processing

Installation

npm install @drew-foxall/upstash-workflow-world @upstash/redis

Quick Start

import { createWorld } from '@drew-foxall/upstash-workflow-world';

// Create a world instance
const world = createWorld({
  redisUrl: process.env.UPSTASH_REDIS_REST_URL,
  redisToken: process.env.UPSTASH_REDIS_REST_TOKEN,
  baseUrl: 'https://my-app.vercel.app', // Your deployed app URL
});

export default world;

Configuration

Environment Variables

The package supports the following environment variables:

| Variable | Description | |----------|-------------| | UPSTASH_REDIS_REST_URL | Upstash Redis REST URL | | UPSTASH_REDIS_REST_TOKEN | Upstash Redis REST Token | | WORKFLOW_UPSTASH_REDIS_REST_URL | Alternative Redis URL (higher priority) | | WORKFLOW_UPSTASH_REDIS_REST_TOKEN | Alternative Redis Token (higher priority) | | DEBUG | Set to workflow:* for debug logging |

Config Options

interface UpstashWorldConfig {
  // Redis connection
  redisUrl?: string;           // Upstash Redis REST URL
  redisToken?: string;         // Upstash Redis REST Token
  redis?: Redis;               // Pre-existing @upstash/redis client

  // Key prefix for all Redis keys (default: 'workflow')
  keyPrefix?: string;

  // Queue configuration
  baseUrl?: string;            // Base URL for self-invoking HTTP triggers
  maxRetries?: number;         // Max retry attempts (default: 3)
  retryDelayMs?: number;       // Initial retry delay (default: 1000)
  idempotencyTtlMs?: number;   // Idempotency key TTL (default: 60000)

  // Streamer configuration
  streamMaxLen?: number;       // Max stream length (default: 10000)
  pollIntervalMs?: number;     // Polling interval (default: 100)
}

Usage with Workflow DevKit

Creating a Workflow

import { createWorkflow } from '@workflow/core';
import { createWorld } from '@drew-foxall/upstash-workflow-world';

const world = createWorld({
  redisUrl: process.env.UPSTASH_REDIS_REST_URL,
  redisToken: process.env.UPSTASH_REDIS_REST_TOKEN,
  baseUrl: process.env.VERCEL_URL,
});

const myWorkflow = createWorkflow({
  world,
  name: 'my-workflow',
  async run(ctx, input) {
    const result = await ctx.step('process', async () => {
      return { processed: true };
    });
    return result;
  },
});

Queue Handler (Edge Function)

For the self-invoking queue pattern to work, you need to expose a /drain endpoint:

// pages/api/drain.ts (Next.js) or similar
import { world } from './world';

export const config = { runtime: 'edge' };

export default world.createQueueHandler('workflow', async (message) => {
  // Process queue message
  console.log('Processing:', message);
});

Architecture

This implementation uses:

1. Storage: Redis Hashes and Sorted Sets for runs, steps, events, and hooks
2. Queue: Redis Streams with self-invoking HTTP triggers (no BullMQ/TCP required)
3. Streamer: Redis Streams with polling-based reads (no Pub/Sub required)

Why Not BullMQ?

BullMQ requires TCP connections which are not available in edge runtimes. This implementation uses:

• Redis Streams for reliable message queuing
• HTTP-based self-invocation for job processing triggers
• Polling instead of blocking reads for stream consumption

Development

# Install dependencies
npm install

# Run tests (requires Upstash Redis credentials)
UPSTASH_REDIS_REST_URL=... UPSTASH_REDIS_REST_TOKEN=... npm test

# Build
npm run build

# Type check
npm run typecheck

License

MIT - See LICENSE for details.

Credits

• Original workflow-worlds by Dustin Townsend
• Upstash for the HTTP-based Redis service
• Workflow DevKit team