RudderStack Workflow Engine - Service Worker Edition

A powerful, config-driven workflow execution engine designed specifically for Chrome service workers and browser extensions. Execute complex data transformations using YAML configurations and template languages, all without server-side dependencies.

🚀 Features

• Service Worker Native: Built specifically for Chrome service workers and browser extensions
• No Node.js Dependencies: Runs entirely in the browser/service worker context
• Template Languages: Support for JSONata and JsonTemplate
• YAML Configuration: Define workflows in readable YAML format
• Registry-Based: All workflows and modules managed through an in-memory registry
• Type-Safe: Full TypeScript support
• Modular Design: Compose complex workflows from simple, reusable steps

📦 Installation

npm install @artpar/workflow-engine-service-worker

🎯 Quick Start

1. Basic Setup in Service Worker

// service-worker.ts
import { 
  ServiceWorkerWorkflowEngine, 
  setupServiceWorkerEnvironment 
} from '@artpar/workflow-engine-service-worker/service-worker';

// Initialize the environment with workflows and modules
setupServiceWorkerEnvironment(
  {
    // Register workflow definitions
    'transform.yaml': `
      name: DataTransformer
      templateType: jsonata
      steps:
        - name: validateInput
          condition: $not($.userId)
          template: |
            $doThrow("userId is required")
        
        - name: transform
          template: |
            {
              "id": $.userId,
              "timestamp": $millis(),
              "data": $.payload
            }
    `
  },
  {
    // Register custom functions/modules
    'bindings': {
      formatDate: (ts) => new Date(ts).toISOString(),
      validateEmail: (email) => /^[^\s@]+@[^\s@]+\.[^\s@]+$/.test(email)
    }
  }
);

// Use in service worker message handler
self.addEventListener('message', async (event) => {
  if (event.data.type === 'TRANSFORM') {
    const engine = await ServiceWorkerWorkflowEngine.createFromRegisteredWorkflow('transform.yaml');
    const result = await engine.execute(event.data.payload);
    event.ports[0].postMessage({ success: true, data: result.output });
  }
});

2. Chrome Extension Example

// background.js (service worker)
import { ServiceWorkerWorkflowEngine } from '@artpar/workflow-engine-service-worker/service-worker';

// Register workflows for different extension tasks
ServiceWorkerWorkflowEngine.registerWorkflow('process-tab-data.yaml', `
  name: TabDataProcessor
  templateType: jsonata
  steps:
    - name: extractDomain
      template: |
        {
          "domain": $substring($.url, $indexOf($.url, "://") + 3, $indexOf($substring($.url, $indexOf($.url, "://") + 3), "/")),
          "title": $.title,
          "timestamp": $millis()
        }
`);

// Process browser tab data
chrome.tabs.onUpdated.addListener(async (tabId, changeInfo, tab) => {
  if (changeInfo.status === 'complete') {
    const engine = await ServiceWorkerWorkflowEngine.createFromRegisteredWorkflow('process-tab-data.yaml');
    const result = await engine.execute({
      url: tab.url,
      title: tab.title
    });
    console.log('Processed tab data:', result.output);
  }
});

📚 Core Concepts

Workflows

Workflows are defined in YAML and consist of:

• name: Identifier for the workflow
• templateType: Template language (jsonata or jsontemplate)
• bindings: External functions and data to import
• steps: Sequential processing steps

Steps

Each step can have:

• name: Step identifier (required)
• description: Human-readable description
• condition: JSONata expression to conditionally execute
• template: Transformation logic
• inputTemplate: Customize input before processing
• loopOverInput: Process arrays element by element

Bindings

Import external functions and data into workflows:

ServiceWorkerWorkflowEngine.registerModule('utils', {
  // Functions
  multiply: (a, b) => a * b,
  
  // Constants
  CONFIG: {
    maxRetries: 3,
    timeout: 5000
  },
  
  // Complex objects
  validators: {
    isEmail: (email) => /^[^\s@]+@[^\s@]+\.[^\s@]+$/.test(email),
    isURL: (url) => /^https?:\/\/.+/.test(url)
  }
});

🔧 API Reference

ServiceWorkerWorkflowEngine

registerWorkflow(path: string, content: string | object): void

Register a workflow definition in the registry.

ServiceWorkerWorkflowEngine.registerWorkflow('my-workflow.yaml', yamlContent);

registerModule(path: string, exports: any): void

Register a module/bindings for use in workflows.

ServiceWorkerWorkflowEngine.registerModule('helpers', {
  formatCurrency: (amount) => `$${amount.toFixed(2)}`
});

createFromYaml(yamlString: string, options?: Partial<WorkflowOptions>): Promise<WorkflowEngine>

Create a workflow engine from YAML string.

const engine = await ServiceWorkerWorkflowEngine.createFromYaml(yamlContent);
const result = await engine.execute(inputData);

createFromRegisteredWorkflow(path: string, options?: Partial<WorkflowOptions>): Promise<WorkflowEngine>

Create a workflow engine from a registered workflow.

const engine = await ServiceWorkerWorkflowEngine.createFromRegisteredWorkflow('transform.yaml');

registerBindings(bindings: Record<string, any>): void

Register multiple bindings at once.

ServiceWorkerWorkflowEngine.registerBindings({
  'math': { add: (a, b) => a + b },
  'string': { capitalize: (s) => s.toUpperCase() }
});

clear(): void

Clear all registered workflows and modules.

ServiceWorkerWorkflowEngine.clear();

setupServiceWorkerEnvironment

Helper function to initialize the environment with workflows and modules in one call.

setupServiceWorkerEnvironment(
  workflows: Record<string, string | object>,
  modules: Record<string, any>
): void

🎨 Template Languages

JSONata

A powerful query and transformation language for JSON data.

name: JSONataExample
templateType: jsonata
steps:
  - name: transform
    template: |
      {
        "fullName": firstName & " " & lastName,
        "age": $number(age),
        "isAdult": age >= 18,
        "email": $lowercase(email)
      }

JSONata Documentation

JsonTemplate

A simple, intuitive template language optimized for data mapping.

name: JsonTemplateExample
templateType: jsontemplate
steps:
  - name: transform
    template: |
      {
        fullName: .firstName + " " + .lastName,
        age: Number(.age),
        isAdult: .age >= 18,
        email: .email.toLowerCase()
      }

JsonTemplate Documentation

💡 Advanced Examples

1. Event Processing with Validation

const eventProcessorYaml = `
name: EventProcessor
templateType: jsonata
bindings:
  - name: validators
steps:
  - name: validate
    template: |
      (
        $assert($.type in ["track", "identify", "page"], "Invalid event type");
        $assert($validators.isEmail($.email), "Invalid email");
        $
      )
  
  - name: enrichEvent
    template: |
      $ ~> | $ | {
        "processedAt": $millis(),
        "version": "1.0",
        "metadata": {
          "source": "browser",
          "ip": $$.context.ip
        }
      } |
`;

ServiceWorkerWorkflowEngine.registerModule('bindings', {
  validators: {
    isEmail: (email) => /^[^\s@]+@[^\s@]+\.[^\s@]+$/.test(email)
  }
});

ServiceWorkerWorkflowEngine.registerWorkflow('event-processor.yaml', eventProcessorYaml);

2. Batch Processing

const batchProcessorYaml = `
name: BatchProcessor
templateType: jsonata
steps:
  - name: processBatch
    loopOverInput: true
    template: |
      {
        "id": $.id,
        "status": "processed",
        "result": $.value * 2,
        "timestamp": $millis()
      }
`;

// Process array of items
const engine = await ServiceWorkerWorkflowEngine.createFromYaml(batchProcessorYaml);
const items = [
  { id: 1, value: 10 },
  { id: 2, value: 20 },
  { id: 3, value: 30 }
];
const result = await engine.execute(items);
// result.output will be an array of processed items

3. Conditional Processing

const conditionalWorkflowYaml = `
name: ConditionalProcessor
templateType: jsonata
steps:
  - name: checkPremium
    condition: $.user.subscription = "premium"
    template: |
      $ ~> | $ | { "features": ["advanced", "priority", "support"] } |
    onComplete: return
  
  - name: checkBasic
    condition: $.user.subscription = "basic"
    template: |
      $ ~> | $ | { "features": ["standard"] } |
    onComplete: return
  
  - name: defaultFeatures
    template: |
      $ ~> | $ | { "features": ["limited"] } |
`;

4. Error Handling

const errorHandlingYaml = `
name: SafeProcessor
templateType: jsonata
steps:
  - name: riskyOperation
    template: |
      (
        $data := $.data;
        $assert($exists($data), "Data is required");
        $data.value / $data.divisor
      )
    onError: continue
  
  - name: fallback
    condition: $not($outputs.riskyOperation)
    template: |
      { "error": "Operation failed", "default": 0 }
`;

🏗️ Architecture

Service Worker Integration

The engine is designed to work seamlessly in service worker contexts:

1. No File System: All workflows stored in memory registry
2. No Dynamic Imports: All code bundled at build time
3. No Node.js APIs: Pure browser-compatible implementation
4. Efficient Memory Usage: Workflows compiled once and cached

Registry System

The registry provides centralized storage for:

• Workflow definitions (YAML/JSON)
• Custom functions and modules
• Configuration constants
• Shared utilities

Path Resolution

Custom path utilities handle path operations without Node.js:

• join(): Combine path segments
• parse(): Extract path components

🔍 Debugging

Enable Debug Logging

import { logger } from '@artpar/workflow-engine-service-worker/service-worker';

// Set log level
logger.setLogLevel(LogLevel.DEBUG);

// Your workflow will now output debug information

Step-by-Step Execution

steps:
  - name: debug1
    template: |
      (
        $log("Input received:", $);
        $
      )
  
  - name: transform
    template: |
      { "result": $.value * 2 }
  
  - name: debug2
    template: |
      (
        $log("Output:", $);
        $
      )

🚦 Limitations

Since this is a service worker implementation:

1. No File System Access: Cannot read files from disk
2. No Dynamic Imports: Cannot load modules at runtime
3. No Node.js APIs: No access to Node.js-specific features
4. Memory Constraints: All workflows stored in memory
5. Bundle Size: All dependencies must be bundled

🤝 Contributing

We welcome contributions! Please see CONTRIBUTING.md for guidelines.

📄 License

This project is licensed under the MIT License.

🔗 Resources

• JSONata Documentation
• JsonTemplate Engine
• Service Worker API
• Chrome Extension Development

💬 Support

• GitHub Issues
• Community Slack
• Documentation

Built with ❤️ by RudderStack