Muriel

A lightweight flow engine that makes complex workflows readable.

Muriel helps you build data processing pipelines that both humans and AI can understand at a glance. Whether you're scanning files, processing images, generating reports, or building anything that moves data through stages—Muriel keeps it simple and visual.

See COOKBOOK.md for ready-to-use patterns

Why Muriel?

When AI generates code, it needs to be human-readable. When humans design workflows, they need to be AI-understandable. Muriel bridges this gap with a syntax that looks like the diagrams you'd draw on a whiteboard:

['input', processData, validate, 'output']

That's a complete pipeline. Data flows from the 'input' pipe, through processData and validate, and lands in 'output'. No classes, no inheritance, no magic—just functions and arrays.

Real-World Example: Blog Generator

Odor is a static blog generator built entirely on Muriel. It processes thousands of posts—encoding images to AVIF, audio to MP3, rendering Markdown to HTML—all while keeping the flow graph readable:

import { flow } from 'muriel';

const blog = flow([
  // Scan posts, skip unchanged ones
  [postScanner(), skipUnchanged(), 'post'],

  // Process each post: cover + audio + files (parallel), then text
  ['post',
    [processCover(), processAudio(), copyFiles()],
    processText(),
    verifyPost(),
    collectPost(),
  'done'],

  // Generate homepage, pages, RSS feed (parallel)
  ['done',
    [homepage(), pagerizer(), rssFeed()],
    useTheme(),
  'finished']
], {
  context: { config: myConfig }
});

blog.start();

What's happening:

1. Posts flow through a scanner
2. Three tasks run in parallel: cover encoding, audio encoding, file copying
3. After parallel work completes, text processing happens in series
4. Finally, the homepage, archive pages, and RSS feed generate in parallel

The code matches the mental model. No hidden state. No callback pyramids. Just pipes and stages.

Installation

npm install muriel

Core Concepts

1. Pipes are Event Channels

A pipe is a named event channel. Think of it like a tube that packets flow through:

import { flow } from 'muriel';

flow([
  ['start', processData, 'end']
], { context: {} });

Here, 'start' and 'end' are pipes. processData is a stage between them.

2. Stages Transform Packets

A stage is a function that receives a packet, does something with it, and sends it forward:

function double(options = {}) {
  return (send, packet) => {
    send({ ...packet, value: packet.value * 2 });
  };
}

Pattern:

• Outer function = configuration (receives options)
• Inner function = execution (receives send and packet)
• Call send(newPacket) to pass data to the next stage

3. Arrays Mean Parallel

When you put stages in an array, they run in parallel:

['input',
  [processImage, processAudio, processText],
'output']

All three stages receive the same input packet simultaneously. Muriel automatically waits for all three to finish, then joins their results into a single packet:

{
  ...packet,
  branches: [
    { ...result from processImage },
    { ...result from processAudio },
    { ...result from processText }
  ]
}

4. Everything Else is Series

Stages written one after another run in sequence:

['input', validate, normalize, save, 'output']

Each stage waits for the previous one to complete before starting.

5. Context Flows Through

You can pass global context (like config, user info, or database connections) into every packet:

flow([
  ['start', processData, 'end']
], {
  context: { userId: 123, db: myDatabase }
});

Every packet will have packet.context.userId and packet.context.db available.

Example: File Processor

Let's build a simple file processor that scans a directory, filters images, resizes them, and saves thumbnails:

import { flow } from 'muriel';
import fs from 'fs/promises';
import sharp from 'sharp';
import path from 'path';

// Producer: scans directory and emits one packet per file
function scanFiles(dir) {
  return async (send) => {
    const files = await fs.readdir(dir);
    for (const file of files) {
      send({ file: path.join(dir, file) });
    }
  };
}

// Transform: only pass image files
function filterImages() {
  return (send, packet) => {
    const ext = path.extname(packet.file).toLowerCase();
    if (['.jpg', '.png', '.webp'].includes(ext)) {
      send(packet);
    }
    // If not an image, packet is dropped (no send)
  };
}

// Transform: resize image to thumbnail
function createThumbnail(size = 200) {
  return async (send, packet) => {
    const output = packet.file.replace(/(\.\w+)$/, `-thumb$1`);

    await sharp(packet.file)
      .resize(size, size, { fit: 'cover' })
      .toFile(output);

    send({ ...packet, thumbnail: output });
  };
}

// Transform: log result
function logResult() {
  return (send, packet) => {
    console.log(`✓ Created thumbnail: ${packet.thumbnail}`);
    send(packet);
  };
}

// Build the flow
const processor = flow([
  [scanFiles('./photos'), 'file'],
  ['file', filterImages(), createThumbnail(200), logResult(), 'done']
], {
  context: {}
});

processor.start();

Output:

✓ Created thumbnail: ./photos/cat-thumb.jpg
✓ Created thumbnail: ./photos/dog-thumb.jpg
✓ Created thumbnail: ./photos/beach-thumb.png

Producers vs Transforms

There are two types of stages:

Producers (1 argument)

Producers create packets from scratch. They receive only send:

function ticker(interval) {
  return (send) => {
    setInterval(() => {
      send({ timestamp: Date.now() });
    }, interval);
  };
}

// Usage
[ticker(1000), 'tick']

Transforms (2 arguments)

Transforms modify packets. They receive send and packet:

function addTimestamp() {
  return (send, packet) => {
    send({ ...packet, timestamp: Date.now() });
  };
}

// Usage
['tick', addTimestamp(), 'timestamped']

Parallel + Series Composition

You can mix parallel and series stages:

['input',
  validateInput,
  [fetchUserData, fetchProductData, fetchOrderData],  // parallel
  mergeResults,
  saveToDatabase,
'output']

Flow diagram:

(input) → validateInput
            ↓
        ┌───┴───┬───────────┐
        ↓       ↓           ↓
   fetchUser fetchProduct fetchOrder  (parallel)
        └───┬───┴───────────┘
            ↓
        mergeResults → saveToDatabase → (output)

Worker Threads (Optional)

For CPU-heavy transforms (like JSON parsing, string processing, or math-intensive work), you can enable a worker thread pool:

flow(graph, {
  context: {},
  workers: 4  // Run transforms in 4 worker threads
});

Important: Worker threads use eval() to serialize functions, so they only work with pure, self-contained transforms. Functions that import modules or use closures won't work in workers.

For I/O-heavy tasks (like image encoding with sharp or audio with ffmpeg), workers aren't needed—those libraries spawn their own processes. Use a concurrency gate instead (see COOKBOOK.md).

Logic Patterns: The Cookbook Approach

Instead of bundling a standard library, Muriel stays lightweight and gives you a cookbook of copy-paste recipes. See COOKBOOK.md for ready-to-use patterns:

Conditional Logic

function when(condition, thenFn) {
  return async (send, packet) => {
    if (condition(packet)) {
      await thenFn(send, packet);
    } else {
      send(packet);
    }
  };
}

// Usage
['input',
  when(p => p.age >= 18, (send, p) => send({ ...p, adult: true })),
'output']

Filtering

function filter(predicate) {
  return (send, packet) => {
    if (predicate(packet)) {
      send(packet);
    }
    // Otherwise packet is dropped
  };
}

// Usage
['input', filter(p => p.status === 'active'), 'output']

Retry Logic

function retry(fn, { times = 3, delay = 1000, backoff = 2 } = {}) {
  return async (send, packet) => {
    let lastError;
    for (let attempt = 1; attempt <= times; attempt++) {
      try {
        await fn(send, packet);
        return;
      } catch (err) {
        lastError = err;
        if (attempt < times) {
          await new Promise(r => setTimeout(r, delay * Math.pow(backoff, attempt - 1)));
        }
      }
    }
    send({ ...packet, error: lastError.message, _retryFailed: true });
  };
}

Throttling

function throttle(ms) {
  let lastTime = 0;
  return (send, packet) => {
    const now = Date.now();
    if (now - lastTime >= ms) {
      lastTime = now;
      send(packet);
    }
  };
}

State Machines

function machine(states, initialState) {
  let currentState = initialState;
  return (send, packet) => {
    const stateConfig = states[currentState];
    const result = stateConfig.onEnter?.(packet) || packet;

    for (const [event, nextState] of Object.entries(stateConfig.on || {})) {
      if (packet.event === event) {
        currentState = nextState;
        break;
      }
    }
    send({ ...result, _state: currentState });
  };
}

See COOKBOOK.md for 30+ more patterns including:

• Debouncing and batching
• Logging and debugging helpers
• Error boundaries
• Windowing and accumulation
• Composition utilities

Why No Standard Library?

Muriel is designed to be supremely lightweight. Adding built-in operators would:

1. Increase bundle size
2. Lock you into our API choices
3. Make the codebase harder to understand
4. Create versioning headaches

Instead, the COOKBOOK.md gives you proven patterns you can copy, paste, and adapt to your exact needs. Every recipe is:

• Self-contained (no dependencies)
• Easy to modify
• Fully visible in your codebase

This approach keeps Muriel minimal while giving you maximum flexibility.

Disposal

Always clean up when you're done:

const processor = flow(graph, { context });

processor.start();

// Later...
processor.dispose();

Disposal:

• Removes all event listeners
• Terminates worker threads (if enabled)
• Tears down the entire graph

API Reference

flow(edges, options)

Creates a flow graph.

Parameters:

• edges - Array of edge definitions: [inputPipe, ...stages, outputPipe]
• options.context - Object merged into every packet
• options.workers - Number of worker threads (optional)

Returns: Flow instance with start() and dispose() methods

Edge Format

An edge connects pipes through stages:

[inputPipe, stage1, stage2, ..., outputPipe]

Producer edge (no input pipe):

[producer, outputPipe]

Parallel stages:

[inputPipe, [stage1, stage2, stage3], outputPipe]

Mixed:

[inputPipe, series1, [parallel1, parallel2], series2, outputPipe]

Stage Signature

Transforms (modify packets):

function myTransform(options) {
  return (send, packet) => {
    send({ ...packet, newField: 'value' });
  };
}

Producers (create packets):

function myProducer(options) {
  return (send) => {
    send({ data: 'value' });
  };
}

Both can be async.

Send API

The send function passed to transforms has built-in routing capabilities:

send(packet) - Normal flow

function myTransform() {
  return (send, packet) => {
    send({ ...packet, processed: true });
    // Packet continues to next stage or output pipe
  };
}

send.to(pipeName, packet) - Route to specific pipe

function route(routeMap) {
  return (send, packet) => {
    for (const [pipeName, condition] of Object.entries(routeMap)) {
      if (condition(packet)) {
        send.to(pipeName, packet);  // Send directly to named pipe
        return;
      }
    }
  };
}

// Usage
flow([
  ['input', route({
    'valid': p => p.status === 'ok',
    'error': p => p.status === 'error'
  }), 'next'],

  ['valid', processValid, 'done'],
  ['error', handleError, 'done']
], { context: {} });

See examples/routing-example.js for a complete routing demonstration.

Design Philosophy

1. If You Can Draw It, You Can Code It

Muriel's syntax mirrors flowchart diagrams. Arrays are splits, sequences are series, pipes are boxes. The code is the diagram.

2. No Hidden Magic

Every stage is a pure function. No lifecycle methods, no implicit state, no framework lock-in. You can test stages in isolation with plain JavaScript.

3. AI-Friendly

AI models excel at generating Muriel code because the syntax is consistent and predictable. There's one way to do parallel (arrays) and one way to do series (sequences). No edge cases.

4. Human-Friendly

When a human reads a Muriel flow, they see the data path immediately. No need to jump between files or trace method chains. The entire flow fits on one screen.

Comparison to Other Tools

| Tool | Approach | Muriel's Take | |------|----------|---------------| | Node-RED | Visual node editor | Code-first, but looks like diagrams | | RxJS | Reactive operators | Simpler, fewer concepts (no hot/cold, no subjects) | | Airflow | DAG scheduler | Lightweight, no external scheduler needed | | EventEmitter | Event bus | Structured flow, not ad-hoc events |

Muriel borrows ideas from FFmpeg filtergraphs and dataflow programming while staying minimal and JavaScript-native.

Who Uses Muriel?

• Odor - Static blog generator with parallel encoding, incremental builds, and AI agent integration

Using Muriel in production? Let us know!

Contributing

Found a bug? Want to add a recipe to the cookbook? Open an issue or PR:

GitHub: (Add your repo URL here)

License

MIT

Learn More

• COOKBOOK.md - 30+ copy-paste recipes for common patterns
• examples/ - Full working examples
• test.js - 11 tests showing all flow patterns

Muriel: Named after the character who always knew where things should go. 🎯