@acepad/worker

Web Worker management for acepad and petit-node. This package enables creating Web Workers that run petit-node modules with RPC (Remote Procedure Call) communication, allowing asynchronous and parallel execution in the browser environment.

Features

• Worker creation from petit-node modules: Run JavaScript modules in Web Workers with full petit-node support
• RPC-based communication: Bidirectional communication between main thread and workers using @hoge1e3/rpc
• Automatic proxy generation: Export functions from modules are automatically exposed as async RPC methods
• Console forwarding: Worker console output is forwarded to the main thread console
• File system access: Workers have access to the petit-node virtual file system
• Callback support: Workers can call back to the main thread using reverse RPC channels

Note: This package is designed to run in the acepad/petit-node environment, not in standard Node.js.

Basic Usage

Creating a Simple Worker

Create a worker module that exports functions:

worker-module.js:

// Functions exported from this module become RPC methods
export function calculate(x) {
    console.log("Calculating in worker:", x);
    return x * 2;
}

export function heavyTask(data) {
    // Perform CPU-intensive work in background
    let result = 0;
    for (let i = 0; i < data.length; i++) {
        result += data[i] * Math.random();
    }
    return result;
}

Main thread:

#!run
import {createProxy} from "@acepad/worker";

export async function main() {
    const workerModule = this.resolve("./worker-module.js");
    
    // Create worker proxy
    const worker = await createProxy(workerModule);
    
    // Call worker methods (all calls are async)
    const result = await worker.calculate(21);
    this.echo(`Result: ${result}`);  // Output: Result: 42
    
    // Call another method
    const data = [1, 2, 3, 4, 5];
    const heavyResult = await worker.heavyTask(data);
    this.echo(`Heavy task result: ${heavyResult}`);
}

Worker with Callbacks

Workers can call back to the main thread using reverse RPC:

callback-worker.js:

export async function processWithProgress(items) {
    for (let i = 0; i < items.length; i++) {
        // Callback to main thread
        this.onProgress(i + 1, items.length);
        
        // Process item
        await new Promise(resolve => setTimeout(resolve, 100));
    }
    
    this.onComplete("All done!");
    return items.length;
}

Main thread:

#!run
import {createProxy} from "@acepad/worker";

export async function main() {
    const workerModule = this.resolve("./callback-worker.js");
    
    // Provide callback handlers in reverse proxy
    const worker = await createProxy(workerModule, {
        onProgress(current, total) {
            this.echo(`Progress: ${current}/${total}`);
        },
        onComplete(message) {
            this.echo(message);
        }
    });
    
    const items = ["a", "b", "c", "d", "e"];
    await worker.processWithProgress(items);
}

Developer Guide

Core API

createProxy(mainModule, reverseProxy?)

Creates a Web Worker that runs a petit-node module and returns an RPC proxy for calling its exported functions.

Parameters:

• mainModule - SFile object or path string pointing to the worker module
• reverseProxy - Optional object with callback methods that the worker can call

Returns: Promise that resolves to an RPC proxy object

Worker Environment:

• Full petit-node runtime initialized
• File system mounted according to /fstab.json
• Console output forwarded to main thread
• All exported functions bound to reverse proxy as this

Example:

import {createProxy} from "@acepad/worker";

const workerFile = this.resolve("/path/to/worker.js");
const proxy = await createProxy(workerFile);

// Call worker methods
const result = await proxy.someMethod(arg1, arg2);

With reverse proxy:

const proxy = await createProxy(workerFile, {
    notify(message) {
        console.log("Worker notification:", message);
    },
    updateUI(data) {
        // Update UI based on worker data
    }
});

// Worker can now call this.notify() and this.updateUI()

create(mainModule)

Creates a Web Worker that simply runs a petit-node module without RPC setup. Useful for fire-and-forget background tasks.

Parameters:

• mainModule - SFile object or path string pointing to the module to execute

Returns: Promise that resolves to the Worker object

Example:

import {create} from "@acepad/worker";

const taskModule = this.resolve("./background-task.js");
const worker = await create(taskModule);

// Worker runs the module and terminates when done
// No direct communication except console forwarding

importExpr(file)

Generates an import expression string for dynamically importing a module in worker context.

Parameters:

• file - SFile object or path string

Returns: String containing import expression

Example:

import {importExpr} from "@acepad/worker";

const moduleFile = this.resolve("./my-module.js");
const expr = importExpr(moduleFile);
// Returns: "await pNode.importModule(FS.get(\"/path/to/my-module.js\"))"

This is primarily used internally for worker code generation.

Console Forwarding

The cons module handles console output forwarding between workers and the main thread.

cons.server(worker)

Sets up console forwarding server on the main thread to receive worker console output.

Parameters:

• worker - Worker object

Example:

import {cons} from "@acepad/worker";

const worker = new Worker("worker.js");
cons.server(worker);  // Now worker console.log/error forwarded to main console

cons.client()

Sets up console forwarding client inside a worker to send console output to main thread. This is automatically called when using createProxy or create.

Worker code:

import {cons} from "@acepad/worker";

cons.client();  // Forward this worker's console to main thread

console.log("This appears in main thread console");

Worker Module Patterns

Pattern 1: Computation Worker

Offload CPU-intensive calculations:

fib-worker.js:

export function fibonacci(n) {
    if (n <= 1) return n;
    return fibonacci(n - 1) + fibonacci(n - 2);
}

export function fibonacciSeries(count) {
    const series = [];
    for (let i = 0; i < count; i++) {
        series.push(fibonacci(i));
    }
    return series;
}

Usage:

#!run
import {createProxy} from "@acepad/worker";

export async function main() {
    const worker = await createProxy(this.resolve("./fib-worker.js"));
    
    // This won't block the UI
    const series = await worker.fibonacciSeries(30);
    this.echo(series);
}

Pattern 2: File Processing Worker

Process large files in the background:

file-processor.js:

import * as fs from "fs";

export async function processLargeFile(inputPath, outputPath) {
    const content = fs.readFileSync(inputPath, "utf-8");
    const lines = content.split("\n");
    
    const processed = lines.map(line => {
        // Heavy processing per line
        return line.toUpperCase().split("").reverse().join("");
    });
    
    fs.writeFileSync(outputPath, processed.join("\n"));
    
    this.onComplete(processed.length);  // Callback to main thread
    return processed.length;
}

Usage:

#!run
import {createProxy} from "@acepad/worker";

export async function main() {
    const worker = await createProxy(
        this.resolve("./file-processor.js"),
        {
            onComplete(lineCount) {
                this.echo(`Processed ${lineCount} lines`);
            }
        }
    );
    
    await worker.processLargeFile(
        "/idb/input.txt",
        "/idb/output.txt"
    );
}

Pattern 3: Data Fetching Worker

Fetch data without blocking the main thread:

data-fetcher.js:

export async function fetchMultipleURLs(urls) {
    const results = [];
    
    for (let i = 0; i < urls.length; i++) {
        this.onProgress(i, urls.length);
        
        try {
            const response = await fetch(urls[i]);
            const data = await response.json();
            results.push({ url: urls[i], data, success: true });
        } catch (error) {
            results.push({ url: urls[i], error: error.message, success: false });
        }
    }
    
    return results;
}

Usage:

#!run
import {createProxy} from "@acepad/worker";

export async function main() {
    const worker = await createProxy(
        this.resolve("./data-fetcher.js"),
        {
            onProgress(current, total) {
                this.echo(`Fetching ${current}/${total}...`);
            }
        }
    );
    
    const urls = [
        "https://api.example.com/data1",
        "https://api.example.com/data2",
        "https://api.example.com/data3"
    ];
    
    const results = await worker.fetchMultipleURLs(urls);
    this.echo(JSON.stringify(results, null, 2));
}

Pattern 4: Stateful Worker

Maintain state across multiple calls:

stateful-worker.js:

let cache = {};
let callCount = 0;

export function addToCache(key, value) {
    cache[key] = value;
    callCount++;
    return Object.keys(cache).length;
}

export function getFromCache(key) {
    callCount++;
    return cache[key];
}

export function getStats() {
    return {
        cacheSize: Object.keys(cache).length,
        totalCalls: callCount
    };
}

export function clearCache() {
    cache = {};
    const prevCount = callCount;
    callCount = 0;
    return prevCount;
}

Usage:

#!run
import {createProxy} from "@acepad/worker";

export async function main() {
    const worker = await createProxy(this.resolve("./stateful-worker.js"));
    
    await worker.addToCache("user1", { name: "Alice", age: 30 });
    await worker.addToCache("user2", { name: "Bob", age: 25 });
    
    const user = await worker.getFromCache("user1");
    this.echo(JSON.stringify(user));
    
    const stats = await worker.getStats();
    this.echo(`Stats: ${JSON.stringify(stats)}`);
    // Output: Stats: {"cacheSize":2,"totalCalls":3}
}

Advanced Features

Accessing File System in Workers

Workers have full access to the petit-node file system:

fs-worker.js:

import * as fs from "fs";
import * as path from "path";

export function listFiles(directory) {
    const files = fs.readdirSync(directory);
    return files.map(name => {
        const fullPath = path.join(directory, name);
        const stats = fs.statSync(fullPath);
        return {
            name,
            size: stats.size,
            isDirectory: stats.isDirectory()
        };
    });
}

export function writeLogEntry(logPath, message) {
    const timestamp = new Date().toISOString();
    const entry = `[${timestamp}] ${message}\n`;
    fs.appendFileSync(logPath, entry);
}

Error Handling

Worker errors are automatically reported:

error-worker.js:

export function riskyOperation(value) {
    if (value < 0) {
        throw new Error("Negative values not allowed");
    }
    return Math.sqrt(value);
}

export async function asyncRiskyOperation(url) {
    const response = await fetch(url);
    if (!response.ok) {
        throw new Error(`HTTP ${response.status}: ${response.statusText}`);
    }
    return await response.json();
}

Error handling in main thread:

#!run
import {createProxy} from "@acepad/worker";

export async function main() {
    const worker = await createProxy(this.resolve("./error-worker.js"));
    
    try {
        const result = await worker.riskyOperation(-5);
    } catch (error) {
        this.echo(`Error: ${error.message}`);
        // Output: Error: Negative values not allowed
    }
    
    try {
        await worker.asyncRiskyOperation("https://invalid.url");
    } catch (error) {
        this.echo(`Async error: ${error.message}`);
    }
}

Using npm Packages in Workers

Workers can use any npm package available in petit-node:

npm-worker.js:

import _ from "lodash";
import moment from "moment";

export function processData(data) {
    // Use lodash
    const grouped = _.groupBy(data, "category");
    const sorted = _.sortBy(data, "timestamp");
    
    return {
        grouped,
        sorted,
        count: data.length
    };
}

export function formatDates(timestamps) {
    // Use moment
    return timestamps.map(ts => 
        moment(ts).format("YYYY-MM-DD HH:mm:ss")
    );
}

Complete Example

Here's a complete example showing worker creation, callbacks, and error handling:

complete-worker.js:

import * as fs from "fs";

export async function analyzeFiles(directory) {
    try {
        const files = fs.readdirSync(directory);
        let totalSize = 0;
        let fileCount = 0;
        const extensions = {};
        
        for (let i = 0; i < files.length; i++) {
            const file = files[i];
            const filePath = `${directory}/${file}`;
            
            try {
                const stats = fs.statSync(filePath);
                
                if (stats.isFile()) {
                    fileCount++;
                    totalSize += stats.size;
                    
                    const ext = file.split(".").pop() || "no-ext";
                    extensions[ext] = (extensions[ext] || 0) + 1;
                    
                    // Progress callback
                    this.onProgress({
                        current: i + 1,
                        total: files.length,
                        file
                    });
                }
            } catch (err) {
                this.onError(`Error reading ${file}: ${err.message}`);
            }
        }
        
        const result = {
            fileCount,
            totalSize,
            averageSize: totalSize / fileCount,
            extensions
        };
        
        this.onComplete(result);
        return result;
        
    } catch (error) {
        this.onError(`Fatal error: ${error.message}`);
        throw error;
    }
}

Main thread:

#!run
import {createProxy} from "@acepad/worker";

export async function main() {
    const sh = this;
    
    const worker = await createProxy(
        this.resolve("./complete-worker.js"),
        {
            onProgress(info) {
                sh.echo(`Processing: ${info.file} (${info.current}/${info.total})`);
            },
            onComplete(result) {
                sh.echo("\n=== Analysis Complete ===");
                sh.echo(`Files: ${result.fileCount}`);
                sh.echo(`Total size: ${result.totalSize} bytes`);
                sh.echo(`Average: ${result.averageSize.toFixed(2)} bytes`);
                sh.echo(`Extensions: ${JSON.stringify(result.extensions)}`);
            },
            onError(message) {
                sh.echo(`⚠️  ${message}`);
            }
        }
    );
    
    try {
        const result = await worker.analyzeFiles("/idb/run/");
        sh.echo("\nAnalysis result:", JSON.stringify(result, null, 2));
    } catch (error) {
        sh.echo(`Failed: ${error.message}`);
    }
}

Testing

The package includes test files demonstrating various worker patterns:

• test/testworker.js - Basic worker with callbacks
• test/simple-worker.js - Simple message-based worker
• test/broadcast.js - BroadcastChannel RPC example
• test/test-acepad-worker.js - Complete test suite

Run tests:

sh: npm test

Implementation Details

Worker Code Generation

When you call createProxy(mainModule), the package generates worker code that:

1. Imports petit-node runtime
2. Initializes file system with fstab configuration
3. Imports the target module
4. Sets up RPC server for exported functions
5. Sets up console forwarding
6. Establishes reverse RPC channel for callbacks

The generated worker code uses Blob URLs to create workers dynamically.

RPC Communication

• Uses @hoge1e3/rpc for bidirectional communication
• Main thread → Worker: "default" channel
• Worker → Main thread: Random channel ID for reverse proxy
• All function calls are async and return Promises

File System Access

Workers access the same file system as the main thread:

• /idb/ - IndexedDB-backed persistent storage
• /tmp/ - RAM-based temporary storage
• Custom mounts from /fstab.json

File system operations in workers are isolated but share the same underlying storage.

Dependencies

• @hoge1e3/str2worker: Worker creation from string source
• @hoge1e3/rpc: RPC communication framework
• @acepad/npm: npm package management
• @acepad/here: Path resolution utilities
• petit-node: Browser-based Node.js runtime (peer dependency)

Limitations

• Workers cannot directly access DOM or UI elements
• All data passed between main thread and workers must be serializable
• Some browser APIs may not be available in workers (e.g., localStorage)
• Error stack traces may be less detailed in workers

See Also

• acepad - The acepad programming environment
• petit-node - Browser-based Node.js runtime
• @hoge1e3/rpc - RPC library
• @hoge1e3/str2worker - Dynamic worker creation
• Web Workers MDN - Web Workers documentation

License

ISC