FlowMaker Worker CLI

A comprehensive CLI tool for managing FlowMaker workers - distributed data processing pipelines built with a source-pipe-sink architecture. This package provides everything you need to scaffold, build, test, and deploy workers across multiple runtimes (Node.js, Python, .NET).

What is FlowMaker?

FlowMaker is a framework for building industrial data processing workflows. Workers are self-contained processing units that can:

• Source: Ingest data from external systems (APIs, databases, file systems)
• Pipe: Transform and process data through custom logic
• Sink: Output processed data to destinations or downstream systems

Each worker includes Docker containerization, optional frontend UI components, and follows a standardized structure for consistency across your data pipeline.

Worker CLI

The worker CLI is available as an npm package and can be run via npx:

npx worker-cli <command> [options]

Commands

init - Initialize a repository

Initialize a new FlowMaker workers repository in the current directory by copying template files (README.md, .gitignore, .gitattributes, .env, AGENTS/).

# Initialize in current directory
npx worker-cli init

update-repo-guidelines - Update guidelines

Update README.repo-guidelines.md and AGENTS/* files from templates.

# Update guidelines in current directory
npx worker-cli update-repo-guidelines

create - Create a new worker

Scaffolds a new worker from templates. Runs interactively by default; switches to non-interactive mode when --type, --name, --box-type, and --icon are all provided (useful for LLMs and CI).

# Interactive
npx worker-cli create

# Non-interactive
npx worker-cli create --type node --name my-worker --box-type pipe --icon code
npx worker-cli create --type dotnet --name my-worker --box-type source --icon sensors --display-name "My Worker" --no-build

All workers are scaffolded with a flowbox-ui Svelte frontend.

Non-interactive flags:

| Flag | Description | |------|-------------| | --type <node\|python\|dotnet> | Worker runtime (required) | | --name <name> | Worker name — letters and hyphens only, no digits (required) | | --box-type <source\|pipe\|sink> | Worker box role (required) | | --icon <icon> | Material Symbols icon name (required) | | --display-name <name> | Human-readable name (optional, defaults to title-cased name) | | --no-build | Skip initial build |

build - Build Docker images

Build Docker images for workers with multi-arch support (linux/amd64, linux/arm64). Supports parallel builds for multiple workers.

Pre-build checks: Before building, the CLI runs consistency checks and version validation. Use --no-check to skip these checks.

# Build a specific worker
npx worker-cli build example-box

# Build multiple workers in parallel
npx worker-cli build example-box another-box

# Build all workers and push to registry
npx worker-cli build --all --push

# Build without running tests
npx worker-cli build example-box --no-test

# Skip pre-build consistency and version checks
npx worker-cli build example-box --no-check

Options:

| Option | Description | |--------|-------------| | --all | Build all workers | | --push | Push to remote registry (default: load locally) | | --test | Test container after build (default for local) | | --no-test | Skip container testing | | --no-check | Skip pre-build consistency and version checks | | --parallel | Enable parallel builds (default for multiple workers) |

test - Test container registration

Test a container's registration against a mock config-hub server.

npx worker-cli test <worker-dir> <image-name>

check - Version alignment

Display version information across all workers.

npx worker-cli check

consistency - Validate workers

Check worker structure, version, syntax, and issues. Runs all checks: naming conventions, required files, version format, TypeScript/Python/.NET syntax, frontend, and issues.yaml.

# Check all workers
npx worker-cli consistency

# Check specific workers
npx worker-cli consistency example-box

See README.repo-guidelines.md for detailed requirements.

Frontend CDN Dependencies

Worker frontends are built as self-contained IIFE bundles (Svelte, compiled by Vite). Dependencies that would be too large to inline — like lodash-es, dayjs, etc. — are loaded at runtime from a CDN rather than bundled.

The @cdn-cache/ import convention

Use the @cdn-cache/ virtual prefix to declare a CDN-loaded dependency:

// In your ConfigForm.svelte or main.js
const { default: dayjs } = await import('@cdn-cache/[email protected]');

The Vite build plugin (cdnCacheProd) enforces two rules at build time:

1. The package must be listed in frontend/package.json under peerDependencies with the exact same version.
2. The version must be exact semver (x.y.z) — no ranges.

The import is kept external in the output bundle as http://@cdn-cache/[email protected]. At runtime, FlowMaker replaces the http://@cdn-cache/ prefix with the actual CDN base URL before executing the bundle.

Self-hosted CDN (cdn-esm docker)

The CDN is served by a self-hosted esm.sh instance built from source — see cdn-cache/cdn-esm/ in the repository root. This allows air-gapped deployments with no dependency on public CDNs.

Packages are resolved and cached on first request. No pre-registration of packages is needed: declaring peerDependencies in package.json is the only tracking mechanism required.

Configuration

The .env file at the repository root configures local development settings. Running npx worker-cli init copies a template .env with all available variables commented out.

| Variable | Required | Description | |----------|----------|-------------| | REGISTRY | Yes (for --push) | Docker registry to push images to. Bring your own registry server or use Docker Hub (docker.io/myorg). | | IMAGE_PREFIX | No | Prefix inserted between the registry and the worker name — e.g. myteam/ → registry.example.com/myteam/my-worker:1.0.0 | | CONFIG_HUB_MOCK_PORT | No | Port for mock config-hub during container tests (default: 3120) |

Example .env:

REGISTRY=registry.example.com
IMAGE_PREFIX=myteam/

Setup

Prerequisites

| Tool | Required for | Install | |------|-------------|---------| | Node.js 22+ | CLI, node workers | nodejs.org | | pnpm | Node dependency management | npm install -g pnpm | | Docker + buildx | Building container images | docker.com | | .NET SDK 9.0+ | dotnet workers only | dotnet.microsoft.com | | Python 3.11+ | python workers only | python.org |

Quick setup

# Verify prerequisites
npx worker-cli doctor

Dependencies & registries

All npm packages (including @industream/flowmaker-sdk and @industream/flowmaker-flowbox-ui) are published on the public npm registry (npmjs.org). No private registry configuration or .npmrc is needed.

Docker images are pushed to whichever registry you configure in REGISTRY (your own registry server or Docker Hub).

Adding a New Worker

The recommended way is to use the generator:

# Interactive
npx worker-cli create

# Non-interactive (for LLMs / CI)
npx worker-cli create --type node --name my-worker --box-type pipe --icon code

This scaffolds the worker directory (including a Svelte flowbox-ui frontend) and optionally runs the initial build.

To do it manually:

1. Create a new directory under workers/ (lowercase letters and hyphens only — no digits; SDK constraint)
2. Add Dockerfile following the existing multi-stage pattern
3. Add README.md documenting the worker
4. Create src/ directory for source code:
   • Node.js: Add package.json with name @industream/flow-box-<worker-name>
   • .NET: Add *.csproj with <OutputType>Exe</OutputType> and <Version>
   • Python: Add pyproject.toml with name and version
5. Optionally add frontend components in frontend/
6. Run npx worker-cli consistency <worker-name> to check structure

Building Locally

# Build a specific worker (loads to local Docker)
npx worker-cli build example-box

# Build and push to registry
npx worker-cli build example-box --push