ymake

A Make-like task runner with YAML files, featuring modern capabilities like matrix builds and structured outputs.

Key Features

• 🎯 Simple YAML syntax: Define tasks clearly and readably
• 🔗 Dependency management: Automatically executes tasks in the correct order
• 🔄 Matrix builds: Run tasks with multiple parameter combinations
• 📤 Structured outputs: Capture and reuse JSON data between tasks
• 🚀 Smart detection: Automatically finds your Ymakefile
• ⚡ Fast: Built with Bun for maximum performance

Installation

Global Installation (Recommended)

npm install -g @jondotsoy/ymake

Or using Bun:

bun install -g @jondotsoy/ymake

Local Installation

npm install --save-dev @jondotsoy/ymake

Or using Bun:

bun add --dev @jondotsoy/ymake

Usage

ymake [target] [options]

Options

• [target...]: Name(s) of the target(s) to execute (optional, uses the first one if not specified). You can specify multiple targets separated by spaces.
• --all: Execute all targets defined in the Ymakefile in order
• --verbose, -v: Shows detailed execution information and outputs
• --version: Display the ymake version
• --help, -h: Show help message with available options
• --file <path>, -f <path>, --ymakefile <path>: Path to the Ymakefile to use (instead of auto-detecting)
• --cwd <path>: Change working directory before executing

Examples

# Execute the first target
ymake

# Execute a specific target
ymake build

# Execute multiple targets in order
ymake test build

# Execute all targets in order
ymake --all

# Execute with verbose output
ymake test --verbose

# Use a custom Ymakefile location
ymake -f path/to/ymakefile.yml build

# Use a Ymakefile from a subdirectory
ymake --file config/prod/ymakefile.yml deploy

# Change working directory before executing
ymake --cwd dist build

# Combine --cwd with custom Ymakefile location
ymake --cwd dist -f prod/ymakefile.yml deploy

Supported file names

ymake automatically searches for these files in order:

• Ymakefile
• Ymakefile.yaml
• Ymakefile.yml
• ymakefile
• ymakefile.yaml
• ymakefile.yml

Custom Ymakefile Location

You can specify a custom Ymakefile location using the --file, -f, or --ymakefile flag. When you do this, ymake will:

1. Load the specified Ymakefile
2. Change the working directory to the directory containing that Ymakefile
3. Execute all commands relative to that directory

This is useful for:

• Managing multiple environments (dev, staging, prod)
• Organizing Ymakefiles in subdirectories
• Running builds from CI/CD pipelines

Example:

# Use a Ymakefile in a subdirectory
ymake -f environments/prod/ymakefile.yml deploy

# Files created by the target will be in environments/prod/

Custom Working Directory

You can change the working directory before executing targets using the --cwd flag. This is particularly useful when:

• Running ymake from a different directory than where your Ymakefile is located
• Working with monorepo structures
• Executing builds in specific subdirectories

When using --cwd:

1. ymake changes to the specified directory first
2. Then loads the Ymakefile (either auto-detected or specified with --file)
3. If --file is used with --cwd, the working directory remains at the --cwd location (not the Ymakefile's directory)

Example:

# Execute in a subdirectory
ymake --cwd dist build

# Combine with custom Ymakefile location
# This will change to dist/, use prod/ymakefile.yml, and execute commands in dist/
ymake --cwd dist -f prod/ymakefile.yml deploy

Given this directory structure:

project/
├── ymake/
│   └── prod/
│       └── ymakefile.yml

And this ymakefile content (ymake/prod/ymakefile.yml):

build:
  run: echo "hello" >> build.txt

Running from the project root:

ymake -f ymake/prod/ymakefile.yml build

Will create build.txt in ymake/prod/build.txt (not in the project root), because the working directory changes to ymake/prod/.

Feature Guide

1. Basic Targets

Define targets with the command to execute. ymake supports two syntaxes:

Object Syntax

build:
  run: echo "Building..."

test:
  need: [build]
  run: echo "Testing..."

String Syntax (Shorthand)

For simple targets without dependencies or options, you can use a string directly:

# Single line
hello: echo "Hello, World!"

# Multi-line with pipe operator
build: |
  echo "Building..."
  mkdir -p dist
  echo "Done"

Mixed Syntax

You can mix both syntaxes in the same file:

# String syntax
hello: echo "Hello"

# Object syntax with dependencies
test:
  need: [hello]
  run: echo "Testing..."

Each target requires either a run property (object syntax) or a command string (string syntax).

Target Environment Variable

The $TARGET environment variable is automatically available in all commands and contains the name of the current target:

hello.txt:
  run: echo "hello" > $TARGET

This creates the file hello.txt using the target name. This is especially useful for file targets where the target name matches the output file.

2. Dependencies

Use need to specify dependencies between targets. You can use a single string or an array of strings:

# Single dependency
test:
  need: build
  run: echo "Testing..."

# Multiple dependencies
deploy:
  need: [build, test]
  run: echo "Deploying..."

Dependencies are executed in order and only once, even if multiple targets need them.

File Dependencies

Dependencies can be either targets or files. If a dependency is not a target, ymake checks if it exists as a file:

hello.txt:
  need: lang.txt
  run: echo "($(cat lang.txt)) hello" > hello.txt

In this example, lang.txt is not a target but a file that must exist. If the file doesn't exist, ymake will fail with an error. This allows you to specify file prerequisites without defining them as targets.

3. Phony Targets

Mark targets as phony so they always execute, without checking files:

clean:
  phony: true
  run: rm -rf dist/

4. Matrix Builds

Execute a target with multiple parameter combinations:

test:
  matrix:
    platform: [linux, macos, windows]
    version: [18, 20]
  run: echo "Testing on $platform with Node $version"

This will execute the command 6 times (3 platforms × 2 versions), with each combination available as environment variables.

File Targets with Matrix

When using matrix with file targets, ymake checks each file individually and only executes the necessary combinations:

files:
  name: files/${foo}.txt
  matrix:
    foo: [tar, biz, fod]
  run: |
    mkdir -p files
    echo hello > files/${foo}.txt

First execution: creates files/tar.txt, files/biz.txt, files/fod.txt

If you add a new element to the matrix:

files:
  name: files/${foo}.txt
  matrix:
    foo: [tar, biz, fod, woo] # new element: woo
  run: |
    mkdir -p files
    echo hello > files/${foo}.txt

Second execution: only creates files/woo.txt (the other files already exist)

This is useful for generating multiple files incrementally without regenerating existing ones.

5. Structured Outputs

Capturing Outputs

Capture JSON data from a target using the $OUTPUTS environment variable:

prepare:
  outputs: true
  run: |
    echo '{"version": "1.0.0", "build": 42}' > $OUTPUTS

With outputs: true, all JSON data is captured but not automatically exposed.

Specific Outputs

Specify which keys you want to capture and reuse:

prepare:
  outputs: [version, build]
  run: |
    echo '{"version": "1.0.0", "build": 42}' > $OUTPUTS

deploy:
  need: [prepare]
  run: |
    echo "Deploying version $prepare_version build $prepare_build"

Outputs are exposed as environment variables with the format {target}_{key}.

Outputs in Matrix

Use arrays in outputs to generate matrix combinations:

prepare:
  outputs: [platforms]
  run: |
    echo '{"platforms": ["linux", "macos", "windows"]}' > $OUTPUTS

build:
  need: [prepare]
  matrix:
    platform: $prepare_platforms
  run: echo "Building for $platform"

This will execute the build 3 times, once for each platform.

Dynamic Matrix from Outputs with Wildcards

You can combine outputs, matrices, and wildcard patterns to create powerful dynamic workflows. When a pattern target's matrix references outputs from another target, ymake automatically:

1. Executes the dependency to get the outputs
2. Uses those outputs to populate the matrix
3. Expands wildcard dependencies based on the dynamic matrix

all:
  phony: true
  need:
    - reports/d/*

generate_dates:
  outputs: true
  run: |
    echo '{"dates": ["2024-01-01", "2024-01-02", "2024-01-03"]}' > "${OUTPUTS}"

reports/d/{date}:
  need: generate_dates
  matrix:
    date: $generate_dates_dates
  run: |
    mkdir -p reports/d
    echo "Report for date: ${date}" > reports/d/${date}

When you run ymake all:

1. generate_dates executes first and produces the list of dates
2. The reports/d/* wildcard is expanded using the dates from the output
3. Each report is generated for the computed dates

This enables scenarios like:

• Generating reports for dynamically determined dates
• Building artifacts for regions/environments from a config service
• Processing files based on a manifest generated at runtime

See the dynamic-matrix-from-outputs sample for a complete example.

Mixing Outputs with Values

Combine output references with static values:

prepare:
  outputs: [version]
  run: |
    echo '{"version": "1.0.0"}' > $OUTPUTS

test:
  need: [prepare]
  matrix:
    env: [dev, staging, $prepare_version]
  run: echo "Testing in $env"

Output:

Testing in dev
Testing in staging
Testing in 1.0.0

6. Output Persistence (Cache)

ymake automatically caches target outputs to avoid unnecessary re-executions, making your builds faster and more efficient.

How It Works

When a target with outputs completes successfully, ymake saves its output to .ymake/info/outputs/<target>.json. On subsequent runs, if the cached output exists, ymake reuses it without re-executing the target.

Key Features:

• Automatic caching: Outputs are persisted automatically after successful execution
• Smart reuse: Cached outputs are loaded and made available to dependent targets
• Phony targets excluded: Targets with phony: true never cache outputs (always re-execute)
• Matrix support: Each matrix combination gets its own cache file
• Git-friendly: .ymake/.gitignore is automatically created with * to exclude cache from version control

Cache File Location

Outputs are stored in:

• Simple targets: .ymake/info/outputs/<target-name>.json
• Matrix targets: .ymake/info/outputs/<target-name>-<key1>_<value1>-<key2>_<value2>.json

Example: Basic Caching

prepare:
  outputs: [version, build_number]
  run: |
    # This expensive computation only runs once
    VERSION=$(git describe --tags)
    BUILD_NUM=$(date +%s)
    echo "{\"version\": \"$VERSION\", \"build_number\": $BUILD_NUM}" > $OUTPUTS

build:
  need: [prepare]
  run: echo "Building version $prepare_version (build $prepare_build_number)"

First execution:

• prepare runs and creates .ymake/info/outputs/prepare.json
• build uses the outputs from prepare

Second execution:

• prepare is skipped (cached output is used)
• build runs with the same cached outputs from prepare

Example: Matrix Caching

test:
  outputs: [test_result]
  matrix:
    platform: [linux, macos, windows]
    node: [18, 20]
  run: |
    # Run tests for this combination
    npm test --platform=$platform --node=$node
    echo "{\"test_result\": \"passed\"}" > $OUTPUTS

This creates 6 cache files:

• .ymake/info/outputs/test-node_18-platform_linux.json
• .ymake/info/outputs/test-node_18-platform_macos.json
• .ymake/info/outputs/test-node_18-platform_windows.json
• .ymake/info/outputs/test-node_20-platform_linux.json
• .ymake/info/outputs/test-node_20-platform_macos.json
• .ymake/info/outputs/test-node_20-platform_windows.json

Incremental builds: If you add a new platform, only the new combinations execute.

Phony Targets and Caching

Targets marked as phony: true always execute and never cache their outputs:

timestamp:
  phony: true # Always runs, never cached
  outputs: [current_time]
  run: |
    echo "{\"current_time\": \"$(date)\"}" > $OUTPUTS

deploy:
  need: [timestamp]
  run: echo "Deploying at $timestamp_current_time"

Cache Invalidation

To force re-execution and regenerate cached outputs:

1. Delete the .ymake/info/outputs directory
2. Delete specific cache files
3. Modify the target's command or dependencies

Static Outputs (No Caching)

Static outputs don't need caching since they're defined directly in the Ymakefile:

config:
  outputs:
    version: "1.0.0"
    environment: "production"

deploy:
  need: [config]
  run: echo "Deploying $config_version to $config_environment"

Complete Example

See Ymakefile.example for a complete example demonstrating all features.

# Prepare build information
prepare:
  outputs: [version, platforms]
  run: |
    echo '{"version": "1.0.0", "platforms": ["linux", "macos"]}' > $OUTPUTS

# Build using outputs
build:
  need: [prepare]
  run: echo "Building version $prepare_version"

# Matrix tests with outputs
test:
  need: [build]
  matrix:
    platform: $prepare_platforms
    node: [18, 20]
  run: echo "Testing on $platform with Node $node"

# Deploy with multiple dependencies
deploy:
  need: [test]
  run: echo "Deploying version $prepare_version"

# Cleanup (always executes)
clean:
  phony: true
  run: rm -rf dist/

Project Architecture

src/
├── ymake.ts       # Main CLI and entry point
├── parser.ts      # Ymakefile parser (YAML)
├── registry.ts    # Target registry
├── resolver.ts    # Dependency resolution
├── executor.ts    # Execution engine
├── matrix.ts      # Matrix build expansion
├── shell.ts       # Shell command execution
├── filesystem.ts  # File verification
└── errors.ts      # Custom error types

Main Components

• YmakefileParser: Parses and validates YAML files
• TargetRegistry: Stores and manages target definitions
• DependencyResolver: Resolves execution order and detects cycles
• ExecutionEngine: Coordinates target execution
• MatrixExpander: Generates combinations for matrix builds
• ShellExecutor: Executes shell commands with output handling

Exit Codes

• 0: Successful execution
• 1: Execution error (command failed)
• 2: Configuration error (Ymakefile not found, invalid YAML, target doesn't exist, circular dependency)

Development

This project was created using bun init with Bun v1.2.21. Bun is a fast all-in-one JavaScript runtime.

Setup for Development

# Clone the repository
git clone https://github.com/JonDotsoy/ymake.git
cd ymake

# Install dependencies
bun install

Run Tests

bun test

Run Locally

bun run src/ymake.ts [target] [options]

Integration Tests

Integration tests cover:

• Dependency resolution
• Matrix builds
• Output capture and reuse
• Ymakefile name detection
• Error validation

License

Private project.