Vale

Vale is a local evaluation and benchmarking workbench for repeatable ComfyUI runs.

It started as a tool for comparing already-trained LoRA variations, but the current shape is slightly broader: Vale is for defining controlled generation benchmarks, running them repeatedly, persisting the results, and reviewing the outputs with enough structure to make decisions.

What Vale Is For

Vale is designed to help answer questions like:

• Which LoRA performs best across the same prompts and seeds?
• How do sampler choices compare under controlled conditions?
• What happens when latent size, steps, or batch size change?
• Which checkpoint or configuration is worth keeping and iterating on?

The core loop is:

1. choose the variables to test
2. expand the benchmark into repeatable jobs
3. run those jobs through a local ComfyUI instance
4. preserve the results and metadata
5. review and compare the outputs
6. reuse the settings and iterate

What Vale Is Not

Vale is not meant to be:

• a replacement for the ComfyUI UI
• a generic ComfyUI workflow editor
• a training platform
• a cloud dashboard
• a multi-user service
• a general image asset manager

Vale integrates with ComfyUI. It does not try to absorb it.

Current Scope (0.1.0)

0.1.0 is intentionally constrained:

• one local operator
• one local ComfyUI installation
• one active benchmark run at a time
• one opinionated benchmark workflow contract for the main product path
• no authentication
• no cloud or remote hosting assumptions

The earlier generic import work is still preserved under src/services/custom-workflows/ as future-facing work, but it is not the primary 0.1.0 direction.

Current Capabilities

Configuration and expansion

• validates and stores a local ComfyUI root path
• stores a ComfyUI API URL
• builds repeatable benchmark plans from fixed and variable controls
• expands Cartesian products of prompts, seeds, latent configs, and related settings
• shows pre-run job count, expected image count, and runtime estimate

Generation runtime

• creates durable run folders under %LOCALAPPDATA%\\Vale\\runs
• snapshots the submitted benchmark configuration into run.json
• connects to or launches ComfyUI as needed
• executes jobs sequentially
• streams live updates to the browser over SSE
• supports immediate pause and resume
• recovers interrupted runs after:
  • Vale restarts
  • ComfyUI crashes
  • operator interruption

Monitoring

• live progress bar and percentage
• rolling ETA during active runs
• final elapsed run time after completion
• per-run and per-job timing data
• runtime stats during generation, including:
  • GPU util
  • CPU util
  • VRAM used
  • RAM used
  • GPU temperature
  • GPU power
  • GPU energy
  • render cost

GPU telemetry currently assumes NVIDIA tooling for the GPU-specific fields. Unsupported values are shown as Unavailable.

Review and history

• chronological generated-image grid with size controls
• optional batch grouping
• modal image review for single images and grouped batch outputs
• run history browser
• reopen saved runs for review
• load a saved run's settings back into Configuration
• delete completed or failed runs
• abandon paused or interrupted runs, including Vale-managed copied artifacts

Data and Storage

Vale stores:

• application settings in SQLite
• run and job metadata in SQLite
• portable per-run snapshots in run.json
• copied generated images inside each Vale-managed run folder

Vale does not delete anything from:

• ComfyUI output folders
• model directories
• LoRA directories

unless a future feature explicitly says so and the operator approves it.

Technology

• Node
• Express
• SQLite
• plain HTML
• plain CSS
• plain JavaScript

The goal is to keep Vale simple, inspectable, and local-first.

Getting Started

Requirements

• Node >=24.14.0 <25
• a local ComfyUI installation
• a working ComfyUI API endpoint

Install dependencies

npm install

Start Vale in development

npm run dev

Vale listens on:

• http://localhost:3100

Other useful scripts

npm run start
npm run lint
npm run test

Workflow Direction

For the main 0.1.x product path, Vale assumes a fixed benchmark workflow contract rather than an open-ended workflow authoring experience.

That is deliberate. The product is moving toward:

• repeatable execution
• comparison and review
• evaluation-oriented decision making

not toward becoming a full workflow editor.

Near-Term Product Direction

The runtime and persistence layers are now strong enough that the next major area is comparison and review.

That means future work is more likely to focus on:

• side-by-side comparison
• filtering and grouping of results
• selecting winners or favorites
• notes and verdicts
• exportable evaluation summaries

instead of broadening Vale into a general-purpose ComfyUI frontend.

Related Documentation

• Runtime ETA heuristics
• Runtime shape analysis

License

Vale is released under the MIT License.