UluOps · The operations layer for agentic work

@uluops/core

The foundational execution engine for UluOps. Orchestrates AI-powered code analysis through a 4-layer execution hierarchy (Agent > Command > Workflow > Pipeline), manages LLM tool loops via Vercel AI SDK, and integrates with the UluOps registry and tracker — every run's findings become tracked issues with fingerprints, so a resolved finding that comes back is a regression, not a new issue.

Prerequisites

• Node.js 20.3+ with an ESM project ("type": "module" in package.json)
• zod 4.6.7+ — a peer dependency since 0.52.0. npm 7+ installs it when your project has none; a project on zod 3 gets ERESOLVE and should upgrade (npm install zod@^4.6.7), not pass --legacy-peer-deps
• tsx for running TypeScript examples: npm install -D tsx
• UluOps API key — get one at app.uluops.ai, or use bundled starter agents offline (see below)
• AI provider key — at least one of ANTHROPIC_API_KEY, OPENAI_API_KEY, or GOOGLE_API_KEY

npm install @uluops/core
export ULUOPS_API_KEY=ulr_your_key_here
export ANTHROPIC_API_KEY=your_anthropic_key  # or OPENAI_API_KEY

Quick Start

Create validate.ts:

import { UluOpsClient } from '@uluops/core';

const client = new UluOpsClient({
  apiKey: process.env.ULUOPS_API_KEY,
});

// Run a single agent
const result = await client.runAgent('code-validator', './src', {
  model: 'sonnet',
  thresholds: { pass: 80 },
});

console.log(`Score: ${result.score} | Decision: ${result.decision}`);

// Or run a saved command configuration (model, thresholds, and aggregation
// come from the command definition — ideal for CI):
const cmd = await client.runCommand('validate', { target: './src' });
console.log(`Score: ${cmd.score} | Decision: ${cmd.decision}`);

npx tsx validate.ts

Plain JavaScript. The snippet above contains no TypeScript-specific syntax — save it as validate.mjs (or validate.js in a "type": "module" package) and run node validate.mjs. The offline snippet below is likewise valid JS as written. tsx is needed only once you add type annotations.

Offline Quick Start (No UluOps API Key)

Use the bundled starter agents without registry access. An AI provider key (e.g. ANTHROPIC_API_KEY) is still required:

import { UluOpsClient, STARTER_DEFINITIONS_DIR } from '@uluops/core';

const client = new UluOpsClient({
  localDefinitions: STARTER_DEFINITIONS_DIR,
  trackingEnabled: false,
});

const result = await client.runAgent('code-validator', './src');
console.log(`Score: ${result.score} | Decision: ${result.decision}`);

Note: runAgent/resolve return immediately from local definitions when found. client.list(), however, always queries the registry first and only falls back to the bundled definitions after the request fails — in a genuinely offline environment that means a few seconds of retry/backoff before the local list returns.

Workflows and pipelines. Local resolution applies the same WDL/PDL authoring→runtime normalization (e.g. steps[] → commands[]/agentRefs[]) the registry applies server-side, so runWorkflow/runPipeline resolve from local definitions too — provided every definition they reference (sub-agents, commands) is itself resolvable, locally or from the registry.

Table of Contents

• Prerequisites
• Quick Start
  • Offline Quick Start (No UluOps API Key)
• Overview
• Installation
• Authentication
  • UluOps API Key
  • AI Provider Keys
• Usage
  • Agent Execution
  • Command Execution
  • Workflow Execution
  • Auto-Routing
  • Pipeline Execution
  • Convenience Methods
  • Discovery
  • Cache Management
  • Result Tracking
  • Usage Metrics
  • Cost
  • Structured-output recovery
  • Extended Thinking
  • Integrity Verification
• OpenRouter (experimental)
• Architecture
• Execution Hierarchy
• Advanced Exports
  • Exported Constants
  • Direct Executor Usage
  • Model Resolution
  • Decision Classification
• Configuration
  • Local Definition File Naming
  • Environment Variables
• TypeScript Support
  • Subpath Exports
• Error Handling
  • Core SDK Errors
  • Re-exported from @uluops/sdk-core
  • Network Error Recovery
• Security
  • Tool Allowlist
  • Security events
  • Filesystem Sandboxing
  • Preflight Checks
  • Stage Steps (opt-in)
• Dependencies
• Development
• Maintainers
• License

Overview

The @uluops/core SDK provides:

• 4-Layer Execution Hierarchy - Agent > Command > Workflow > Pipeline orchestration
• AI SDK v6 Integration - Vercel AI SDK for LLM communication with automatic tool loops (maxSteps) and built-in retry
• Registry-Backed Model Resolution - Model aliases resolved via UluOps Registry with provider metadata
• Multi-Provider AI - Anthropic-first with deepest optimization (caching, context management, bash tools); OpenAI + Google bundled; Mistral, Cohere, and 10+ others via dynamic @ai-sdk/* import; OpenRouter (experimental) via @openrouter/ai-sdk-provider. See SCOPE.md for provider strategy.
• Filesystem Sandboxing - ToolHandler restricts LLM file access to the target directory with symlink-aware path validation
• Content-Addressed Integrity Verification - Registry-resolved definitions carry a SHA-256 YAML content hash (sha256:…) and, for agents/commands, a promptHash over the frozen rendered prompt. Hashing uses the shared @uluops/sdk-core implementation, so the client and registry hash identically. Remote resolution executes the frozen runtimeMd the promptHash certifies (not a live re-render). Callers can pin expectedHash/expectedPromptHash (from a trusted channel) on resolve()/ExecutionOptions; pins are verified fail-closed on every resolve path (cache/local/remote) and a mismatch throws IntegrityError. Verification is opt-in — unpinned resolves behave as before. See Integrity Verification.
• Universal Agent Output - Single agentOutputSchema with categories + artifacts for all 6 agent types (validator, executor, analyst, generator, explorer, forecaster)
• Structured Output Extraction - 4-strategy fallback: AI SDK structured output > JSON code fence > inline JSON > regex text parsing. When a structured answer fails the output schema it is salvaged or repaired before falling back (see Structured-output recovery below)
• Result Tracking - Automatic submission of every run's findings to the tracker: issue correlation by fingerprint, regression detection, per-agent execution recording, and analytics
• Analysis Summary Extraction - Automatic extraction of category scores, cognitive system metrics, epistemic assessments, and exploration maps from agent results at submission time. Execution telemetry (tokens, model, duration) travels first-class on agents[], never inside analysis data
• Local Development Support - Load definitions from local YAML files with registry fallback
• Bundled Starter Agents - 5 built-in agents for immediate use without registry access

Installation

npm install @uluops/core

Bundled starter agents (no registry needed): code-validator, docs-validator, public-interface-validator, security-analyst, test-architect.

Authentication

UluOps API Key

Required for registry and tracker access:

# Environment variable (recommended)
export ULUOPS_API_KEY=ulr_your_api_key_here

Or pass directly in config:

const client = new UluOpsClient({ apiKey: 'ulr_your-key' });

The SDK checks for ULUOPS_API_KEY then ULU_API_KEY environment variables. Keys use a ulr_ prefix. Generate one at app.uluops.ai.

AI Provider Keys

Set environment variables for the providers you want to use. The SDK auto-detects configured providers:

export ANTHROPIC_API_KEY=your_anthropic_key
export OPENAI_API_KEY=your_openai_key        # optional
export GOOGLE_API_KEY=your_google_key        # optional (also accepts GOOGLE_GENERATIVE_AI_API_KEY)

Or configure explicitly:

const client = new UluOpsClient({
  apiKey: process.env.ULUOPS_API_KEY,
  ai: {
    providers: {
      anthropic: { apiKey: process.env.ANTHROPIC_API_KEY },
      openai: { apiKey: process.env.OPENAI_API_KEY },
      google: { apiKey: process.env.GOOGLE_API_KEY },
    },
    defaultProvider: 'anthropic',
  },
});

OpenRouter models are covered in OpenRouter (experimental), after Usage.

Usage

Agent Execution

Direct agent execution with call-time options. Best for interactive/ad-hoc validation:

const result = await client.runAgent('code-validator', './src', {
  model: 'sonnet',
  thresholds: { pass: 80, warn: 60 },
  trackResults: true,
  project: 'my-project',
});

console.log(`Score: ${result.score} | Decision: ${result.decision}`);
console.log(`Recommendations: ${result.recommendations.length}`);

thresholds gates the run's decisionCategory, with the same bands a multi-agent command uses: below warn it is 'negative', and a 'positive' below pass becomes 'conditional'. It only ever lowers the category the agent's own decision resolves to; it does not rewrite decision (a PASS at 65 under pass: 80 stays "PASS", category 'conditional'), leaves a 'neutral' decision alone, and does nothing when the agent reports no score. Gate on decisionCategory, not the decision string. Without thresholds, the category is the agent's own.

Operator Prompt

Pass a prompt to give the agent a directive or focus. Especially useful for generators and executors that need to know what to create:

// Generator: tell it what to create
const generated = await client.runAgent('aristotle-generator', {
  target: './src',
  prompt: 'Create a health check endpoint for the Express API',
}, { model: 'opus' });

// Validator: provide focus context
const focused = await client.runAgent('security-analyst', {
  target: './src',
  prompt: 'Focus on the authentication middleware and JWT handling',
});

The prompt appears as a prominent Directive: section in the initial message, before project context.

Run Completeness & Degradation Markers

Every agent run carries a completeness signal — distinct from the agent's decision — describing whether the run actually finished its work:

const result = await client.runAgent('security-analyst', './src');

// Decision = what the agent concluded; completeness = whether the run finished its work.
console.log(`${result.decision} · ${result.completeness ?? 'complete'}`);

if (result.completeness !== 'complete') {
  for (const m of result.degradationMarkers ?? []) {
    console.log(`[${m.severity}] ${m.code}${m.detail ? ` — ${m.detail}` : ''}`);
  }
}

• completeness: 'complete' | 'partial' | 'failed', derived from degradation markers (any critical ⇒ failed; any degraded ⇒ partial; else complete). Absent ⇒ treat as complete. PASS + partial is a legitimate, gate-satisfying pass (decided 2026-07-10, recorded in types/degradation.ts): the agent passed the scope it could cover — forced wrap-up and context eviction are normal operation on large repos. Gates never downgrade on completeness; consumers that care about evidence span read completeness alongside the decision.

• degradationMarkers: typed { code, phase, severity, detail? }[]. code is the stable contract (e.g. budget.forced-wrap-up, steps.forced-wrap-up, context.evicted, steps.near-exhaustion, extraction.low-confidence, usage.provider-metadata-shape-drift, provider.warnings, model.unregistered-defaults, tools.shell-schema-fallback, render.raw-yaml-fallback, structured-output.repaired, structured-output.salvaged, structured-output.repair-failed); detail is human-only — never match on it. phase is 'resolution' | 'execution'.

• On a command result (runCommand, a command-ref pipeline stage, a workflow phase's command and agent steps), the agents' own markers and completeness live on agentResults, the agent results the command ran (0.56.0). The command's completeness is the worst among them and its own markers. Its degradationMarkers carry only core's stop marks (execution.run-stopped and siblings), never an agent's. The same worst-of rule gives an inline-agents stage's result.completeness, and a stopped or deadline-marked crash record reads failed. Tracker rows for a command are one per agent, under the agent's own name and version. Gating does not read agentResults: a ref stage is judged by the command's decision (PDL D9).

  budget.forced-wrap-up and steps.forced-wrap-up mean the tools were withdrawn before the agent finished exploring: the context budget crossed 80%, or the run reached 80% of its step limit. Both are degraded, so the run reads partial. The wrap-up withdraws the tools (activeTools: []), which holds on Anthropic structured output too. budget.brake-inert means the configured contextBudget was not a usable number.

• The engine observes completeness from how the run actually executed; agents never self-report it. deriveCompleteness(markers) is exported to recompute an agent result's. A container's (a CommandResult, a stage result) is the worst of its agents' and its own markers', so recompute it from agentResults.

• decisionCategory: the vocabulary-resolved category of decision ('positive' | 'negative' | 'conditional' | 'invalid' | 'neutral'), stamped on every result. 'invalid' is the reserved decision INVALID: the run could not judge the target (one of its own run auto-fails could not be corrected), its score is null, and every gate fails closed on it. For custom-vocabulary agents (cognitive lens agents, WDL-remapped workflow decisions) gate on this — or on resolveDecisionCategory(result) — instead of the raw decision string. See Decision Classification.

• degradations: string[] is the deprecated legacy alias (resolution-phase strings only), retained for compatibility — prefer degradationMarkers.

Empty-output step exhaustion is a thrown MaxStepsExhaustedError, not a marker; near-exhaustion with output is the steps.near-exhaustion marker. With the step wrap-up (0.54.0) both should be rare: at 80% of the limit the tools are withdrawn and the next turn is the answer.

Step limit

An agent run gets 100 tool-loop steps by default; a step is one model turn, which may make several tool calls. Set it per client (maxSteps in the config, or ULUOPS_MAX_STEPS) or per run (ExecutionOptions.maxSteps on runAgent); the per-run value wins, and every value is clamped to 1–500. The client value applies to every agent the client runs, including those inside commands, workflows and pipelines. A per-run value reaches runAgent and the inline agents of a PipelineExecutor called directly, but not yet runCommand, runWorkflow or runPipeline. An agent definition will be able to declare its own limit once ADL gains agent.defaults.max_steps (the minor after v1.20).

At 80% of the limit the tools are withdrawn, so the agent writes its answer from what it has read instead of being cut off mid-exploration with no output, and the result carries steps.forced-wrap-up (degraded). The limit is not derived from the model's context window, which bounds what a turn can hold, not how many turns a job needs. Raising it costs more than linearly: each step re-sends the conversation, so input tokens grow roughly with the square of the step count.

Command Execution

Execute saved command configurations. Uses model, thresholds, and aggregation from the command definition when it declares them (the execution block is optional; with no model declared, the agent's own model applies). Ideal for CI/CD:

const result = await client.runCommand('validate', { target: './src' });

// Override the definition's default model at runtime (e.g., for CI cost control)
const fast = await client.runCommand('validate', { target: './src' }, { model: 'haiku' });

// CI: pin the definition so a mutated registry entry is refused, not executed
// (see Integrity Verification). Strongly recommended when `bash` is enabled.
const pinned = await client.runCommand('validate', { target: './src' }, {
  expectedHash: 'sha256:…',
  expectedPromptHash: 'sha256:…',
});

console.log(`Score: ${result.score}`);
console.log(`Categories:`, result.categories);

Workflow Execution

DAG-based multi-phase orchestration with quality gates. Independent phases execute in parallel; dependent phases wait for their dependencies:

const result = await client.runWorkflow('ship', { target: './src' });

console.log(`Overall: ${result.decision} (score: ${result.score})`);

for (const phase of result.phases) {
  // decision: 'passed' | 'warned' | 'blocked' | 'skipped' | 'aborted'
  console.log(`  ${phase.name}: ${phase.decision} (${phase.score})`);
}

// Metrics break down phase outcomes
const { phasesExecuted, phasesPassed, phasesBlocked, phasesAborted } = result.metrics;

Workflows define phase dependencies and failure behaviors in their WDL definition:

WDL's abort is not PDL's: in a workflow, abort (like stop) skips every remaining phase, while a PDL stage gated abort skips only its own dependents. A pipeline stage that refs a workflow runs it under the workflow's meaning.

orchestration:
  on_failure: stop       # stop | abort | continue | warn
  max_parallel: 3        # optional concurrency limit
  phases:
    - id: lint
      commands: [lint-validator@latest]
    - id: test
      commands: [test-architect@latest]
    - id: security
      commands: [security-analyst@latest]
      depends_on: [lint, test]   # DAG dependency — waits for lint + test
      gate:
        threshold: 85
        on_fail: abort

Auto-Routing

Universal execution — auto-detects definition type and routes to the correct executor:

// Routes automatically based on whether name resolves to agent, command, workflow, or pipeline
const result = await client.run('code-validator', { target: './src' });
console.log(`Decision: ${result.decision}`);

Pipeline Execution

Synchronous execution (blocks until complete):

// `params` supplies run parameters read by stage/agent `condition:` expressions
// and by `{{ params.x }}` substitutions in stage `steps:` commands. Declared
// defaults are bound first; an absent param reads as false (see below).
const result = await client.runPipeline('foundations', {
  target: './src',
  params: { frontend: true, tier: 'pro' },
});

console.log(`Overall: ${result.decision} (score: ${result.score})`);
for (const stage of result.stages) {
  console.log(`  ${stage.name}: ${stage.status}`);
}

Stage & agent conditions (PDL 1.6.0). A stage's condition (and per-agent agents[].condition in inline-agent stages) is a run-gate: the stage or agent runs when it is true and is skipped when it is false.

• What it reads, exactly three things: a declared run parameter (params.frontend, from ExecutionInput.params, declared defaults bound first), whether an ancestor stage passed (stages.review.passed), and an ancestor's step output (stages.preflight.steps['Detect TypeScript'].output == 'DETECTED').
• Operators. == / != against a string literal, combined with !, &&, || and parentheses. Values compare as strings; '', 'false' and '0' are false.
• An absent param is false and equals nothing (so !params.x is true).
• A step that did not pass (failed, or never ran) reads as an error value. false && … and true || … absorb it; otherwise the stage or agent is not run and the stage is ERRORED (status: 'errored', erroredReasons: ['condition_error'], a STAGE_ERRORED warning on the run).

Conditions are checked before anything runs. start() / runPipeline() validate every stage and agent condition, depends_on (a stage declared later, or the stage itself, is a load error), undeclared keys on stages, gates and agent refs, and step templates, and throw a PipelineError whose context.loadErrors lists each { code, at, message } (codes as the PDL 1.6.0 spec names them, e.g. undeclared_parameter at stage:deploy). A condition core cannot read, an ordering operator, a number literal and skip_if are each a load error. See the PDL spec for the full grammar.

Dependencies. A stage runs once every depends_on stage has ended, whatever its status, with one exception: the transitive dependents of a stage that ended FAILED or ERRORED behind an abort gate are skipped (see the failure-mode table below). A skipped dependency, or one that failed behind fail or warn, no longer skips its dependents. Whether a stage runs is otherwise decided by its own condition.

Stage output forwarding (0.31.0, ON by default). Any inline-agent stage with depends_on automatically receives an ## Upstream Analysis section in each of its agents' initial messages — a severity-sorted slice (decision, decisionCategory, score, summary, top-5 recommendations) of every dependency's results, placed after the operator Directive: and before the project context. Forwarding is one hop (direct dependencies only) and never flows between parallel siblings. Controls:

stages:
  - id: gate            # producer-side opt-out: this stage's outputs are never forwarded
    forward: none
  - id: deep-analysis   # escalation: also forward head+tail-retained rawOutput (16K+8K chars)
    forward: full
  - id: synthesis
    depends_on: [deep-analysis]
    # receives: none    # consumer-side opt-out: depend for ordering only

Caps (provisional; exported as UPSTREAM_STAGE_SLICE_CAP 8K / UPSTREAM_STAGE_FULL_CAP 24K / UPSTREAM_TOTAL_CAP 32K chars) reduce deterministically — findings first, then narratives; stage headers and verdict lines are never dropped, and all truncation is marked in-place. Fleet-wide kill switch: ULUOPS_DISABLE_STAGE_FORWARDING=1 (or true). The forwarded slices ride ExecutionInput.upstreamContext (type UpstreamStageContext) — engine-populated; not an operator input.

Stage gates and the run verdict (PDL 1.6.0). A stage's gate: block (PDL $defs/gate) judges the stage once it has run. Every stage ends with one outcome (StageResult.outcome): PASSED, WARNED (a warn gate failed), FAILED, SKIPPED, or ERRORED (it could not be judged). What a gate reads from each agent is fixed by the agent-output contract (see Decision Classification):

• Score. If any agents[] entry declares threshold, the gate is in per-agent mode: each agent must reach its own threshold, or gate.threshold when it has none. Otherwise aggregate(scores) >= gate.threshold (aggregate: min | max | average, default min). A gate with no threshold is decision-only. A ref stage gates on its result's score, and a missing one there is not compared.
• The stage's own result is descriptive. result.score is always the average of its agents' scores, so a stage can report 78 and still fail a min gate of 70. result.decision reports the stage's outcome (PASS, WARN, FAIL, or ERROR for ERRORED; a stage a stop reached reads FAIL or ABORTED); each agent's own decision is on agentResults.
• Decision (gate.decision): not_negative (a negative fails; a conditional passes, and a blocking gate that passes one makes the run's allGatesPassed false, with a CONDITIONAL_ACCEPTED warning), positive (a conditional fails too), or ignore. Default: not_negative on blocking gates, ignore on warn gates. An INVALID decision fails the gate at every key, ignore included (its score is null, so no threshold can catch it). A command or workflow with an INVALID child decides INVALID itself, ahead of any negative, so a ref stage to it fails the same way. A neutral decision never fails a gate; the score decides. An absent one decides on its score only behind warn or in a stage with no gate: at a blocking gate it makes the agent unjudgeable (below).
• Unjudgeable agents. An agent makes its stage ERRORED when it gave no decision at a blocking gate (abort or fail), whatever its score; when it gave no score at a gate that has a threshold; when it gave no score and no classifiable decision; or when it crashed (a timeout included: agents[].timeout, in ms, overrides the run's timeout for that agent). A decision counts only from the agent's output envelope (the JSON object carrying result, or core's structured output) or its Decision: line; a JSON fragment that text extraction lifts out of a truncated block is not read.

stages:
  - id: build-gate
    steps: [...]
    gate:
      on_failure: abort      # FAILED; this stage's transitive dependents are skipped (PDL default)
  - id: validate
    ref: [email protected]
    depends_on: [build-gate]
    gate:
      threshold: 70
      on_failure: fail       # FAILED; the run continues, dependents run
  - id: review
    agents:
      - { ref: architect, threshold: 80 }   # per-agent thresholds
      - { ref: docs-validator, threshold: 75 }
    gate:
      on_failure: warn       # WARNED; decision ignored unless `decision:` says otherwise
      # on_success: skip_remaining  # early exit when the gate passes

| on_failure | stage on failure | rest of the run | |---|---|---| | abort (and absent) | FAILED | its transitive dependents are skipped; every other stage runs | | fail | FAILED | continues | | warn | WARNED | continues | | skip (deprecated) | as fail (load warning SKIP_DEPRECATED) | as fail | | no gate | WARNED (a failed step, a thrown stage or a negative decision; D43) | continues; never fails the run |

• An ERRORED stage takes its gate's mode: behind abort its dependents are skipped, behind fail it fails the run, behind warn it counts as WARNED.
• Every ERRORED stage raises a STAGE_ERRORED warning (PipelineResult.warnings, with reason: 'agent_unjudgeable' | 'condition_error' | 'stopped'). The stage's skipReason carries the same reasons as text, naming each unjudgeable agent's cause: agent_unjudgeable: "a@1" gave no decision at a blocking gate, "b@1" crashed (the other causes read gave no score at a gate with a threshold and gave no score and no classifiable decision). Match on outcome and erroredReasons, not on that text.
• A stage whose every agent was skipped by its own condition is SKIPPED, its gate not evaluated.

The run verdict.

• PipelineResult.allGatesPassed is false when a stage ended FAILED, a stage ended ERRORED behind abort or fail, a blocking gate accepted a CONDITIONAL, or a stop (cancel, deadline, credit limit) reached a stage before the run ended; otherwise true.
• decision follows it: FAIL when false, WARN when a stage WARNED or ended ERRORED behind warn, else PASS.
• An abort does not throw. wait() and runPipeline() resolve with status: 'complete' and decision: 'FAIL'; skipped stages carry skipReason: 'gate_abort' | 'gate_early_exit' | 'condition_not_met' | ....
• A tracked run saves allGatesPassed exactly as computed, and its raw markdown begins with a params: line holding the resolved run parameters.
• An unexecutable mandatory gate refuses the run. A steps: stage behind a blocking gate (abort, fail, skip) that cannot execute because allowStageSteps is off refuses the run before any stage starts (see Stage Steps).

Async execution with handle-based control:

// Start async pipeline
const handle = await client.startPipeline('full-validation', {
  target: './src',
});

// Monitor progress
const status = await handle.status();
console.log(`Stage ${status.stages.length} of pipeline`);

// Wait for completion
const result = await handle.wait();

// Or cancel — aborts the in-flight provider request, not just the stage loop.
// The running agent's HTTP call ends; the run reports `status: 'cancelled'`.
await handle.cancel();

Pass your own abortSignal on ExecutionOptions to tie a run to a lifetime you already have (an inbound request, a parent job). It is merged with the pipeline's own signal, not replaced, so handle.cancel() keeps working on the same run. An explicit abort() of your signal stops the run exactly like cancel() — status cancelled, later stages skipped — and a cancel() on a cancelled run, or while any stopped run is still unwinding, is a no-op. A deadline is different: if your signal is AbortSignal.timeout(ms) (its abort reason is a TimeoutError), the run ends failed, wait() throws a PipelineError naming the deadline, and the agents still in flight are recorded as crashes — the slow agent is the likeliest broken one, so a deadline is never filed away as a neutral stop.

Convenience Methods

Shorthand methods for common workflows:

// Run the built-in validate command
const result = await client.validate('./src');

// Security audit
const secResult = await client.security('./src');

// Code optimization analysis
const optResult = await client.optimize('./src');

// Ship workflow (full pre-release validation)
const shipResult = await client.ship('./src');

// Post-implementation validation
const postResult = await client.postImplementation('./src');

Discovery

// List available definitions
const definitions = await client.list({ type: 'agent', domain: 'software' });

// Inspect a definition
const info = await client.describe('code-validator');
console.log(info.name, info.version, info.interface);

Cache Management

// Clear the definition resolution cache — call after registry updates in long-lived processes
client.clearCache();

Result Tracking

Submit execution results, preview submissions, and query run history:

// Automatic tracking: results are submitted automatically when trackingEnabled is true
const result = await client.runAgent('code-validator', './src', {
  trackResults: true,
  project: 'my-project',
});

// Manual submission: submit a result you ran yourself (an agent result is accepted since 0.57.0).
// Its analysis (summary + records) is extracted and sent, with the decision vocabulary and
// category weights core stamped on the result when it ran.
const response = await client.submitResults('my-project', 'post-implementation', result);
console.log(`Run #${response.runNumber}: ${response.dashboardUrl}`);
console.log(`New issues: ${response.correlation.newIssues}, Regressions: ${response.correlation.regressions}`);
console.log('Dropped to fit the tracker:', response.truncated); // see "Truncation tallies" below

// Dry run: preview what a submission would do without saving
const preview = await client.previewSubmission('my-project', 'post-implementation', result);
if (preview.validationErrors.length > 0) {
  console.error('Validation errors:', preview.validationErrors);
}

// Query history: list past runs for a project
const history = await client.getHistory('my-project');
for (const entry of history) {
  console.log(`Run #${entry.runNumber} — ${entry.workflowType} — Score: ${entry.averageScore}`);
}

// Run details: fetch full details for a specific run
const run = await client.getRun('run-uuid');

response.correlation carries counts. The per-finding detail — which recommendation matched which existing issue by fingerprint, and which resolved issue a run caught again — is typed here as FingerprintedRecommendation and RegressionInfo (re-exported from the package root) but is read from the tracker through @uluops/ops-sdk directly (runs.get, issues.getHistory); core submits, it does not fetch it back.

allGatesPassed on history entries and run reads is boolean | null (since v0.34.0): null = NOT_A_GATE — the run carried no gate-bearing agents (e.g. a cognitive-lens-only run), distinct from false (a gate ran and failed). Render null neutrally and exclude null runs from pass-rate math. Submission inputs are unchanged — the client still asserts an explicit boolean verdict; null is never a valid input value.

On the auto-tracking path, a failed submission (e.g. a free-tier 402 PROJECT_LIMIT, SUBSCRIPTION_REQUIRED, or a transient 5xx) is non-fatal — the agent run still resolves successfully. The result carries trackingFailed: true plus a typed trackingError ({ code, statusCode, message, requestId, details }), so callers can surface the reason — and any details.upgradeUrl — instead of silently dropping the dashboard link. code (e.g. PROJECT_LIMIT) is the stable contract; message is human-readable and should not be matched on.

What lands in analysis data. At submission time, AnalysisSummaryExtractor builds the run's analysisSummary + analysisRecords:

• One summary per agent (since 0.57.0). An agent run sends one summary; a command, workflow or pipeline run sends one per agent it ran, including agents inside a pipeline's workflow stages (which got none before 0.57.0), up to the tracker's 20. Before 0.57.0 a composite run sent only its first agent's summary.

• Each agent's own definition facts. Core stamps decisionVocabulary and categoryWeights on every AgentResult it builds; extraction reads them from the result, so each agent in a composite run carries its own vocabulary and declared weights (before 0.57.0 they were taken from the parent definition, which yielded null and equal shares). An agent that emits its own category_scores in its analysis block keeps those, weights included. A declared weight is matched to a category by normalised name (case and punctuation ignored); an unmatched category gets the equal share. A result not built by core's executors (hand-built, or serialized by core ≤ 0.56.2) is still extracted; those two fields are omitted and one warning names the agents. Stamp such a result with definitionFacts(definition) (exported).

• Repaired to the tracker's limits, never dropped for them. The tracker rejects some values, and a rejection costs that agent's whole analysis (its records and summary, since ops-api R2; before that, the whole save), so core repairs them before sending: category scores and weights rounded to integers, at most 20 exploration maps of at most 50 sections with object items, audit implications cut to 500 characters, and a systemMetrics / epistemicAssessment / record data blob over 64 KiB or 8 levels deep replaced (blob → null, record data → a stub). A summary that is still invalid after repair is sent as it is (since 0.58.0): the tracker judges it, and a rejection comes back in analysisRejected with that agent's records (see "Rejected analysis"). The whole payload is kept under the tracker's 2 MB body limit by dropping summaries, then records, from the end.

• Record ids are unique per agent. A record that repeats an earlier record's id (compared as the tracker does: case- and accent-insensitive) gets ~2, ~3, …, so the first occurrence keeps its id. Analysis-block records are read from recordId/record_id and recordType/record_type (the corpus templates' snake_case: before 0.57.0 these lost the whole save), and are always attributed to the agent that produced them; a different agentName in the record is kept in data.declaredAgentName.

• systemMetrics carries the agent's cognitive measurements only — its analysis-block system_metrics, else structured-output domainMetrics, else null. Execution telemetry (tokens, model, duration) is never merged in; it travels on agents[].

• Extraction facts (extraction_confidence, extraction_method) are epistemic facts about the parse and merge into epistemicAssessment — the agent's own keys always win.

• Every agent-authored field on a record is sanitized against the API's storage contract, at one seam, for all four extraction tiers (v0.36.0). The tracker validates the whole analysisRecords array before the network, so a single out-of-contract value used to lose the entire run's analysis rather than one record. Each field is normalized in the way that preserves the most meaning, and anything rejected is kept under a raw* key rather than dropped:

  | Field | Contract | On violation | |---|---|---| | severity | the tracker enum | case-normalized; register-style values (structural, NOTABLE) → null, original in data.rawSeverity | | recordType | non-empty, ≤50 | trimmed and lowercased; no vocabulary check — see below; unusable → evidence_finding, original in data.rawRecordType | | title | non-empty, ≤500 | prose, so truncated to the bound with the full text in data.rawTitle; blank → (untitled record) | | classification | ≤50, optional | categorical, so nulled rather than truncated — a clipped category is a different category — original in data.rawClassification | | data | a plain object | entries-form [{key,value}] converted; any other array, primitive or null → {} or { rawData } |

  recordType is deliberately not checked against a fixed vocabulary. The tracker stores it as a bounded string so registry-defined agents can emit new record shapes without a coordinated release. If you consume recordType, expect a wide, open set (over 300 distinct values are in storage).

Truncation tallies. response.truncated (and result.submissionTruncated on the auto-tracking path) counts what the client dropped so the save could go through: agents (wire cap 100), recommendations (500), analysisRecords (100, plus any dropped for the body limit), analysisSummaries (past 20, or dropped for the body limit) and analysisSummariesInvalid (always 0 since 0.58.0, deprecated). Zeros are measured: { agents: 0, … } means nothing was dropped. The tally is absent on getRun(), which reads a stored run. A dropped agent, recommendation or record also adds a marker recommendation to the run so issue correlation sees it; a dropped summary does not (summaries do not feed correlation), so the tally and the warning log are the only record that one was dropped.

Rejected analysis (0.58.0). The tracker (ops-api, since 2026-11-03) judges analysis per agent: one invalid entry rejects that agent's records and summary as a unit, and the run is still saved. response.analysisRejected (and result.analysisRejected on the auto-tracking path) carries the server's list, ops-sdk's AnalysisRejection[] (agentName, an open-string reason, the counts not stored, detail), and each rejected agent is logged at warn. It is not a tracking failure: trackingFailed stays unset. [] means the server reported nothing rejected; the field is absent for an older server, a local (tracking-off) response and getRun(). Unlike truncated, this is the server's verdict, not a client-side drop. Repair with ops-sdk's runs.update on the run, sending each rejected agent's complete set; submitting again creates a second run. Since 0.58.1 result.trackedRun ({ runId, runNumber, project }) names that run, on every result that was saved; do not look it up as the project's latest run, which can be another one. Core still repairs what it can before sending (rounds scores, truncates text). It no longer drops summaries it predicts the tracker would reject (0.57.0 did; analysisSummariesInvalid is now always 0): an invalid summary is sent and comes back in analysisRejected, with that agent's records.

Usage Metrics

All execution results include token usage metrics. Provider-specific token fields are mapped to a unified format:

const result = await client.runAgent('code-validator', './src');
const { metrics } = result;

console.log(`Input: ${metrics.inputTokens}, Output: ${metrics.outputTokens}`);
console.log(`Cache: ${metrics.cacheCreationTokens ?? 0} created, ${metrics.cacheReadTokens ?? 0} read`);
console.log(`Harness: ${metrics.harness ?? 'unknown'}`); // producing runtime ('uluops-core'), v0.26.0

// inputTokens is CACHE-EXCLUSIVE — input the model processed fresh, excluding cache reads
// and writes. It is NOT the provider's headline input count (AI SDK v6 reports that as a
// cache-INCLUSIVE total); do not subtract a cached figure from it again, or you undercount.
// cachedInputTokens is a recorded component only and no longer participates in the total.
// reasoningOutputTokens (non-Google families) and thinkingTokens (Google family, direct or via OpenRouter) are subsets of gross
// outputTokens — exposed for cost breakdown only, NEVER added (the AI SDK already folds them in).
if (metrics.cachedInputTokens) console.log(`Cached input: ${metrics.cachedInputTokens}`);
if (metrics.reasoningOutputTokens) console.log(`Reasoning: ${metrics.reasoningOutputTokens}`);
if (metrics.thinkingTokens) console.log(`Thinking: ${metrics.thinkingTokens}`);

// Canonical: inputTokens (cache-exclusive) + outputTokens (gross) + cacheCreationTokens.
// Summed across every step of the tool loop, not just the last one.
console.log(`Effective total: ${metrics.totalEffectiveTokens}`);

// Provider warnings — settings the provider could not honor (unsupported parameter,
// clamped max_tokens, unknown context-management strategy). Provider option schemas
// STRIP unknown keys rather than rejecting them, so a warning is the only runtime
// evidence that a request setting silently didn't apply.
//
// On an AgentResult these surface as an info-severity degradation marker (the same
// route usage.provider-metadata-shape-drift takes). The raw string[] lives on
// AIGenerateResult if you are calling AIProvider.generate() directly.
const warned = result.degradationMarkers?.filter((m) => m.code === 'provider.warnings');
if (warned?.length) {
  console.warn('Provider warnings:', warned.map((m) => m.detail));
}

Cost

Every result level (agent, command, workflow, pipeline, stage) carries four cost fields:

| Field | Meaning | |---|---| | costUsd | The registry estimate: usage priced at the catalog's rates. undefined when any child's model is unpriced. | | costUsdBilled | The billed amount, summed over every request. Only OpenRouter reports one today. On a BYOK request it is OpenRouter's charge plus the upstream provider's charge to your own key — total spend across both invoices, not OpenRouter's invoice alone. undefined unless every request reported it. | | costUsdTotal | The best available total: per agent the bill if there is one, else the estimate; per parent the sum of children's totals. undefined only when some child has neither. | | costBasis | What costUsdTotal is made of: 'billed', 'estimated', 'mixed', 'unpriced' (no total), or 'none' (no model was called, e.g. a steps stage; the total is a real 0). |

const { costUsdTotal, costBasis } = result.metrics;
// Six decimals: a single OpenRouter request can bill $0.000003, which toFixed(4) shows as $0.0000.
console.log(costUsdTotal === undefined ? 'cost unknown' : `$${costUsdTotal.toFixed(6)} (${costBasis})`);

Read costUsdTotal for spend, and costUsd and costUsdBilled side by side to reconcile the estimate against the bill. Reconcile against an invoice only when costBasis is 'billed': a 'mixed' total adds bills to estimates. Limits worth knowing:

• Cost is in-process only. The Tracker wire format has no cost field, so none of the four is submitted with a run; the client logs a warning on each run that carries one.
• A failed or cancelled agent has no cost figure at all. The in-flight request's cost is unknowable and the completed requests' totals are not carried across the throw, so the agent reads 'unpriced' and every parent above it does too (no total). Money was spent; core cannot say how much. The exception is an agent stopped by the step ceiling (MaxStepsExhaustedError), whose requests all completed: it keeps its real cost and basis.
• A structured-output fallback is not a failure. When structured output cannot be parsed and the run falls back to text extraction, every request has already completed, so it keeps its billed figure under the same every-request rule. A repair turn (below) is one more request, and its cost is included.
• SDK retries are invisible. A request the AI SDK retried is billed but never reported to core, so both figures can understate a run that hit retries.

Structured-output recovery

A structured answer that fails the output schema is salvaged or repaired before falling back (core 0.56.0). On Claude, core asks for structured output through a json tool (structuredOutputMode: 'jsonTool'), because native enforcement with the filesystem tools answers HTTP 400 "The compiled grammar is too large". That tool is not strict, so the provider enforces nothing: an answer can miss a required field or break a nested one (both seen live; see the 0.56.0 CHANGELOG entry).

When the SDK rejects the answer, core tries, in order:

1. Salvage. If the answer's required fields (decision, score, maxScore, read off the schema) are valid, core keeps it. An optional value sent as a string that encodes the expected array or object is decoded in place. Otherwise the smallest failing piece is dropped: the key, else the array element, else the top-level optional field. No request is sent. The run carries structured-output.salvaged, which is degraded when something was dropped and not carried back (its detail names each path) and info otherwise. Salvage never supplies a verdict.
2. Repair. Otherwise core sends one more request: the run's conversation so far, a message naming each schema issue, and no tools. A valid answer becomes the structured output (structured-output.repaired, info). An answer that fails only on optional content is salvaged as in step 1. Optional content the repair answer leaves out, or salvage drops, is carried over from the first answer when the first answer had it in valid form (structuredOutputRepair.carriedOver names the fields). The verdict always comes from the repair answer.
3. Fallback. Otherwise the run falls back to text extraction as before (structured-output.repair-failed, info; the extraction.* markers carry any loss). This includes the case where no repair turn was possible because the provider reported no conversation (repairTurn: false).

AIGenerateResult.structuredOutputRepair records the outcome, whether a repair turn was sent, and each salvage action. The repair is attempted once, only for a schema or parse failure, and only when the provider reported the conversation. salvageStructuredObject and carryOverOptional are internal; the SalvageAction type is exported. A cancellation or a provider credit refusal during the repair is thrown, as on the first request; any other repair error falls back. The repair request does not feed the context-budget tracker, because withdrawing the tools shrinks its input and the tracker would read that as an eviction.

Extended Thinking

Off by default; opt in per client or per run (core 0.51.0+). Thinking is billed as output on every step of a tool loop, and scores measured with it are not comparable with scores measured without it, so nothing turns it on for you.

| Lever | Values | Notes | |---|---|---| | runAgent(name, target, { extendedThinking }) | true / false | Wins over everything below. runAgent only — commands, workflows and pipelines take the client setting. Any non-boolean (the string "false" included) is off with a warning. | | ai.extendedThinking | 'on' / 'off' | Every run this client makes. Booleans are malformed here (off + warning). | | ULUOPS_EXTENDED_THINKING | on / off | Same, from the environment; the config field wins. A value in a .env file the CLI loaded is sticky across runs — the CLI warns when that is where it came from. |

A malformed value at any layer is off at that layer and does not fall through. A provider-native option you pass yourself (providerOptions.openai.reasoningEffort, google.thinkingConfig, openrouter.reasoning, anthropic.thinking) always wins and is not capped. An explicit native "off" (thinking: {type:'disabled'}, reasoningEffort: 'none', thinkingBudget: 0, reasoning: {enabled: false}) is recorded as not applied ('caller-native'); any other native thinking option is recorded as applied from source 'native', and logs a notice.

What "on" sends, per provider, on a model that can think (capabilities.reasoning, or the reasoning tier):

| Provider | Sent | Caps | |---|---|---| | OpenAI | reasoningEffort: 'medium' | — (effort, not budget; defaultThinkingBudget is ignored) | | Google | thinkingConfig.thinkingBudget | Capped at half of maxTokens: Gemini counts thinking inside maxOutputTokens, and a budget at the cap leaves almost no answer. | | OpenRouter | reasoning.max_tokens | Capped so the answer keeps at least half of maxTokens (budget ≤ sent max_tokens − ½ maxTokens) — OpenRouter counts thinking inside max_tokens, and a budget at or above it is not rejected: OpenRouter silently raises the cap and bills past it. On anthropic/… models max_tokens is raised by the budget (up to the model's output limit) and the interleaved-thinking beta is sent, so tool steps after the first can think too (probed: claude-sonnet-4.5 thought on every step; the adaptive claude-sonnet-5, claude-opus-4.8 and claude-haiku-5.5 accepted the request and thought on some steps, choosing how much themselves; other anthropic/ ids unprobed). A max_tokens you set in providerOptions.openrouter replaces core's on the wire, so core does not raise it and sizes the budget against yours (the answer keeps at least half); a value that is not a positive integer gets no budget. | | Anthropic (direct) | nothing | Not mapped yet: it needs the structured-output degrade (Anthropic rejects thinking with forced tool use), which is planned but not built. A run records "not applied: no-mapping". Use an openrouter:anthropic/… model for Claude thinking. | | Others | nothing | Recorded "not applied: no-mapping". |

Below a 1024-token budget after capping, nothing is sent and the run records 'invalid-budget'.

"Off" means the provider default, not "no thinking". Off sends nothing — and gpt-5 and gpt-5.5 reason anyway (measured); Gemini 2.5, DeepSeek/xAI reasoners and the always-thinking Claude models (Sonnet 5.5, Opus 5.5, Fable 5) are expected to. Those runs record offMeans: 'provider-default', by heuristic (upstream plus the provider's always-thinking list), not by measurement — read thinkingObserved for what the tokens showed.

Every agent result records what happened, on result.runConditions: what was requested, from which layer, whether a thinking option was actually sent (thinkingApplied) and why not, the budget sent, whether the interleaved beta was sent, the max_tokens core sent when it raised it (maxTokensSent — a thinking-on run that scores better may simply have had more room), and whether thinking tokens were observed (thinkingObserved). Crash and stopped-run placeholders carry it too (core reads it off the thrown error; there is no public reader). It is in-process only: the tracker does not receive it yet.

The applied-keyed notice: a run that requested thinking logs one line — "on" when it was sent and a warning naming the reason when it was not. Thinking turned on by the environment, or by your own provider-native options, warns once per client (then debug), so a pipeline does not repeat it for every agent. The info-level "on" line for request- and config-sourced thinking shows only with debug: true.

Not covered by the switch: the agent's model choice. An agent definition's defaults.model outranks the client's model choice, so a definition naming a model that reasons by default is billed reasoning even with thinking off. The counter is ai.modelOverride. Thinking cost appears in costUsd like any output (see Cost); reasoning tokens are in reasoning_tokens (every non-Google model family) or thinking_tokens (Google-family models, direct or google/* through OpenRouter).

Integrity Verification

Pin a definition's expected hashes so execution is refused if the resolved content doesn't match. Pins come from a trusted, independent channel (a lockfile, a reviewed value) — recomputing against the registry's own returned hash only catches an internally-inconsistent registry, not a compromised one.

import { IntegrityError } from '@uluops/core';

try {
  const result = await client.runAgent('code-validator', './src', {
    expectedHash: 'sha256:…',        // pins the YAML (source + config)
    expectedPromptHash: 'sha256:…',  // pins the rendered prompt (agents/commands)
  });
} catch (err) {
  if (err instanceof IntegrityError) {
    // err.kind: 'yaml' | 'prompt' | 'unavailable'; err.expected / err.actual
    // err.definitionName / err.definitionVersion identify which definition failed
    console.error(`Execution refused (${err.kind}) for ${err.definitionName}@${err.definitionVersion}: ${err.message}`);
  }
}

• Every execution entrypoint accepts pins: runAgent/runCommand take them in their options/overrides; runWorkflow/runPipeline/startPipeline/run take a trailing ResolvePinOptions. Pipeline pins cover the pipeline YAML only — stage refs are resolved separately downstream and are not individually pinned (per-stage pinning is lockfile territory).
• Both pins are optional. Unpinned resolves are unverified and behave exactly as before.
• expectedHash verifies computeHash(resolved.yaml) — covers source and execution config. For WDL/PDL the YAML is the runtime, so the YAML pin alone fully covers execution.
• expectedPromptHash verifies the frozen rendered prompt and is required (with expectedHash) for full agent/command executed-prompt integrity. Supplying it for a definition with no rendered prompt (workflow/pipeline, local, content-gated, schema-stale) throws IntegrityError(kind: 'unavailable') — never a silent pass.
• Verification runs on every resolve path, including cache hits — a prior unpinned resolve cannot let a later pinned one through unchecked.
• ResolvedDefinition also surfaces promptHash and translatorVersion so callers can detect a retranslation restamp.

Trust bootstrap. This ships the verification mechanism and explicit pin inputs, not pin provenance. A pin seeded from a first unpinned resolve() against an already-compromised registry is trust-on-first-use. A pin manifest (lockfile) is the natural completion.

OpenRouter (experimental)

Route any OpenRouter model through core with an openrouter:<slug> model string. Install the provider yourself, at the version core is built against:

npm install @openrouter/[email protected]
export OPENROUTER_API_KEY=your_openrouter_key   # auto-detected like the keys above

// `client` is the UluOpsClient from Quick Start.
const result = await client.runAgent('code-validator', './src', {
  model: 'openrouter:deepseek/deepseek-v4-flash',
});

Data governance. Every file the agent reads, and every shell command's output, is sent to OpenRouter and on to whichever upstream provider serves the request. OpenRouter itself retains no prompts unless you opt in to its prompt logging; the upstreams differ. Since 0.49.0 core sends provider.data_collection: 'deny' by default, so only upstreams that do not retain or train on your data are eligible. Fewer endpoints qualify; when none does (OpenRouter's routing step "Filter by Data Policy" — typical of :free models), the run fails with a CapabilityError that names provider.data_collection = 'deny' and how to opt in.

To opt in for one run, pass it per request in runAgent's options: providerOptions: { openrouter: { provider: { data_collection: 'allow' } } }. Commands, workflows, pipelines and the CLI take no providerOptions; for them the levers are ai.openRouterDataCollection: 'allow' in the client config or OPENROUTER_DATA_COLLECTION=allow in the environment, and both apply to every run that client or shell makes. While allow is in effect, the first OpenRouter request it applies to from each lever logs a warning naming that lever. Per request beats config, config beats the environment. At every layer an empty or null value means unset; any other value than allow/deny means deny at that layer — it does not fall through to a lower layer's allow.

These levers are a preference, not a lock: a more specific one always overrides a less specific one, so an environment deny cannot stop an embedding app's allow. To enforce deny, use your OpenRouter account's privacy settings (openrouter.ai/settings/privacy), which apply on top of all of this. deny is OpenRouter's per-provider data-policy flag, not zero data retention: for the strictest routing add zdr: true (zero-retention endpoints only) and pin upstreams with provider.only. Core keeps any provider fields you set and adds its own require_parameters.

• Pin 2.10.0. The package's npm latest is 3.x, which targets ai@7; core runs ai@6 and refuses a different major at load with an error naming both versions. Under strict pnpm isolation the package must be hoisted, because core resolves it with a dynamic import() from its own install location (the same as the other non-bundled providers).
• Catalog data. OpenRouter slugs resolve against the registry catalog (core 0.45.0+), so a listed slug gets its real capabilities, context window and price. A slug the catalog does not list runs on defaults and the run carries an info marker, model.unregistered-defaults (programmatic callers of AIProvider.generate() also get modelRegistered: false on the result).
• Routing guards. Core always sends provider.require_parameters: true (route only to endpoints that support every parameter sent, the only capability guard for a routed model) and usage.include: true; caller options cannot turn either off. Extended thinking is opt-in (see Extended Thinking): when it is on, a capable model gets reasoning.max_tokens from defaultThinkingBudget, and anthropic/… models also get a raised max_tokens and the interleaved-thinking beta. Adding reasoning narrows the endpoints require_parameters admits; a no-endpoint error names it and the switch that turns it off.
• Shell. OpenRouter has no provider-defined shell tool, so routed agents that request bash get a schema-fallback bash tool. The allowedTools gate applies exactly as for Anthropic's native tool (bash stays off unless you allow it), and runs offered it carry an info marker, tools.shell-schema-fallback. Commands run unsandboxed in the target directory, as with the native tools, but are generated by whichever upstream OpenRouter routed to; allow bash only with upstreams pinned (provider.only) and a target you would let that model run commands in.
• Errors. Each OpenRouter failure arrives typed (core 0.48.0+):
  • Out of credit (402) is a ProviderCreditError carrying OpenRouter's limitSource and its own text. A pre-flight refusal ("can only afford N") means the balance cannot cover this request's worst case (maxTokens × price): lowering maxTokens works, credit remains. Otherwise the credit or the key's limit is spent. Inside a pipeline a 402 stops the whole run and fails it; what that does to later stages, in-flight siblings and their records is in Stopped agents.
  • Outside a pipeline there is no run to stop. runAgent and single-agent commands reject with ProviderCreditError. A multi-agent command or a workflow contains a failing agent the way it contains any crash: the result carries a failing verdict and a crash recommendation whose title includes the provider's text, and it throws only if every agent failed.
  • Spend on a 402 mid-run is not recorded. The completed requests of an agent that hit the 402, and of siblings the stop aborted, are not carried across the throw, so the run's cost reads unpriced (see Cost).
  • No endpoint (404) is a CapabilityError naming the routing step and every constraint the request carried: the parameters require_parameters holds endpoints to, any provider.only / ignore / quantizations, and for an allowed-providers miss, the providers that do serve the model.
  • Unknown slug (400 "is not a valid model ID") is a ModelNotFoundError naming the slug.
  • Rate limited (429) is a RateLimitError whose retryAfter (seconds) comes from OpenRouter's X-RateLimit-Reset, since it sends no retry-after; the message names whose limit it was (limit_source).
  • A context-length 400 on an uncatalogued slug names the budget it ran with, and an error OpenRouter returns inside an HTTP 200 body maps by its own code.
• Cost. OpenRouter's billed amount is summed per request into costUsdBilled, and costUsdTotal prefers it over the registry estimate (see Cost). On a BYOK request the upstream provider's charge is added; on a normal one it is not, since OpenRouter reports the same figure twice there.
• Not yet: route tagging in the Tracker. A routed run is recorded under its openrouter:<slug> model string.

Architecture

For a detailed, hop-by-hop trace of every execution chain (resolution → LLM generation → persistence), see ARCHITECTURE.md (in the repository; not shipped to npm).

UluOpsClient (facade)
  |
  +-- AgentExecutor        (single-agent LLM execution)
  |     +-- AIProvider     (AI SDK v6 wrapper, provider registry, context management)
  |     +-- ToolHandler    (sandboxed filesystem tools with symlink protection)
  |     +-- ToolAdapter    (converts tools to AI SDK format)
  |     +-- OutputExtractor (4-strategy: structured output > JSON fence > inline JSON > regex)
  |
  +-- CommandExecutor      (single/multi-agent aggregation via Promise.allSettled)
  |     +-- AgentExecutor
  |     +-- preflight      (prerequisite checks with path traversal protection)
  |
  +-- WorkflowExecutor     (DAG-based parallel phase orchestration with quality gates)
  |     +-- CommandExecutor
  |
  +-- PipelineExecutor     (multi-stage async pipelines)
  |     +-- WorkflowExecutor
  |     +-- CommandExecutor
  |
  +-- RegistryClient       (definition resolution + content hash)
  +-- SubmissionClient     (result submission + history)
  |     +-- AnalysisSummaryExtractor (auto-extract analysis from AgentResult)
  +-- ModelCatalog         (registry-backed model alias resolution)

Execution Hierarchy

| Level | Definition | Description | |-------|-----------|-------------| | Agent | ADL | Atomic unit: single LLM with filesystem tools. 6 types: validator, executor, analyst, generator, explorer, forecaster — all produce a universal AgentResult (score/maxScore are number \| null: null for generators/executors that produce artifacts not scores), with categories and optional artifacts | | Command | CDL | Wraps 1+ agents with preflight checks and aggregation | | Workflow | WDL | Sequences commands in