@cuylabs/agent-runtime

Workload orchestration layer for agent and background workloads.

@cuylabs/agent-core focuses on single-turn agent execution.
@cuylabs/agent-runtime manages scheduling and execution of recurring/background work.

The package now makes one architectural seam explicit:

• agent-core defines turn/task execution semantics
• agent-runtime consumes a generic workload contract
• infrastructure packages such as agent-runtime-dapr implement that contract

Package Boundary

Use @cuylabs/agent-runtime when you need outer orchestration:

• job definitions and schedules
• dispatch, retries, and concurrency limits
• pluggable runtime drivers
• runtime observers and metrics
• orchestration APIs for invoking and guiding long-running work

This package does not own in-process model or tool interception.

• agent-core owns middleware, tool execution, model execution, and turn semantics
• agent-runtime owns workload scheduling and dispatch around that execution

So most users start with agent-core, then add agent-runtime when they need workers, jobs, or orchestrated execution.

What This Package Owns

• Job definitions and lifecycle (schedule, update, pause, resume, runNow, dispatch)
• Schedule evaluation (at, every, cron)
• Execution loop with concurrency limits
• Pluggable runtime drivers (in-memory now, Dapr/others later)
• Neutral workload adapter for plugging application work into the runtime
• Optional app-level orchestration API (invoke, listInvocations, waitForInvocation, close, guide)

Layered Architecture

agent-core and agent-runtime should stay separate:

host app / worker process
  -> agent-runtime (job registry + dispatch loop)
    -> agent-core (actual agent logic)

• agent-core: tool loop + model calls + business logic
• agent-runtime: scheduling/execution orchestration + workload contract
• driver package (for example agent-runtime-dapr): infrastructure adapter

The runtime does not require Dapr. Dapr is one driver/store implementation.

See docs/architecture.md for a concrete package/folder layout across agent-core, agent-runtime, agent-runtime-dapr, agent-code, and a deployable host app. See docs/capability-matrix.md for the difference between shared runtime contract and backend-specific guarantees. See docs/production-hardening.md for the current path from local/in-memory runtime to durable/distributed deployments.

How It Connects To agent-core

agent-runtime does not depend on the Agent class directly. It runs generic workloads.

When you want to schedule agent work, the usual handoff is:

agent-core
  -> createAgentTaskRunner(agent)
    -> agent-runtime workload adapter
      -> WorkloadRuntime

That means:

• agent-core turns a live Agent into a task-shaped function
• agent-runtime schedules and dispatches that task like any other workload
• backend packages such as agent-runtime-dapr can then add durability, persistence, and host integration on top

This separation is intentional:

• agent-core owns agent semantics
• agent-runtime owns outer orchestration semantics
• the bridge between them is the workload contract

Package Structure

src/
  index.ts                 # Public exports
  types.ts                 # Runtime/job contracts
  workload/
    index.ts               # Generic workload contract + adapter helper
  observer.ts              # Runtime lifecycle/queue/run observer contracts
  logger.ts                # Logger interface + defaults
  schedule.ts              # Schedule normalization + next-run calculation
  driver.ts                # Driver interface
  runtime.ts               # WorkloadRuntime orchestration class
  drivers/
    in-memory.ts           # Default local runtime driver
  orchestration/
    service.ts             # invoke/listInvocations/waitForInvocation/close/guide API
    store.ts               # run-record persistence contract
    stores/
      in-memory.ts         # default orchestration run store
tests/
  unit/
    runtime.test.ts
    orchestrator.test.ts

Installation

npm install @cuylabs/agent-runtime
# or
pnpm add @cuylabs/agent-runtime

Focused imports are also available when you want the package surface to match the folder structure:

import { createAgentOrchestrator } from "@cuylabs/agent-runtime/orchestration";
import { InMemoryRuntimeDriver } from "@cuylabs/agent-runtime/drivers/in-memory";
import { createRuntimeWorkloadExecutor } from "@cuylabs/agent-runtime/workload";

Quick Start

import {
  createWorkloadRuntime,
  createPrometheusRuntimeMetrics,
  InMemoryRuntimeDriver,
  type RuntimeJobRecord,
} from "@cuylabs/agent-runtime";

type Payload = { message: string };

const metrics = createPrometheusRuntimeMetrics<Payload>({
  defaultLabels: { service: "digest-worker" },
});

const runtime = createWorkloadRuntime<Payload>({
  driver: new InMemoryRuntimeDriver(),
  execute: async (job: RuntimeJobRecord<Payload>) => {
    console.log(`[job:${job.id}]`, job.payload.message);
  },
  maxConcurrentRuns: 2,
  maxQueuedDispatches: 32,
  onDeadLetter: async (event) => {
    console.error("dead-lettered", event.job.id, event.error);
  },
  observers: [metrics.observer],
});

await runtime.start();

const job = await runtime.schedule({
  name: "hourly-digest",
  payload: { message: "run digest" },
  schedule: { kind: "cron", expr: "0 * * * *", timezone: "UTC" },
  retryPolicy: {
    maxAttempts: 3,
    backoffMs: 1_000,
    strategy: "exponential",
    maxBackoffMs: 30_000,
  },
});

console.log("scheduled", job.id);

Workload Contract

Use the workload adapter when you want your app logic to stay independent from the runtime internals:

import {
  createWorkloadRuntime,
  createRuntimeWorkloadExecutor,
  InMemoryRuntimeDriver,
} from "@cuylabs/agent-runtime";

const runDigest = async (
  payload: { message: string },
  context: { jobId: string; signal: AbortSignal },
) => {
  console.log(context.jobId, payload.message);
  return { delivered: true };
};

const runtime = createWorkloadRuntime({
  driver: new InMemoryRuntimeDriver(),
  execute: createRuntimeWorkloadExecutor({
    run: runDigest,
  }),
});

For agent workloads, @cuylabs/agent-core already provides createAgentTaskRunner(...), which fits naturally into this workload boundary.

Dispatch API

dispatch lets host apps route explicit triggers into the runtime:

await runtime.dispatch({ jobId: "job-123", trigger: "due" });
await runtime.dispatch({ jobId: "job-123", trigger: "manual" });

Use this from transport callbacks (HTTP/gRPC/event handlers) after you map external payloads to a runtime jobId.

Production Guardrails

createWorkloadRuntime(...) includes baseline guardrails:

• maxConcurrentRuns (default 4)
• maxQueuedDispatches (optional cap for pending dispatches)
• executionTimeoutMs (default 15m; set <= 0 to disable)
• abortInFlightOnStop (default true)
• per-job retryPolicy with fixed or exponential backoff
• optional onDeadLetter(...) hook when retries are exhausted
• optional observers hook for runtime lifecycle, queue depth, retries, and completion events
• job input validation:
  • non-empty bounded id / name
  • bounded metadata keys/values

Runtime Observers

WorkloadRuntime exposes a small observer surface for production hooks, and the package now ships a concrete Prometheus-style collector for the common case.

import {
  createPrometheusRuntimeMetrics,
  PROMETHEUS_TEXT_CONTENT_TYPE,
} from "@cuylabs/agent-runtime";

const runtimeMetrics = createPrometheusRuntimeMetrics({
  defaultLabels: { service: "maintenance" },
});

const body = runtimeMetrics.render();
const contentType = PROMETHEUS_TEXT_CONTENT_TYPE;

The collector tracks runtime starts/stops, queued/dropped dispatches, retries, dead letters, queue depth, in-flight runs, and run duration histograms.

Keep observers side-effect-safe: they should record telemetry, not mutate runtime state or retry work themselves.

Drivers

In-memory (included)

• Good for local development and tests
• No persistence across process restarts
• Matches the runtime API and lifecycle semantics, not Dapr's infrastructure guarantees

Future drivers

The runtime is intentionally driver-based so durable backends (for example Dapr-backed drivers) can be added without changing the orchestration API.

See @cuylabs/agent-runtime-dapr for a Dapr sidecar-backed driver package.

Important boundary

Do not describe in-memory as “the same as Dapr”.

Describe it as:

• the same runtime contract
• different operational guarantees

Orchestration API (Dapr Optional)

agent-runtime now includes an orchestration layer for parent/child run control:

• invoke: create a run record and dispatch execution
• listInvocations: inspect active/recent runs
• waitForInvocation: await run completion with optional timeout or AbortSignal
• close: cancel a running or queued run
• guide: restart a running run with refined instructions

This orchestration API is runtime-backed but infrastructure-agnostic:

• Start with in-memory runtime driver + in-memory orchestration store.
• Add Dapr later by swapping runtime driver and/or orchestration store via @cuylabs/agent-runtime-dapr.

Minimal Example

import {
  createAgentOrchestrator,
  InMemoryRuntimeDriver,
} from "@cuylabs/agent-runtime";

const orchestrator = createAgentOrchestrator<
  { message: string },
  { response: string }
>({
  driver: new InMemoryRuntimeDriver(),
  execute: async (run, context) => {
    // Bridge to agent-core task runner here.
    // context.signal supports close/timeout cancellation.
    return { response: `handled: ${run.input.message}` };
  },
});

await orchestrator.start();

const { run } = await orchestrator.invoke({
  label: "analysis",
  input: { message: "Review this PR" },
});

const completed = await orchestrator.waitForInvocation(run.id, {
  timeoutMs: 30_000,
});
console.log(completed.state.status, completed.state.result);

License

Apache-2.0