@cuylabs/physical-capx

Agent-agnostic CaP-X runtime adapter.

This package connects @cuylabs/physical-core sessions to the CaP-X Agent Runtime service. It owns the HTTP client, runtime mappers, artifact discovery, and CapxSession.

This package does not create an agent and does not depend on an agent runtime. It is the TypeScript session/client layer for a running capx-agent-runtime service.

TypeScript host
  -> @cuylabs/physical-capx CapxSession
    -> capx-agent-runtime HTTP server
      -> CaP-X Python runtime, selected YAML config, simulator or robot

Use capx-agent-runtime directly when a host should call CaP-X over HTTP or MCP. Use this package when TypeScript code needs a PhysicalSession for that runtime.

Install

npm install @cuylabs/physical-core @cuylabs/physical-capx

Runtime Setup

First start the CaP-X Agent Runtime on the machine that has CaP-X, simulator or robot dependencies, and the selected task config installed. The runtime service owns the CaP-X checkout, config path, output directory, skill-library path, perception services, and Python execution namespace.

Then point CapxSession at the running runtime server:

import { createCapxSession } from "@cuylabs/physical-capx";

const session = createCapxSession({
  mode: "runtime",
  runtimeServerUrl: "http://127.0.0.1:8210",
  physicalMode: "simulation",
  enablePolicyCodeExecution: true,
});

await session.start();
const observation = await session.observe({ includeImages: true });

Runtime Loop

The external host owns reasoning and code generation. CaP-X owns robotics execution.

start CapxSession
  -> observe returns task prompt, state, frame, and codeContext
  -> host or agent writes one Python policy-code step
  -> executeCode submits that code to the live runtime session
  -> host observes the result, reward, artifacts, and next frame
  -> host continues, reports completion, resets, or stops

executeCode is the operation that performs physical work. It can mutate a simulator or move hardware, so host applications should keep it approval-gated.

Ownership Boundary

The runtime server owns the robotics side:

• CaP-X clone and selected YAML config
• simulator or robot dependencies
• CUDA, perception services, and model assets
• output directory and skill-library path
• execution of submitted Python Code-as-Policies programs

This TypeScript package owns the adapter side:

• CapxSession, which maps to one live runtime session
• CapxLiveRuntime, the HTTP client wrapper
• observation, rendered frame, turn history, and artifact mapping
• agent-agnostic PhysicalSession compatibility

Boundary

This package does not create an agent and does not depend on an agent runtime. For @cuylabs/agent-core tools and createCapxAgent(...), use @cuylabs/physical-capx-agent-core.

Docs

• docs/README.md
• docs/how-it-works.md
• docs/limitations.md

License

Apache-2.0