@cuylabs/physical-capx-agent-core

Agent-core CaP-X agent adapter.

This package composes @cuylabs/physical-capx, @cuylabs/physical-agent-core, and @cuylabs/agent-core. It provides CaP-X tool creation, packaged CaP-X model guidance, and createCapxAgent(...).

Use this package when the host harness is @cuylabs/agent-core. Use @cuylabs/physical-capx directly when a TypeScript host only needs a backend-neutral CapxSession.

agent-core model loop
  -> @cuylabs/physical-capx-agent-core tools
    -> @cuylabs/physical-capx CapxSession
      -> capx-agent-runtime HTTP server
        -> CaP-X Python runtime, selected YAML config, simulator or robot

Install

npm install @cuylabs/agent-core @cuylabs/physical-core @cuylabs/physical-agent-core @cuylabs/physical-capx @cuylabs/physical-capx-agent-core

Use

import { createCapxAgent } from "@cuylabs/physical-capx-agent-core";

const { agent, session } = await createCapxAgent({
  model,
  startSession: true,
  toolExecutionMode: "plan",
  sessionOptions: {
    mode: "runtime",
    runtimeServerUrl: "http://127.0.0.1:8210",
    physicalMode: "simulation",
    enablePolicyCodeExecution: true,
  },
});

The package also exposes a migration-friendly ./session entrypoint:

import {
  createCapxPhysicalTools,
  createCapxSession,
} from "@cuylabs/physical-capx-agent-core/session";

Runtime Loop

The external agent owns reasoning and code generation. CaP-X owns robotics execution.

start CapxSession
  -> capx_observe returns task prompt, state, frame, and codeContext
  -> agent writes one Python policy-code step
  -> capx_run_policy_code submits that code to the live runtime session
  -> agent observes the result, reward, artifacts, and next frame
  -> host or agent continues, reports completion, resets, or stops

capx_run_policy_code is the tool that performs physical work. It can mutate a simulator or move hardware, so it should stay approval-gated and dispatched by the host harness.

Tools

This package generates the agent-core tools below. They are the TypeScript-facing tool surface for a CapxSession; internally they call the capx-agent-runtime HTTP API through @cuylabs/physical-capx.

• capx_status
• capx_observe
• capx_artifacts
• capx_stop
• capx_run_policy_code when enablePolicyCodeExecution: true
• capx_turn_history

The raw runtime server exposes HTTP endpoints such as /sessions, /sessions/{id}/observe, /sessions/{id}/execute-code, /sessions/{id}/artifacts, and /sessions/{id}/turns. Its optional MCP wrapper has its own MCP-native tool names. The list above is the smaller agent-core default surface.

The default model-facing surface is intentionally small. It gives the agent enough context to write better Code-as-Policies Python without exposing every runtime operation as a separate robot-action tool.

CaP-X Skills

There are two different skill concepts:

• agent-core skills are SKILL.md instruction packs that teach the model how to use the CaP-X tools.
• CaP-X skills are Python helper functions in the runtime skill library.

The default agent tool surface exposes CaP-X skill-library information through capx_observe as policy-code context. Programmatic clients can still use CapxLiveRuntime from @cuylabs/physical-capx to inspect, extract, or inject runtime skills deliberately with host approval.

Boundary

Use this package only when the host runtime is @cuylabs/agent-core. The agent-agnostic CaP-X session adapter remains in @cuylabs/physical-capx.

The runtime server owns the robotics side: CaP-X checkout, selected YAML config, simulator or robot dependencies, CUDA/perception services, model assets, output directory, skill-library path, and execution of submitted Python programs.

This TypeScript package owns only the agent-core side: tool creation, approval-aware policy-code execution, packaged model guidance, and createCapxAgent(...).

Docs

• docs/README.md
• docs/agent-core-integration.md
• ../physical-capx/docs/how-it-works.md for the runtime/session adapter
• examples/README.md

License

Apache-2.0