@cuylabs/physical-agent-core

Agent-core tools for physical AI sessions.

This package binds @cuylabs/physical-core sessions to @cuylabs/agent-core tools. It contains generic status, observe, artifact, stop, and policy-code tool factories.

The package does not define physical session contracts and does not connect to any robotics backend. It turns an existing PhysicalSession into a small, model-facing agent-core tool surface.

Install

npm install @cuylabs/agent-core @cuylabs/physical-core @cuylabs/physical-agent-core

Use

import { createAgent } from "@cuylabs/agent-core";
import { createPhysicalSessionTools } from "@cuylabs/physical-agent-core";

const agent = createAgent({
  model,
  tools: createPhysicalSessionTools(session),
  systemPrompt:
    "You are controlling a physical session. Observe before acting.",
});

For sessions that support direct code execution, the generated toolset includes physical_run_policy_code. For sessions that only expose observation/status or external trial control, that tool is omitted.

Backend adapters can add extra tools next to the generic set. For example, a CaP-X adapter may expose turn history while still using the same generic status, observe, artifact, stop, and policy-code contracts. Backend-specific helper libraries should normally appear as observation/context for generated policy code, not as extra model-facing robot tools.

Generated Tools

createPhysicalSessionTools(session) can produce:

• physical_status
• physical_observe
• physical_artifacts
• physical_stop
• physical_run_policy_code when session.executeCode exists

Use the prefix option when composing backend-specific surfaces:

const tools = createPhysicalSessionTools(session, { prefix: "capx" });

Boundary

Use this package only when the host runtime is @cuylabs/agent-core. The physical session contracts remain in @cuylabs/physical-core.

This package should not contain backend-specific launchers, service clients, simulator mappings, or robot APIs. Those belong in backend packages such as @cuylabs/physical-capx.

Design Notes

• A physical capability is not just a normal tool. It may depend on calibrated sensors, timing, world state, hardware readiness, and explicit safety policy.
• Physical tool outputs include structured metadata so UIs and middleware can inspect observations, traces, artifacts, and outcomes.
• Direct physical execution is marked dangerous by default and uses manual replay semantics because repeated execution can move hardware or mutate a simulator.

Docs

• docs/README.md for the documentation map
• ../physical-core/docs/safety.md for the core safety model

License

Apache-2.0