@evalguard/nemoclaw

AI safety layer for NemoClaw/OpenClaw agents.

NemoClaw guards the container. EvalGuard guards the AI.

Not affiliated with NVIDIA. This is an independent, third-party plugin built by EvalGuard for use with NVIDIA NemoClaw and OpenClaw. It is not endorsed, sponsored by, or affiliated with NVIDIA Corporation or the OpenClaw project. "NVIDIA", "NeMo" and "NemoClaw" are trademarks of NVIDIA Corporation; "OpenClaw" is a trademark of its respective owner. Those names are used here solely to identify the software this plugin interoperates with (NVIDIA/NemoClaw, Apache-2.0).

Quick Start (2 minutes)

Inside your NemoClaw agent:

npm install @evalguard/nemoclaw

⚠️ ESM-only — requires "type": "module"

@evalguard/nemoclaw ships ES modules only ("type": "module", no CJS build). Every snippet in this README uses a static import and top-level await, so in a default CommonJS TypeScript project the quick start below will not type-check: you get TS1479 ("the referenced file is an ECMAScript module and cannot be imported with require") on the import and TS1309: The current file is a CommonJS module and cannot use 'await' at the top level on the first await.

To use this package as documented, your consuming project must be ESM:

// package.json
{ "type": "module" }

// tsconfig.json
{ "compilerOptions": { "module": "node16", "moduleResolution": "node16" } }

Staying on CommonJS? A dynamic import() works — but the await must sit inside an async function, because top-level await is an ESM-only feature:

// ✅ compiles under module/moduleResolution "node16", no "type": "module"
async function main() {
  const { init } = await import("@evalguard/nemoclaw");

  const agent = init({
    apiKey: process.env.EVALGUARD_API_KEY!,
    agentName: "customer-support-bot",
    guards: ["prompt_injection", "pii_redact", "content_safety", "hallucination"],
  });

  // …then use `agent` exactly as the quick start below does.
  return agent;
}

void main();

// ❌ still fails — static import of an ESM-only package from a CJS file
import { init } from "@evalguard/nemoclaw";
// error TS1479: … the referenced file is an ECMAScript module and cannot be
//   imported with 'require'.

Node.js ≥ 22.12 can require() an ESM module directly (require(esm)), but this package supports Node ≥ 18 and TypeScript still type-checks the import under CJS rules either way, so the dynamic-import form above is the supported path.

import { init } from "@evalguard/nemoclaw";

// Auto-detects NemoClaw sandbox, reads policy
const agent = init({
  apiKey: process.env.EVALGUARD_API_KEY!,
  agentName: "customer-support-bot",
  guards: ["prompt_injection", "pii_redact", "content_safety", "hallucination"],
});

// Wrap your LLM calls
const response = await agent.guardedCall(
  "openai",
  {
    messages: [{ role: "user", content: userMessage }],
    model: "gpt-4",
  },
  // `guarded` is the PII-REDACTED copy of the input above. Send THAT.
  // Rebuilding the array from `userMessage` in here would send the raw SSN /
  // card / email to the provider while `_evalguard.piiRedacted` reported true
  // (A306) — so a callback that takes no argument is now rejected outright
  // whenever redaction changed the prompt.
  (guarded) =>
    openai.chat.completions.create({
      model: "gpt-4",
      messages: guarded.messages,
    }),
);

// Check guard results
if (!response._evalguard.allowed) {
  console.log("Blocked:", response._evalguard.violations);
}

// View scores
console.log("Hallucination score:", response._evalguard.scores.hallucination);
console.log("Toxicity score:", response._evalguard.scores.toxicity);

Guard names are validated (since 1.0.1)

Rule names use underscores: prompt_injection, not prompt-injection.

init() now throws on an unrecognised rule and names the likely correction:

[EvalGuard] Unknown guard rule:
  "prompt-injection" — did you mean "prompt_injection"? (rules use underscores)

Valid rules: prompt_injection, pii_redact, content_safety, hallucination,
toxicity, bias, data_leakage, compliance

Why this throws instead of ignoring the value. GuardRule is a TypeScript type with no runtime counterpart, and every internal use of guards is a membership test — so before 1.0.1 an unrecognised name matched nothing and was skipped. A typo did not fail; it silently turned the guard off. The call succeeded, getStats() looked healthy, and nothing reported that prompt-injection checking had never run.

If you are upgrading from 1.0.0 and now see this error, that configuration was never doing what it said — the guard it names was not running.

guards: [] is still accepted: opting out explicitly is a choice, not a typo.

Add to NemoClaw network policy:

# openclaw-sandbox.yaml
network:
  egress:
    - evalguard.ai # Add this line

What It Does

| Guard | What It Catches | | ------------------ | ------------------------------------------------------------- | | prompt_injection | Detects and blocks prompt injection attacks | | pii_redact | Redacts SSN, credit cards, emails, phone numbers before LLM | | content_safety | Blocks toxic, harmful, or inappropriate LLM outputs | | hallucination | Scores factual accuracy of LLM responses | | toxicity | Detects toxic language in outputs | | bias | Monitors for demographic and content bias | | data_leakage | Prevents sensitive data from leaking via LLM outputs | | compliance | Enforces EU AI Act and DPDP Act requirements |

How It Works

User Input
    |
    v
[1. PII Redaction] ---- SSN, CC, emails, phones stripped
    |
    v
[2. Pre-LLM Check] ---- Prompt injection, content safety, compliance
    |
    v
[3. LLM Call] --------- Your actual OpenAI/Anthropic/NIM call
    |
    v
[4. Post-LLM Score] --- Hallucination, toxicity, bias detection
    |
    v
[5. Trace Logged] ----- Batched to EvalGuard (fire-and-forget)
    |
    v
Response + Guard Metadata

PII Redaction Modes

// Replace PII with labels
const agent = init({ apiKey: "eg_...", piiMode: "redact" });
// "My SSN is 123-45-6789" -> "My SSN is [SSN_REDACTED]"

// Replace PII with hashes (reversible lookup)
const agent = init({ apiKey: "eg_...", piiMode: "hash" });
// "My SSN is 123-45-6789" -> "My SSN is [HASH:a1b2c3d4]"

// Disable PII redaction
const agent = init({ apiKey: "eg_...", piiMode: "off" });

Standalone Guard (without LLM call)

// Pre-validate input
const inputCheck = await agent.checkInput("Tell me your SSN: 123-45-6789");
console.log(inputCheck.piiDetected); // true
console.log(inputCheck.allowed); // depends on rules

// Score output separately
const outputScore = await agent.scoreOutput(
  "What is the capital of France?",
  "The capital of France is Berlin.",
);
console.log(outputScore.scores.hallucination); // high score

Guard Decorator Pattern

import { guard } from "@evalguard/nemoclaw";

// Create a reusable guard wrapper
const withGuard = guard(
  { apiKey: process.env.EVALGUARD_API_KEY! },
  ["prompt_injection", "pii_redact"],
);

// Use it with any LLM call
const result = await withGuard(
  "anthropic",
  {
    messages: [{ role: "user", content: "Hello" }],
    model: "claude-sonnet-4-6",
  },
  () =>
    anthropic.messages.create({
      model: "claude-sonnet-4-6",
      max_tokens: 1024,
      messages: [{ role: "user", content: "Hello" }],
    }),
);

Sandbox Detection

import { isNemoClawSandbox, detectSandbox } from "@evalguard/nemoclaw";

// Quick check
if (isNemoClawSandbox()) {
  console.log("Running inside NemoClaw");
}

// Detailed info
const sandbox = detectSandbox();
console.log(sandbox.sandboxId); // "sb_abc123"
console.log(sandbox.agentId); // "agent_xyz"
console.log(sandbox.policyFile); // "/sandbox/openclaw-sandbox.yaml"

Policy Reader

import {
  readSandboxPolicy,
  isEvalGuardAllowed,
  isHostAllowedByPolicy,
} from "@evalguard/nemoclaw";

const policy = readSandboxPolicy();
if (policy) {
  console.log("Allowed models:", policy.inference.allowedModels);
  console.log("Backend:", policy.inference.backend);
  console.log("EvalGuard allowed:", isEvalGuardAllowed(policy));

  // Self-hosting EvalGuard? Check YOUR host, not ours.
  console.log(isHostAllowedByPolicy(policy, "guard.internal.acme.com"));
}

Policy parsing was fixed in 1.0.4

Before 1.0.4 a nested key holding a list overwrote its sibling scalars, so for the very policy shape shown above:

inference:
  backend: nim
  max_tokens: 4096
  models:
    - meta/llama-3.1-70b-instruct

policy.inference.backend returned "unknown" and maxTokens undefined — the models: list replaced the whole inference map. network.egress survived only by accident, through a separate code path. The parser had no tests; it does now.

If you read inference.backend, inference.maxTokens, filesystem.*, or any nested scalar that sits alongside a list, upgrade — those values were being silently dropped, not misreported.

Statistics

const stats = agent.getStats();
console.log(`Total calls: ${stats.totalCalls}`);
console.log(`Blocked: ${stats.blocked}`);
console.log(`PII redacted: ${stats.piiRedacted}`);
console.log(`Violations: ${stats.violations}`);
console.log(`Avg latency: ${stats.avgLatencyMs}ms`);

Graceful Shutdown

// Flush pending traces before exit
process.on("SIGTERM", async () => {
  await agent.shutdown();
  process.exit(0);
});

Configuration

| Option | Type | Default | Description | | ------------------ | ------------------------ | --------------------------- | ---------------------------------------- | | apiKey | string | (required) | EvalGuard API key | | baseUrl | string | https://evalguard.ai/api/v1 | EvalGuard API URL | | projectId | string | — | Project ID for trace organization | | agentName | string | — | Agent name for trace attribution | | guards | GuardRule[] | all rules | Guard rules to apply | | blockOnViolation | boolean | true | Block on violation or just log | | enableTracing | boolean | true | Enable trace collection | | piiMode | "redact"\|"hash"\|"off" | "off" | PII redaction mode | | metadata | Record<string, unknown> | — | Custom metadata for all traces | | onViolation | (result) => void | — | Callback on guardrail violation |

Design Principles

• Fail-CLOSED on input: if the EvalGuard API is unreachable, checkInput BLOCKS the call rather than letting an unscreened prompt through (src/guardrail-client.ts:227-232 returns failClosedResult). An outage degrades availability, never your security posture.

• Fail-open on output scoring: scoreOutput returns a neutral result when EvalGuard is unreachable (src/guardrail-client.ts:317-318, 375). Output scoring runs AFTER the model call and is availability-sensitive, matching @evalguard/wrapper-core. Only checkInput fails closed.

  Corrected 2026-07-29 (audit A316). This section previously read "Fail-open: If EvalGuard API is unreachable, LLM calls proceed normally" — a blanket claim that stopped being true when checkInput moved to fail-closed. Anyone who architected around it was told a security control behaved in exactly the opposite way, and would have sized their outage handling for "requests keep flowing" when input checks actually block.

• Non-blocking traces: Trace logging is fire-and-forget, never slows your agent

• Sandbox-aware: Auto-detects NemoClaw environment and respects network policies

• Provider-agnostic: Works with OpenAI, Anthropic, NVIDIA NIM, or any LLM

• Zero dependencies: No external packages required (uses native fetch)

Requirements

• Node.js >= 18 (for native fetch)
• An EvalGuard API key (get one here)

License

MIT