@canary-ops/sdk

The Operating System for AI Security. Monitor, secure, and audit every AI agent and LLM application in production — in under five minutes, with one zero-dependency package.

npm install @canary-ops/sdk

import canary from '@canary-ops/sdk';

canary.init({ apiKey: 'sk-canary-...', agentId: 'agt_...' });

// All telemetry now flowing to Canary ✓

Why

You can't secure what you can't see. Canary captures every prompt, response, tool call, token, cost, and latency from your agents, blocks prompt injections and jailbreaks before they reach your model, scans LLM outputs for leaked secrets and PII before they reach your users, and redacts sensitive data before any content leaves your process — so the thing watching your agents never becomes the thing that leaks your data.

• 🔌 Plugs in — wrap your OpenAI, Anthropic, Gemini, or Bedrock client, use the LangChain callback handler, or integrate via the Vercel AI SDK telemetry hook. Streaming included.
• 🛡️ Input firewall — injection/jailbreak heuristics run in-process at zero latency, with optional remote deep-scan.
• 🔍 Output scanning — scanResponse() checks LLM outputs for secrets, PII, harmful content, and canary token exfiltration.
• 🪤 Canary tokens — plant synthetic decoy credentials; get a critical alert the instant they appear in a response.
• 🔒 Redaction built in — API keys, JWTs, private keys stripped by default; PII opt-in.
• 🪶 Zero dependencies — isomorphic across Node 18+, browsers, Deno, Bun, edge.
• 🧱 Production-safe — timeouts, retries with backoff, a circuit breaker, and fail-open by default so it never takes your app down.

Quickstart

import canary from '@canary-ops/sdk';

canary.init({
  apiKey: process.env.CANARY_API_KEY, // sk-canary-...
  agentId: process.env.CANARY_AGENT_ID,
});

// 1. Block malicious prompts (local heuristics + remote scan)
await canary.guard(userPrompt); // throws CanaryBlockedError if blocked

// 2. Track model calls (cost auto-computed from model + tokens)
canary.trackPrompt({
  content: userPrompt,
  model: 'gpt-4o',
  tokens_in: 42,
  tokens_out: 128,
  latency_ms: 340,
});

// 3. Scan the response before returning it to the user
await canary.scanResponse(responseText, { throwOnUnsafe: true });

Events are batched and flushed in the background (every 5s or 20 events), so tracking never blocks your request path.

Auto-instrumentation

Wrap your existing client and Canary captures every call — streaming and non-streaming — automatically.

import OpenAI from 'openai';
import canary from '@canary-ops/sdk';

canary.init({
  apiKey: 'sk-canary-...', agentId: 'agt_...',
  guard: { responses: true }, // auto-scan every response for secrets/PII/harmful content
});
const openai = canary.wrapOpenAI(new OpenAI());

// tokens, cost, latency, prompt + response all captured — even when streaming
// responses are scanned automatically; CanaryBlockedError thrown if unsafe
const stream = await openai.chat.completions.create({
  model: 'gpt-4o', stream: true, messages,
});
for await (const chunk of stream) { /* your normal loop, untouched */ }

import Anthropic from '@anthropic-ai/sdk';
const anthropic = canary.wrapAnthropic(new Anthropic());
await anthropic.messages.create({ model: 'claude-opus-4-8', messages, max_tokens: 1024 });

Google Gemini:

import { GoogleGenerativeAI } from '@google/generative-ai';
const genAI = new GoogleGenerativeAI(process.env.GEMINI_API_KEY);
const model = canary.wrapGemini(genAI.getGenerativeModel({ model: 'gemini-1.5-pro' }));
// generateContent + generateContentStream both captured
const result = await model.generateContent('Hello');

AWS Bedrock:

import { BedrockRuntimeClient, InvokeModelCommand } from '@aws-sdk/client-bedrock-runtime';
const client = canary.wrapBedrock(new BedrockRuntimeClient({ region: 'us-east-1' }));
// InvokeModelCommand + InvokeModelWithResponseStreamCommand captured
// supports Claude, Titan, Llama 3, and Mistral model families

LangChain (any version):

import { ChatOpenAI } from '@langchain/openai';
const llm = new ChatOpenAI({ callbacks: [canary.callbackHandler()] });
// LLM start/end/error + chain + tool events all captured

Vercel AI SDK:

import { generateText } from 'ai';
const result = await generateText({
  model: openai('gpt-4o'),
  prompt: 'Hello',
  experimental_telemetry: canary.vercelTelemetry(),
  // compatible with AI SDK v3 (ai.*) and v4+ (gen_ai.*) attribute names
});

Instrument your tool/HTTP calls too:

const fetch = canary.wrapFetch(globalThis.fetch, { match: 'api.weather.com' });
// latency, status, and errors for matching requests are now recorded

Tracing

Wrap any async step to record its latency (and errors) as a span:

const docs = await canary.trace('retrieve_docs', () => vectorStore.search(query));

Firewall

const verdict = await canary.scan(userPrompt);
// { allowed, action, threats: [...], degraded? }
if (!verdict.allowed) return refuse(verdict.threats);

• Local heuristics (in-process, no network) catch instruction-override, DAN/jailbreak, system-prompt extraction, exfiltration, and embedded secrets.
• Remote deep-scan adds semantic classification and your custom firewall rules.
• A high-confidence local block short-circuits the network call.
• If the remote scanner is down, behaviour follows failOpen (default true — the local verdict stands and degraded: true is set).

Covered threat categories (v0.3.0): prompt injection (direct + indirect tool injection), jailbreaks (DAN, persona, hypothetical framing), system-prompt extraction (verbatim dump, context echo), data exfiltration (URL send, markdown link, webhook instructions), adversarial encoding (base64/rot13/hex decode-and-run, leet-speak), and embedded secrets.

Output scanning

Scan an LLM response before returning it to the user:

const result = await canary.scanResponse(responseText);
if (!result.allowed) {
  return refuse(result.threats); // PII, secret, harmful content, or canary token
}

// Or throw on the spot
await canary.scanResponse(responseText, { throwOnUnsafe: true });

Checks for: leaked secrets and API keys, PII (SSNs, credit cards, emails, phones), canary token exfiltration, harmful content (LLM-as-judge), and off-topic responses (when agent.purpose is configured on the dashboard).

Enable automatic output scanning for every completion:

canary.init({
  apiKey, agentId,
  guard: { responses: true }, // scan every wrapOpenAI / wrapAnthropic response
});

Canary tokens

Plant synthetic decoy credentials in your agent's context or system prompt. If a response ever includes one, Canary fires a critical data_exfiltration alert — catching exfiltration attacks the instant they succeed.

// Plant a fake API key in the system prompt
const { token_value } = await canary.plantToken({ tokenType: 'api_key' });
const systemPrompt = `You are a helpful assistant.\n\nInternal ref: ${token_value}`;

// scanResponse() (or guard.responses: true) catches it automatically if leaked

Token types: api_key, email, ssn, credit_card. All tokens are prefixed (sk-canary-trap-, 0000- Luhn-valid cards, etc.) to distinguish them from real credentials with zero false positives.

RAG / context scanning

Scan retrieved documents for embedded injection attacks before injecting them into a prompt (OWASP LLM04 — indirect prompt injection via context):

const docs = await vectorStore.search(query);

// Scan all chunks — up to 5 in parallel by default
const results = await canary.scanContext(docs.map(d => d.text));

// Drop poisoned documents before sending to the model
const safeDocs = docs.filter((_, i) => results[i].allowed);

Threats found in context documents are re-typed as context_poisoning and tagged LLM04 in the dashboard, so you can track RAG-specific attacks separately from direct user input.

Redaction & privacy

Sensitive data is stripped before it ever leaves the process:

canary.init({
  apiKey, agentId,
  redaction: { secrets: true, pii: true, mode: 'mask' }, // secrets default on
  captureContent: true,   // set false for metadata-only mode (no prompt text)
  maxContentLength: 10000, // truncate long content
  sampleRate: 1.0,         // 0–1; errors always kept
  beforeSend: (e) => e,    // inspect / mutate / drop (return null) each event
});

Detected out of the box: OpenAI/AWS/GitHub/Slack/Google keys, bearer tokens, JWTs, private keys (secrets); emails, phones, SSNs, credit cards (Luhn-checked), IPs (PII). Add your own with redaction.customPatterns.

API

canary.init(config)

| Option | Type | Default | Description | | ----------------- | ----------------------- | ---------------- | -------------------------------------------------------- | | apiKey | string | — | Required. sk-canary-... | | agentId | string | — | Required. Agent these events belong to. | | baseUrl | string | Canary functions | Override the API host. | | flushIntervalMs | number | 5000 | Background flush cadence. | | maxBatchSize | number | 20 | Flush early once this many events queue. | | sessionId | string | server-assigned | Group related events into a session. | | sampleRate | number | 1 | Keep this fraction of events (errors exempt). | | captureContent | boolean | true | false = metadata-only, no prompt text sent. | | maxContentLength| number | 10000 | Truncate captured content. | | redaction | RedactionConfig\|bool | secrets on | Strip secrets/PII before sending. | | beforeSend | function | — | Inspect/mutate/drop each event. | | guard | GuardConfig | local+remote | Firewall behaviour (see below). | | failOpen | boolean | true | Allow on remote-scan failure (vs block). | | timeoutMs | number | 10000 | Per-request timeout. | | retry | RetryConfig | 3 / 250ms | Retry attempts + backoff for transient errors. | | environment | string | — | Tag events (e.g. production). | | release | string | — | Tag events with your app version. | | flushOnExit | boolean | true | Flush on process exit / page unload. | | debug | boolean | false | Log SDK internals. | | onError | function | warn-on-debug | Handle background flush/scan failures. |

guard options:

| Option | Type | Default | Description | | --------------- | --------- | ------- | ------------------------------------------------------------------------------- | | local | boolean | true | Run in-process heuristics on each scan() call. | | remote | boolean | true | Run the remote deep-scan. | | blockOnLocal | boolean | true | Short-circuit to block on a high-confidence local hit (skips network round-trip). | | responses | boolean | false | Auto-scan every LLM response via wrapOpenAI / wrapAnthropic. |

Methods

Telemetry

• track(event) / trackPrompt / trackResponse / trackToolCall / trackError(err, extra?)
• flush() — send queued events now
• shutdown() — flush and stop

Firewall (input)

• scan(prompt, opts?) → ScanResult — check a prompt; never throws on a verdict
• guard(prompt, opts?) → ScanResult — like scan() but throws CanaryBlockedError if blocked

Output scanning

• scanResponse(text, opts?) → ScanResult — scan an LLM response for PII, secrets, harmful content, and canary tokens; set throwOnUnsafe: true to throw CanaryBlockedError instead of returning

Canary tokens

• plantToken(opts?) → CanaryToken — plant a synthetic decoy credential; returns { token_id, token_value, token_type }

RAG / context

• scanContext(docs[], opts?) → ScanResult[] — scan an array of retrieved documents; returns one result per document in order

Instrumentation

• trace(name, fn, opts?) → result of fn, recorded as a span
• wrapOpenAI(client) / wrapAnthropic(client) / wrapFetch(fetch, opts?)

Also exported

Canary (class for multiple instances), computeCost, hasPricing, localScan, newSessionId, CanaryBlockedError, CanaryNotInitializedError, and all TypeScript types (ScanResult, ScanOptions, ScanResponseOptions, PlantTokenOptions, CanaryToken, ScanContextOptions, TelemetryEvent, …).

Error reference

CanaryNotInitializedError

When: Any SDK method is called before canary.init(...).

CanaryNotInitializedError: Canary SDK is not initialized. Call canary.init(...) first.

Fix: Call canary.init({ apiKey, agentId }) once at app startup, before any track*, scan, guard, or wrap* calls.

CanaryBlockedError

When: canary.guard(prompt) is called and the firewall verdict blocks the prompt, or canary.scanResponse(text, { throwOnUnsafe: true }) detects a threat.

try {
  await canary.guard(userPrompt);
} catch (err) {
  if (err instanceof CanaryBlockedError) {
    console.log(err.action);  // 'block' | 'quarantine' | …
    console.log(err.threats); // DetectedThreat[]
  }
}

Properties: action (the highest-severity verdict), threats (array of DetectedThreat).

Note: canary.scan() never throws — use it when you want to inspect the verdict without blocking execution.

CanaryApiError

When: The Canary API returns a non-2xx response that exhausts all retries.

CanaryApiError: POST /ingestEvents failed with status 401

Common causes:

| Status | Cause | Fix | |---|---|---| | 401 | Invalid or missing API key | Check apiKey in init() | | 403 | Key lacks the required scope | Create a key with telemetry and scan scopes | | 404 | agentId not found | Verify the agent ID on the Agents page | | 500 | Server error | Check [status.canary-ops.com] or contact support |

By default the SDK fails open (failOpen: true) on API errors — it logs via onError and continues. Set failOpen: false to block on remote-scan failures.

Troubleshooting

No events appearing in the dashboard

1. Confirm canary.init() was called before any track* call.
2. Enable debug: true in init() — the SDK logs every flush to the console.
3. Check that the API key has telemetry scope and the agent ID matches an existing agent.
4. Events are batched and flushed every 5 seconds (or at 20 events). Call await canary.flush() to send immediately.

wrapOpenAI / wrapAnthropic not recording anything

• Make sure you call canary.init(...) before canary.wrapOpenAI(client).
• The wrapper patches the client in-place — if you create the client before wrapping, the patched version is the same object, so no re-assignment is needed.
• Streaming responses record on the final chunk. If you break out of the for await loop early (e.g. an error), the onFinalize still fires.

guard() blocking legitimate prompts (false positives)

• Use canary.scan(prompt) first to inspect the threats array and see which rule matched.
• Tune the local guard: set guard: { blockOnLocal: false } to let local hits through to the remote deep-scan for a second opinion.
• The score field on each DetectedThreat indicates confidence (0–100). Local patterns with score < 80 warn but don't hard-block unless blockOnLocal: true.

Content appearing redacted when you don't want it to

• Secrets redaction (redaction.secrets) is on by default. If your prompts contain intentional API key patterns for testing, set redaction: { secrets: false }.
• Check the mode: 'mask' replaces with [REDACTED], 'hash' with an FNV-1a hex digest, 'remove' deletes the field entirely.
• PII redaction is off by default. It only activates if you set redaction: { pii: true }.

Cost shows as undefined

• computeCost returns undefined when the model string isn't in the built-in pricing table. Pass cost_usd explicitly to your track* calls, or ensure model matches a known model name (e.g. 'gpt-4o', 'claude-opus-4-8').
• Call hasPricing('your-model') to check coverage.

Process exits before all events are flushed

• The SDK attaches a beforeExit handler automatically (flushOnExit: true). For short-lived scripts that exit via process.exit() (which skips beforeExit), call await canary.shutdown() before exiting.

Environments

Works anywhere with a global fetch + AbortController: Node 18+, modern browsers, Deno, Bun, and edge runtimes. Ships ESM + CommonJS + TypeScript types. Zero runtime dependencies.

Authors

Built by CryptoSuess & Txnchi.

License

MIT