tracetel

Turns every Composio tool call into a proper OpenTelemetry span and streams it into Agnost — or, since it's standard OTel, into any other OTel backend (Datadog, SigNoz, Grafana Tempo, etc.).

"See what your AI agent's Composio tool calls actually did — in real time, in the observability stack you already use."

Curious how this was built and why? See docs/DESIGN.md for the full research/design log, and docs/LEARNING_NOTES.md for a deep dive into the AsyncLocalStorage mechanism this package relies on.

Install

npm install tracetel

@composio/core is a peer dependency — install whichever Composio provider package your app already uses (e.g. @composio/vercel).

Usage

Direct execution (composio.tools.get)

import { Composio } from '@composio/core';
import { VercelProvider } from '@composio/vercel';
import { agnostModifiers } from 'tracetel';

const composio = new Composio({
  apiKey: process.env.COMPOSIO_API_KEY,
  provider: new VercelProvider(),
});

const modifiers = agnostModifiers({
  endpoint: 'otel.agnost.ai',
  orgId: process.env.AGNOST_ORG_ID, // find this at app.agnost.ai
});

const tools = await composio.tools.get('user_123', { toolkits: ['github'] }, modifiers);

Every tool call made through tools now shows up as a span in Agnost's dashboard.

Note: modifiers is the third argument to composio.tools.get(userId, filters, modifiers) — it is not a constructor option on VercelProvider or any other provider (confirmed against the @composio/core/@composio/vercel source; provider constructors don't accept a modifiers field). Set it up once per tools.get() call — typically once per session/request, not once per individual tool call.

Not using Agnost? Use otelModifiers instead — see Exporting to other OTel backends below.

Session-based (composio.create → session.tools)

import { Composio, SessionPreset } from '@composio/core';
import { VercelProvider } from '@composio/vercel';
import { agnostSessionModifiers } from 'tracetel';

const composio = new Composio({
  apiKey: process.env.COMPOSIO_API_KEY,
  provider: new VercelProvider(),
});

const modifiers = agnostSessionModifiers({
  endpoint: 'otel.agnost.ai',
  orgId: process.env.AGNOST_ORG_ID,
  userId: 'user_123',           // required — sessions are one-user-per-lifetime
});

const session = await composio.create(userId, {
  toolkits: ['hackernews'],
  sessionPreset: SessionPreset.DIRECT_TOOLS, // preloads actual app tools, disables meta-tools
});

const tools = await session.tools(modifiers);

Same tracing behavior, but now session.id is captured on every span — closing the "no session grouping in Agnost" gap (see Known limitations).

Not using Agnost? Use otelSessionModifiers instead — see Exporting to other OTel backends below.

Where tracetel fits in the flow

tracetel sits entirely inside Composio's own tool-execution step — it doesn't sit between your app and the LLM, and it doesn't sit between your app and Composio either. It only wraps the moment a tool call actually happens:

flowchart LR
  LLM["LLM<br/>(Groq / OpenAI / Gemini / ...)"] -->|"decides to call a tool"| App["Your app<br/>(Vercel AI SDK agent loop)"]
  App -->|"composio.tools.get(userId, filters, modifiers)"| Composio["Composio SDK<br/>(@composio/core)"]

  subgraph tracetel["tracetel (this package)"]
    direction TB
    Before["beforeExecute<br/>open OTel span<br/>redact + record args"]
    After["afterExecute<br/>record response<br/>close span"]
  end

  Composio --> Before
  Before -->|"real API call"| Real["Real toolkit API<br/>(GitHub, Slack, Notion, ...)"]
  Real -->|"response"| After
  After -->|"OTLP/HTTP export"| Backend["Agnost or any OTLP backend<br/>(SigNoz, Honeycomb, Grafana Tempo, ...)"]

Nothing between "LLM" and "composio.tools.get(...)" is tracetel's concern — see Scope below for exactly what that means in practice.

Why this exists

Composio has no standards-based (OpenTelemetry) way to export tool-call telemetry. Its only observability offering is a closed, proprietary logs/audit API inside Composio's own dashboard — you can't get that data into your own observability stack, and you can't correlate it with the rest of your app's traces.

This isn't a guess. Four independent signals confirm it, strongest first:

1. Composio's own docs/support, confirmed directly. There is no built-in OpenTelemetry exporter anywhere in the product. Their own suggested workaround for structured, exportable tool-call telemetry is either polling the tool_execution Logs API manually, or "building a custom provider that emits OpenTelemetry spans" — which is precisely what this package is. This is Composio stating the gap themselves, not a third-party inference.
2. Nango's comparison article (nango.dev/blog/composio-alternatives, Apr 2026): "Limited observability: Composio provides basic debugging information, but you cannot add custom log messages, inspect full API request/response details, or export traces with OpenTelemetry."
3. Scalekit's comparison article (scalekit.com/blog/composio-alternatives, Apr 2026), independently, in a section not about Nango at all: "Observability stops at the surface. No full API request/response visibility. No custom log messages. No OpenTelemetry export."
4. An unanswered GitHub issue on ComposioHQ/composio, from an AI agent asking for a compact, portable, structured record of what a tool action did — a different motivation (audit/compliance vs. debugging), same underlying gap: no structured, exportable record of tool-call activity exists outside Composio's own UI.

Across the wider category (Composio, Arcade, Workato, Merge, Paragon, Pipedream, Scalekit), only Nango claims real OpenTelemetry export — and only by requiring a full migration off your existing integration layer onto theirs. There is currently no way to get OTel-based tool-call observability while staying on Composio. tracetel is a drop-in that fills exactly that gap, without migrating anything.

What gets captured

Each tool call becomes a span named tool.<TOOL_SLUG> with:

• tool.name, tool.toolkit
• user.id — from params.userId (direct-execution path) or from the config's userId field (session-based path)
• session.id — session-based path only; captured from the beforeExecuteMetaModifier context
• tool.arguments — the arguments passed to the tool, JSON-stringified and redacted
• tool.response — the tool's response data, same redaction treatment
• tool.duration_ms — wall-clock time between beforeExecute and afterExecute
• composio.log_id — Composio's own log ID for that call, if present, for cross-referencing with Composio's own dashboard
• Status: OK on success, ERROR (with error.message) on failure

agnostModifiers only also adds agnost.user_id, input, and output — Agnost's own documented dashboard conventions (docs.agnost.ai/otel) for its Tools/Conversations views. otelModifiers omits these since they're Agnost-specific, not general OTel conventions other backends would expect.

Redaction: any argument/response key that looks like a token, secret, password, API key, credential, bearer, or cookie is replaced with [REDACTED] before it's ever serialized into a span. Bodies are truncated past 4096 characters by default (maxBodyChars is configurable).

Correlation between calls uses OpenTelemetry's own trace/span IDs by default. The session-based path (agnostSessionModifiers/otelSessionModifiers) additionally captures session.id from the beforeExecuteMetaModifier context — closing the "no session grouping" gap for that path. The direct-execution path still relies on OTel trace/span IDs alone, since the plain beforeExecute/afterExecute modifiers don't receive a sessionId (confirmed against source).

Scope: tool-call observability, not full conversation/LLM observability

tracetel only instruments Composio's beforeExecute/afterExecute hooks around tool execution. It never touches the LLM call itself — the actual prompt sent to your model (Groq, OpenAI, Gemini, whatever you use) is not captured anywhere in this pipeline.

This matters in practice: if you view spans in Agnost's Conversations view, what you'll see is tool.arguments/tool.response (input/output) formatted to resemble a chat transcript. It is not the real LLM prompt/completion — it's tool-call data rendered by Agnost's UI to look conversational. Don't mistake one for the other. If you need full conversation/LLM-call observability (actual prompts, completions, token usage), you need separate LLM instrumentation alongside this package — tracetel intentionally covers tool calls only.

Known limitations

• Unclosed spans on pre-afterExecute failures. If a tool call throws before Composio reaches afterExecute (e.g. a network error mid-call), the span opened in beforeExecute is never closed or exported. Composio's modifier API has no onError hook to catch this without monkey-patching, which this package deliberately avoids in favor of official SDK hooks only.
• No Composio-native session ID on the direct-execution path. The plain beforeExecute/afterExecute modifiers don't receive a sessionId (confirmed against source). The session-based path (agnostSessionModifiers/otelSessionModifiers) does capture session.id — this limitation only applies to composio.tools.get(...) usage.
• Session/conversation grouping in Agnost's dashboard is a heuristic for the direct-execution path. On the session-based path, session.id is emitted and provides proper grouping. On the direct-execution path, since we don't emit a session.id/agnost.session_id, Agnost's Conversations view falls back to agnost.user_id + time proximity. That fallback is observed dashboard behavior, not a documented guarantee, and could change without notice.
• tracing/allowTracing on Composio's provider classes is unrelated to this package. Confirmed by reading the shipped @composio/core source: that flag only controls whether Composio logs a call in its own internal audit system (allow_tracing sent to Composio's backend) — it has no OpenTelemetry export capability, and zero references to @opentelemetry, OTLP, or TracerProvider exist anywhere in that package.

Exporting to other OTel backends

agnostModifiers / agnostSessionModifiers are Agnost-native — they require orgId and always emit Agnost's dashboard-specific attributes. For any other OTLP-compatible backend, use otelModifiers / otelSessionModifiers instead: same tracing behavior, but you provide the auth headers your backend expects directly, and it emits only the generic, backend-agnostic attributes.

Direct execution

import { otelModifiers } from 'tracetel';

// Self-hosted collector (SigNoz, Jaeger, an OTel Collector, etc.)
const modifiers = otelModifiers({ endpoint: 'http://localhost:4318', headers: {} });

// Honeycomb
const modifiers = otelModifiers({
  endpoint: 'api.honeycomb.io',
  headers: { 'x-honeycomb-team': process.env.HONEYCOMB_API_KEY },
});

// Grafana Cloud Tempo (Basic auth)
const modifiers = otelModifiers({
  endpoint: 'tempo-prod-your-instance.grafana.net',
  headers: { Authorization: `Basic ${Buffer.from(`${instanceId}:${apiToken}`).toString('base64')}` },
});

const tools = await composio.tools.get('user_123', { toolkits: ['github'] }, modifiers);

Session-based

import { otelSessionModifiers } from 'tracetel';

const modifiers = otelSessionModifiers({
  endpoint: 'http://localhost:4318',
  headers: {},
  userId: 'user_123',
});

// ...composio.create(userId, config) → session.tools(modifiers)

Flushing spans: when to call shutdownTracing()

OpenTelemetry's batch span processor doesn't export spans the instant span.end() runs — it batches them and flushes periodically (every ~5 seconds by default) or once a batch fills up. shutdownTracing() forces an immediate flush and tears down the exporter. When you need to call it depends entirely on how long your process lives.

Short-lived scripts (CLI tools, one-off scripts, the examples/ script in this repo) — call it once, right before the process exits, to force-flush anything still batched:

async function main() {
  const tools = await composio.tools.get(userId, { toolkits: ['github'] }, modifiers);
  // ...make your tool calls...

  await shutdownTracing();
}

main();

Without this, the process can exit before the batch timer ever fires, silently dropping whatever spans hadn't been exported yet.

Long-running servers (Express, Next.js API routes, anything that stays alive) — don't call it per-request. Hook it into the process's actual shutdown signal instead:

process.on('SIGTERM', shutdownTracing);
process.on('SIGINT', shutdownTracing);

While the process is alive, the batch processor's own timer keeps flushing spans on its own — you don't need to do anything per-request. Calling shutdownTracing() after every request would be both wasteful (forces a flush and tears down the exporter every time, which then just gets recreated on the next call) and pointless (the timer would have flushed it anyway). Only call it once, when the whole process is actually going down, so any final in-flight spans get out before the process dies.

Configuration reference

agnostModifiers({
  endpoint: string;       // e.g. "otel.agnost.ai", or a full URL like "http://localhost:4318" for local/self-hosted OTel collectors
  orgId: string;          // Agnost org ID, sent as the X-Agnost-Org-ID header. Find it at app.agnost.ai.
  serviceName?: string;   // service.name resource attribute; defaults to "tracetel"
  maxBodyChars?: number;  // truncation limit for arguments/response bodies; defaults to 4096
})

agnostSessionModifiers({
  endpoint: string;       // same as agnostModifiers
  orgId: string;          // same as agnostModifiers
  userId: string;         // required — sessions are one-user-per-lifetime, bound at modifier-creation time
  serviceName?: string;   // same as agnostModifiers
  maxBodyChars?: number;  // same as agnostModifiers
})

otelModifiers({
  endpoint: string;                 // OTLP/HTTP host or full URL
  headers: Record<string, string>;  // auth headers your backend expects — required, no default
  serviceName?: string;   // service.name resource attribute; defaults to "tracetel"
  maxBodyChars?: number;  // truncation limit for arguments/response bodies; defaults to 4096
})

otelSessionModifiers({
  endpoint: string;                 // same as otelModifiers
  headers: Record<string, string>;  // same as otelModifiers
  userId: string;                   // required — sessions are one-user-per-lifetime
  serviceName?: string;   // same as otelModifiers
  maxBodyChars?: number;  // same as otelModifiers
})