@raindrop-ai/deep-agents

Raindrop integration for LangChain Deep Agents. Automatically captures LLM calls, tool usage (planning, filesystem, shell, subagent delegation), chains, and agent actions via LangChain's callback system.

Installation

npm install @raindrop-ai/deep-agents @langchain/core deepagents

Usage

import { createDeepAgent } from "deepagents";
import { ChatAnthropic } from "@langchain/anthropic";
import { createRaindropDeepAgents } from "@raindrop-ai/deep-agents";

const raindrop = createRaindropDeepAgents({
  writeKey: "your-write-key",
  userId: "user-123",
  convoId: "session-abc",
});

const agent = createDeepAgent({
  model: new ChatAnthropic({ model: "claude-sonnet-4-20250514" }),
  tools: [/* filesystem, shell, etc. */],
});

const result = await agent.invoke(
  { messages: [{ role: "user", content: "Create a hello world file" }] },
  { callbacks: [raindrop.handler] },
);

await raindrop.shutdown();

What gets captured

• LLM calls: model name, input, output, token usage (prompt_tokens, completion_tokens)
• Extended token categories: ai.usage.cached_tokens (OpenAI prompt cache hits, Anthropic cache_read), ai.usage.thoughts_tokens (o1/o3 reasoning tokens) — when reported by the provider
• Finish reason: captured as ai.finish_reason in event properties ("stop", "length", "tool_calls", "content_filter", etc.)
• Tool calls: tool name (write_todos, read_file, write_file, edit_file, ls, glob, grep, execute, task), input, output
• Chains: execution spans with parent-child nesting
• Agent actions: tool selection and finish events
• Multi-modal messages: typed-parts content ([{type: "text", text: "..."}, ...]) is extracted cleanly — text-type parts are concatenated; non-text parts (images, audio) are skipped
• Errors: captured with error.type + error.message properties and [Error] ... output on the event, plus OTLP error status on the span

Options

| Option | Type | Default | Description | |--------|------|---------|-------------| | writeKey | string | - | Raindrop API write key (omit to disable telemetry) | | endpoint | string | https://api.raindrop.ai/v1/ | API endpoint | | userId | string | - | Associate all events with a user | | convoId | string | - | Group events into a conversation | | debug | boolean | false | Enable verbose logging | | traceChains | boolean | true | Create spans for chain execution |

LangGraph Support

Deep Agents is built on LangGraph. The handler automatically:

• Filters LangGraph-internal chain events (__start__, __end__, ChannelWrite:*, ChannelRead:*, Branch:*) — the colon is part of LangGraph's naming convention, user-defined chains whose names happen to start with "Branch" (e.g. BranchRouter) are NOT filtered
• Keeps the root LangGraph chain so the OTel association context is in place before any inner span fires (filtering it orphans all downstream spans)
• Deduplicates LLM callbacks that LangGraph fires multiple times with the same runId
• Links parent-child spans for chain→LLM→tool hierarchies

API Surface

The createRaindropDeepAgents() factory returns:

• handler — LangChain BaseCallbackHandler to pass via config.callbacks
• lastEventId — event_id of the most recently finalized root event (or undefined if none has landed). Use this to attach a follow-up signals.track() to the agent invocation you just ran without polling the dashboard. Example:

  await agent.invoke({ messages: [...] }, { callbacks: [raindrop.handler] });
  if (raindrop.lastEventId) {
    await raindrop.signals.track({
      eventId: raindrop.lastEventId,
      name: "thumbs_up",
      type: "feedback",
      sentiment: "POSITIVE",
    });
  }

• events.patch(eventId, patch) — update event properties
• events.finish(eventId, patch) — finalize an event
• events.addAttachments(eventId, attachments) — attach files/text to an event
• events.setProperties(eventId, properties) — set custom properties
• users.identify(users) — identify users with traits
• signals.track(signal) — track user signals/feedback
• flush() — flush pending telemetry
• shutdown() — flush and close connections

Known Limitations

• Beta status: API surface may change in future releases.
• Deep Agents version: Requires deepagents >= 0.5.0.
• Streaming: Token-by-token streaming events are not captured individually; only the final aggregated response is tracked.
• Subagent isolation: When using the task tool for subagent delegation, each subagent's callbacks fire independently.
• Concurrent invocations on a shared handler: A single handler instance keeps one in-flight span map at a time. Running multiple agent.invoke(...) calls concurrently with the same handler (e.g. Promise.all([agent.invoke(...), agent.invoke(...)]) with the same raindrop.handler) can scramble the linkage between events and traces. Instantiate one createRaindropDeepAgents() per concurrent request; sequential invocations on a shared handler are fully supported.
• Long chain inputs/outputs are truncated: Chain-level input and output captured on the root event are truncated to ~8 KB to stay within the SDK's payload-size limit. Per-LLM child events still carry full prompts.

Testing

# Unit tests (no external services; use MSW to intercept HTTP)
pnpm test -- tests/handler.test.ts

E2E tests run REAL Deep Agents workflows via createDeepAgent() against a REAL LLM (OpenAI gpt-4o-mini) and verify events / tool calls / traces against the production dashboard TRPC API. They skip automatically when any of RAINDROP_WRITE_KEY, OPENAI_API_KEY, or RAINDROP_DASHBOARD_TOKEN is missing.

# E2E — requires all three keys
RAINDROP_WRITE_KEY=xxx \
OPENAI_API_KEY=sk-... \
RAINDROP_DASHBOARD_TOKEN=eyJ... \
  pnpm test -- tests/e2e.test.ts

The dashboard token comes from app.raindrop.ai → DevTools → Network → any backend.raindrop.ai request → Authorization: Bearer ... (expires every ~30 min, so grab a fresh one immediately before running).

Python

A Python version of this integration is available as raindrop-deep-agents on PyPI. See the Python package for details.