@know-your-ai/node

A lightweight SDK to monitor and capture AI model interactions in Node.js applications. Inspired by Sentry's architecture, this SDK provides a clean and extensible way to track your AI usage.

Features

• 🔍 Automatic Instrumentation - Capture AI inputs, outputs, and token usage automatically
• 🎯 Google GenAI Support - Full support for Google's Generative AI SDK (Gemini)
• 🔌 Extensible Architecture - Easy to add support for other AI providers (OpenAI coming soon)
• 📊 Flexible Transport - Send data to your own backend or use built-in console logging
• ⚙️ Configurable - Control what data is captured with fine-grained options

Installation

npm install @know-your-ai/node
# or
yarn add @know-your-ai/node
# or
pnpm add @know-your-ai/node

Quick Start

ESM (Recommended)

import * as KnowYourAI from '@know-your-ai/node';
import { GoogleGenAI } from '@google/genai';

// Initialize the SDK
KnowYourAI.init({
  endpoint: 'https://your-backend.com/api/ai-events',
  apiKey: 'your-api-key',
  integrations: [
    KnowYourAI.googleGenAIIntegration({
      recordInputs: true,
      recordOutputs: true,
    }),
  ],
});

// Create and instrument your AI client
const genAI = new GoogleGenAI({ apiKey: process.env.GOOGLE_API_KEY });
const client = KnowYourAI.instrumentGoogleGenAIClient(genAI);

// Use the client as normal - all interactions will be captured
const response = await client.models.generateContent({
  model: 'gemini-1.5-pro',
  contents: 'Hello, world!',
});

CommonJS

const KnowYourAI = require('@know-your-ai/node');
const { GoogleGenAI } = require('@google/genai');

// Initialize the SDK
KnowYourAI.init({
  endpoint: 'https://your-backend.com/api/ai-events',
  apiKey: 'your-api-key',
  integrations: [
    KnowYourAI.googleGenAIIntegration({
      recordInputs: true,
      recordOutputs: true,
    }),
  ],
});

// Create and instrument your AI client
const genAI = new GoogleGenAI({ apiKey: process.env.GOOGLE_API_KEY });
const client = KnowYourAI.instrumentGoogleGenAIClient(genAI);

// Use the client as normal
const response = await client.models.generateContent({
  model: 'gemini-1.5-pro',
  contents: 'Hello, world!',
});

Configuration Options

init(options)

| Option | Type | Default | Description | |--------|------|---------|-------------| | endpoint | string | - | URL to send captured AI data | | apiKey | string | - | API key for authentication | | recordInputs | boolean | true | Whether to record input messages | | recordOutputs | boolean | true | Whether to record AI responses | | debug | boolean | false | Enable debug logging | | sampleRate | number | 1.0 | Sample rate for capturing (0.0-1.0) | | integrations | Integration[] | [] | List of integrations to use | | onCapture | function | - | Callback when data is captured | | transport | Transport | - | Custom transport for sending data |

Google GenAI Integration Options

KnowYourAI.googleGenAIIntegration({
  recordInputs: true,   // Record prompt messages
  recordOutputs: true,  // Record AI responses
});

Using the onCapture Callback

You can use the onCapture callback to handle captured data yourself:

KnowYourAI.init({
  onCapture: (data) => {
    console.log('AI Interaction:', {
      model: data.model,
      operation: data.operation,
      duration: data.duration,
      tokenUsage: data.tokenUsage,
    });
    
    // Send to your analytics service
    analytics.track('ai_interaction', data);
  },
  integrations: [
    KnowYourAI.googleGenAIIntegration(),
  ],
});

Captured Data Structure

Each captured AI interaction includes:

interface CapturedAIData {
  id: string;                    // Unique identifier
  timestamp: number;             // Unix timestamp
  provider: string;              // 'google_genai', 'openai', etc.
  model: string;                 // Model name (e.g., 'gemini-1.5-pro')
  operation: string;             // Operation type (e.g., 'generateContent')
  duration?: number;             // Duration in milliseconds
  input?: AIMessage[];           // Input messages (if recordInputs)
  output?: string;               // Response text (if recordOutputs)
  tokenUsage?: {
    inputTokens?: number;
    outputTokens?: number;
    totalTokens?: number;
  };
  toolCalls?: ToolCall[];        // Function/tool calls
  streaming?: boolean;           // Whether streaming was used
  error?: { message: string };   // Error info if failed
}

Custom Transport

You can create a custom transport to control how data is sent:

import { createHttpTransport, createConsoleTransport } from '@know-your-ai/node';

// Use built-in HTTP transport
KnowYourAI.init({
  transport: createHttpTransport({
    endpoint: 'https://your-api.com/events',
    apiKey: 'your-key',
    headers: {
      'X-Custom-Header': 'value',
    },
  }),
  integrations: [KnowYourAI.googleGenAIIntegration()],
});

// Or use console transport for debugging
KnowYourAI.init({
  transport: createConsoleTransport(),
  integrations: [KnowYourAI.googleGenAIIntegration()],
});

Streaming Support

The SDK automatically handles streaming responses:

const stream = await client.models.generateContentStream({
  model: 'gemini-1.5-pro',
  contents: 'Tell me a story',
});

// Iterate through the stream as normal
for await (const chunk of stream) {
  console.log(chunk.text());
}

// Data is captured automatically when the stream completes

Chat Sessions

Chat sessions are fully supported:

const chat = client.chats.create({
  model: 'gemini-1.5-pro',
  history: [
    { role: 'user', parts: [{ text: 'Hello!' }] },
    { role: 'model', parts: [{ text: 'Hi there!' }] },
  ],
});

// Each message is captured individually
const response = await chat.sendMessage({ message: 'How are you?' });

Architecture

The SDK follows a modular architecture similar to Sentry:

@know-your-ai/
├── core/           # Core library with client, integrations, and utilities
│   ├── client.ts   # Main client implementation
│   ├── integration.ts
│   ├── transport.ts
│   └── integrations/
│       └── google-genai/  # Google GenAI integration
└── node/           # Node.js entry point

Extending the SDK

Creating Custom Integrations

import { defineIntegration } from '@know-your-ai/node';

const myCustomIntegration = defineIntegration((options = {}) => {
  return {
    name: 'MyCustomIntegration',
    
    setupOnce() {
      // Called once when first installed
    },
    
    setup(client) {
      // Called for each client
    },
  };
});

KnowYourAI.init({
  integrations: [myCustomIntegration()],
});

Roadmap

• [ ] OpenAI integration
• [ ] Anthropic integration
• [ ] AWS Bedrock integration
• [ ] Azure OpenAI integration
• [ ] Automatic instrumentation (without manual wrapping)

License

MIT