sensoit

Production-ready Node.js SDK for Sensoit AI observability & safety platform.

Features

• Auto-tracing - Automatically trace LLM calls with zero code changes
• Multi-provider - OpenAI, Anthropic, Groq, Mistral, Google, and any OpenAI-compatible API
• Budget enforcement - Set limits on tokens, cost, and time per session
• Async guardrails - Non-blocking safety checks that never slow down your app
• Framework integrations - LangChain.js and Vercel AI SDK support
• TypeScript-first - Full type definitions with CommonJS and ESM support

Installation

npm install sensoit
# or
yarn add sensoit
# or
pnpm add sensoit

Quick Start

import { init, wrapOpenAI, forceFlush } from 'sensoit';
import OpenAI from 'openai';

// Initialize the SDK
init({
  apiKey: process.env.SENSOIT_API_KEY,
});

// Wrap your OpenAI client
const openai = wrapOpenAI(new OpenAI());

// All calls are now automatically traced!
const response = await openai.chat.completions.create({
  model: 'gpt-4o',
  messages: [{ role: 'user', content: 'Hello!' }],
});

// Flush before exit
await forceFlush();

Provider Wrappers

OpenAI

import OpenAI from 'openai';
import { wrapOpenAI } from 'sensoit';

const openai = wrapOpenAI(new OpenAI());

Anthropic

import Anthropic from '@anthropic-ai/sdk';
import { wrapAnthropic } from 'sensoit';

const anthropic = wrapAnthropic(new Anthropic());

Groq

import Groq from 'groq-sdk';
import { wrapGroq } from 'sensoit';

const groq = wrapGroq(new Groq());

Mistral

import { Mistral } from '@mistralai/mistralai';
import { wrapMistral } from 'sensoit';

const mistral = wrapMistral(new Mistral({ apiKey: process.env.MISTRAL_API_KEY }));

Google (Gemini)

import { GoogleGenerativeAI } from '@google/generative-ai';
import { wrapGoogle } from 'sensoit';

const genAI = new GoogleGenerativeAI(process.env.GOOGLE_API_KEY);
const model = wrapGoogle(genAI.getGenerativeModel({ model: 'gemini-1.5-pro' }));

LiteLLM / OpenAI-Compatible

import OpenAI from 'openai';
import { wrapLiteLLM } from 'sensoit';

// Works with LiteLLM proxy, vLLM, Ollama, etc.
const client = wrapLiteLLM(
  new OpenAI({ baseURL: 'http://localhost:4000' }),
  { provider: 'litellm' }
);

Sessions with Budget Enforcement

Sessions group related operations and enforce budgets:

import { startSession, wrapOpenAI, BudgetExceededError } from 'sensoit';

const session = startSession({
  agentId: 'my-agent',
  userId: 'user-123',
  budget: {
    maxTokens: 10000,    // Max 10,000 tokens
    maxCost: 1.00,       // Max $1.00
    maxSeconds: 300,     // Max 5 minutes
    autoAbort: true,     // Throw error if exceeded
  },
});

try {
  // Create spans within the session
  const span = session.span('process-data');
  // ... do work ...
  span.end();

  // LLM calls are automatically tracked against the budget
  const response = await openai.chat.completions.create({
    model: 'gpt-4o',
    messages: [{ role: 'user', content: 'Hello!' }],
  });

  session.end('Success');
} catch (error) {
  if (error instanceof BudgetExceededError) {
    console.log('Budget exceeded:', error.type, error.limit, error.used);
    session.abort('Budget exceeded');
  } else {
    session.fail(error);
  }
}

Framework Integrations

LangChain.js

import { ChatOpenAI } from '@langchain/openai';
import { SensoitLangChainHandler } from 'sensoit';

const handler = new SensoitLangChainHandler();

const model = new ChatOpenAI({
  callbacks: [handler],
});

// All LangChain operations are now traced
const response = await model.invoke('Hello!');

Vercel AI SDK

import { generateText } from 'ai';
import { openai } from '@ai-sdk/openai';
import { useVercelAITracing } from 'sensoit';

// Wrap the function
const tracedGenerateText = useVercelAITracing(generateText);

// Use normally - tracing is automatic
const result = await tracedGenerateText({
  model: openai('gpt-4o'),
  prompt: 'Hello!',
});

Client Classes

Feedback

Collect end-user feedback:

import { FeedbackClient, getClient } from 'sensoit';

const feedback = new FeedbackClient({ client: getClient() });

// Thumbs up/down
await feedback.thumbsUp({ sessionId: 'session-123' });
await feedback.thumbsDown({ sessionId: 'session-123' });

// Star rating
await feedback.rating({ sessionId: 'session-123', value: 4, maxValue: 5 });

// Comment
await feedback.comment({ sessionId: 'session-123', text: 'Great response!' });

Judge

Evaluate AI outputs:

import { JudgeClient, getClient } from 'sensoit';

const judge = new JudgeClient({ client: getClient() });

// Evaluate a response
const result = await judge.evaluate({
  input: 'What is 2+2?',
  output: 'The answer is 4.',
  expected: '4',
});

console.log(result.verdict); // 'PASS' or 'FAIL'
console.log(result.score);   // 0-1
console.log(result.reason);  // Explanation

Dataset

Manage test datasets:

import { DatasetClient, getClient } from 'sensoit';

const datasets = new DatasetClient({ client: getClient() });

// Create dataset
const dataset = await datasets.create({
  name: 'Math Questions',
  description: 'Test cases for math problems',
});

// Add test cases
await datasets.addTestCases(dataset.id, [
  { input: 'What is 2+2?', expected: '4' },
  { input: 'What is 10*5?', expected: '50' },
]);

// Run evaluation
const run = await datasets.runEval(dataset.id, {
  promptId: 'prompt-123',
});

Configuration

init({
  // Required
  apiKey: 'your-api-key',

  // Optional
  baseUrl: 'https://api.sensoit.io',  // Custom API URL
  timeout: 30000,                      // Request timeout (ms)
  flushInterval: 5000,                 // Background flush interval (ms)
  maxBatchSize: 100,                   // Max spans per batch
  guardrailsEnabled: true,             // Enable async guardrails
  debug: false,                        // Enable debug logging

  // Connection pooling
  keepAlive: true,
  maxConnections: 20,

  // Retry configuration
  retryConfig: {
    maxRetries: 3,
    initialDelayMs: 500,
    maxDelayMs: 10000,
    multiplier: 2,
    retryableStatuses: [429, 500, 502, 503, 504],
  },

  // Rate limiting
  rateLimitRps: 100,

  // Shutdown hooks
  registerShutdownHook: true,  // Auto-flush on process exit
});

Manual Tracing

Create custom spans for non-LLM operations:

import { createSpan, SpanType } from 'sensoit';

const span = createSpan({
  name: 'process-data',
  type: SpanType.CUSTOM,
  metadata: { source: 'api' },
});

span.setInput({ items: ['a', 'b', 'c'] });

// Do work...

span.setOutput({ processed: 3 });
span.end();

Decorator

Use the @trace decorator for class methods:

import { trace, SpanType } from 'sensoit';

class MyService {
  @trace('process-item', { type: SpanType.CUSTOM })
  async processItem(item: string): Promise<string> {
    // Automatically traced
    return item.toUpperCase();
  }
}

Lifecycle

import { init, forceFlush, shutdown, isReady } from 'sensoit';

// Initialize
init({ apiKey: 'your-key' });

// Check if ready
if (isReady()) {
  // Make calls...
}

// Force flush (call before exit)
await forceFlush();

// Shutdown (releases resources)
await shutdown();

Environment Variables

• SENSOIT_API_KEY - Your Sensoit API key (can be passed to init() instead)

Examples

See the examples directory:

• basic_openai.ts - Basic OpenAI tracing
• agent_session.ts - Session with budget enforcement
• multi_provider.ts - Multiple LLM providers

Run examples:

npx ts-node examples/basic_openai.ts

TypeScript Support

Full TypeScript support with all types exported:

import {
  // Config types
  SensoitConfig,
  RetryConfig,
  BudgetConfig,

  // Span types
  SpanType,
  SpanData,
  LLMCallData,

  // Session types
  SessionStatus,
  SessionConfig,
  SessionData,

  // Error types
  SensoitError,
  BudgetExceededError,
  GuardrailBlockedError,
  ApiError,

  // Response types
  ApiResponse,
  GuardrailResult,
  JudgeResult,
  FeedbackResponse,
} from 'sensoit';

License

MIT