@lens-os/sdk

AI Agent SDK for building conversational AI experiences in web applications.

Features

• Server/Client Architecture - API keys stay on the server, frontend communicates via SSE
• Multi-turn Agent Loop - Autonomous LLM orchestration with tool execution
• Streaming Responses - Real-time text and tool call events via Server-Sent Events
• React Hooks - useLensAgent and useChat for easy frontend integration
• Tool System - 3-tier priority: Manual executors > Customer endpoints > Platform built-ins
• Session Management - Conversation history with automatic memory compaction
• Custom Tools - Define your own tool executors with full metadata for prompt generation
• Debug Mode - Configurable logging via debug flag
• TypeScript - Full type definitions included

Installation

npm install @lens-os/sdk
# or
bun add @lens-os/sdk

Peer dependencies:

• openai >= 4.0.0 (required)
• react >= 18.0.0 (optional, only for React hooks)

Architecture

Browser (React)                          Server (Next.js / Node)
┌──────────────────┐                    ┌──────────────────────┐
│  useLensAgent()  │── POST /agent ───> │  createAgentHandler  │
│                  │<── SSE stream ──── │    SupervisorAgent   │
│                  │── POST /action ──> │    (LLM + Tools)     │
└──────────────────┘                    └──────────────────────┘

• Frontend sends messages and receives streamed responses
• Server holds API keys, runs the agent loop, calls LLM and tools
• Action requests allow the server to ask the client to perform DOM operations (click, scroll, navigate, etc.)

Quick Start

1. Server-side: Create API Route

// app/api/agent/route.ts (Next.js App Router)
import { createAgentHandler } from '@lens-os/sdk/server';

const handler = createAgentHandler({
  apiKey: process.env.LENS_API_KEY!,
  openaiKey: process.env.OPENAI_API_KEY!,
  model: 'gpt-4o',          // optional, default: gpt-4o
  debug: true,               // optional, enables SDK logging
});

export const POST = handler.POST;

2. Server-side: Create Action Result Route

// app/api/agent/action/route.ts
import { createActionResultHandler } from '@lens-os/sdk/server';

// Share the pendingActions store from the agent handler
import { handler } from '../route';

const actionHandler = createActionResultHandler(handler._pendingActions);
export const POST = actionHandler.POST;

3. Client-side: React Hook

import { useLensAgent } from '@lens-os/sdk/react';

function ChatWidget() {
  const {
    messages,
    isLoading,
    sendMessage,
    abort,
  } = useLensAgent({
    endpoint: '/api/agent',
  });

  return (
    <div>
      {messages.map((msg, i) => (
        <div key={i} className={msg.role}>
          {typeof msg.content === 'string' ? msg.content : '...'}
        </div>
      ))}
      <input
        onKeyDown={(e) => {
          if (e.key === 'Enter') {
            sendMessage(e.currentTarget.value);
            e.currentTarget.value = '';
          }
        }}
        placeholder="Type a message..."
        disabled={isLoading}
      />
      {isLoading && <button onClick={abort}>Stop</button>}
    </div>
  );
}

Entry Points

| Import path | Usage | |---|---| | @lens-os/sdk | Core types, LensClient, SupervisorAgent, utilities | | @lens-os/sdk/react | React hooks (useLensAgent, useChat) | | @lens-os/sdk/server | Server handler (createAgentHandler, createActionResultHandler) |

Server Configuration

import { createAgentHandler } from '@lens-os/sdk/server';

const handler = createAgentHandler({
  // Required
  apiKey: string,              // Lens OS API key
  openaiKey: string,           // OpenAI API key

  // Optional
  baseUrl?: string,            // Default: https://osapi.ask-lens.ai
  model?: string,              // Default: gpt-4o
  maxTurns?: number,           // Default: 10
  language?: 'zh-TW' | 'en-US',
  debug?: boolean,             // Enable SDK logging (default: false)
  actionTimeout?: number,      // Action request timeout ms (default: 30000)

  // Shared client (reuse across handlers to share config cache)
  client?: LensClient,

  // Callbacks
  onTrace?: (trace: LLMTrace) => void,

  // Custom tool executors
  toolExecutors?: Record<string, ToolExecutorFunction | ToolExecutorConfig>,
});

React Hooks

useLensAgent

Full-featured hook for building chat interfaces.

import { useLensAgent } from '@lens-os/sdk/react';

const {
  // State
  messages,           // Message[] - conversation history
  isLoading,          // boolean - agent is running
  sessionId,          // string - current session ID
  error,              // Error | null

  // Actions
  sendMessage,        // (message: string, context?) => Promise<void>
  abort,              // () => void - cancel current execution
  clearMessages,      // () => void - clear local messages
  newSession,         // () => void - start a new session
  loadSession,        // (sessionId: string, messages: Message[]) => void
} = useLensAgent({
  endpoint: '/api/agent',

  // Optional
  actionResultEndpoint?: string,    // Custom action result endpoint
  userId?: string,
  headers?: Record<string, string> | (() => Record<string, string>),
  onEvent?: (event: SSEEvent) => void,
  getPageState?: () => Promise<PageState>,
  onActionRequest?: (action: string, params: Record<string, any>) => Promise<ToolResult>,
});

useChat

Simplified wrapper with shorter method names.

import { useChat } from '@lens-os/sdk/react';

const {
  messages,
  isLoading,
  error,
  send,     // sendMessage
  stop,     // abort
  clear,    // clearMessages
  reset,    // newSession
} = useChat({
  endpoint: '/api/agent',
});

Custom Tool Executors

Define custom tools on the server. Tool metadata is automatically injected into the LLM prompt.

const handler = createAgentHandler({
  apiKey: '...',
  openaiKey: '...',
  toolExecutors: {
    // Simple function
    my_tool: async (params, context) => {
      return { success: true, result: 'done' };
    },

    // Full config with metadata (recommended)
    order_lookup: {
      description: 'Look up order details by order ID',
      whenToUse: 'User asks about order status or delivery',
      schema: {
        orderId: { type: 'string', required: true, description: 'The order ID' },
      },
      output: 'Order object with status, items, tracking info',
      execute: async (params, context) => {
        const order = await db.orders.findById(params.orderId);
        if (!order) {
          return { success: false, error: 'Order not found' };
        }
        return { success: true, result: order };
      },
    },
  },
});

Tool Execution Priority

When the LLM calls a tool, the SDK resolves it in this order:

1. Manual toolExecutors - Code-defined executors (highest priority)
2. CUSTOMER mode - Database-configured external endpoint (POST to customerEndpoint)
3. PLATFORM mode - Built-in SDK implementations (knowledge_search, DOM tools, etc.)

Shared LensClient

When running multiple handlers or agents, share a LensClient instance to reuse the config cache (60s TTL with automatic retry):

import { LensClient } from '@lens-os/sdk';
import { createAgentHandler } from '@lens-os/sdk/server';

const sharedClient = new LensClient({
  apiKey: process.env.LENS_API_KEY!,
  baseUrl: 'https://osapi.ask-lens.ai',
});

const handler = createAgentHandler({
  apiKey: process.env.LENS_API_KEY!,
  openaiKey: process.env.OPENAI_API_KEY!,
  client: sharedClient,  // Reuses config cache
});

Debug Mode

By default, the SDK produces no console output. Enable debug logging:

// Via config
const handler = createAgentHandler({
  apiKey: '...', openaiKey: '...',
  debug: true,
});

// Or manually
import { setDebug } from '@lens-os/sdk';
setDebug(true);

SSE Event Types

The server streams these events to the client:

| Event type | Description | |---|---| | text | Streamed text content from the LLM | | tool_call | LLM is invoking a tool (name + parameters) | | tool_result | Tool execution result | | action_request | Server asks client to perform a DOM action | | error | Error occurred (may be fatal) | | done | Agent execution complete |

Page State & DOM Actions

The SDK supports multimodal context. Send page state with messages:

sendMessage('What products are on this page?', {
  pageState: {
    url: window.location.href,
    title: document.title,
    markdown: '...',           // Page content as markdown
    screenshot: 'data:...',    // Base64 screenshot
    actionableElements: [],    // Clickable elements
  },
});

The server can request DOM actions on the client (click, scroll, navigate, etc.) via the action request protocol. Handle them with onActionRequest:

useLensAgent({
  endpoint: '/api/agent',
  onActionRequest: async (action, params) => {
    // Execute DOM action and return result
    return { success: true, result: 'clicked' };
  },
});

Requirements

• Node.js >= 18
• React >= 18 (optional, for React hooks)
• OpenAI API key
• Lens OS API key

License

MIT