@deepslate-labs/livekit

LiveKit Agents plugin for Deepslate's realtime voice AI API (TypeScript).

@deepslate-labs/livekit provides a RealtimeModel implementation for the LiveKit Agents Node framework, enabling seamless integration with Deepslate's unified voice AI infrastructure — speech-to-speech streaming, server-side VAD, LLM inference, and optional ElevenLabs TTS, all in a single WebSocket connection.

Features

• Realtime Voice AI Streaming — Low-latency bidirectional audio streaming over WebSockets
• Server-side VAD — Voice Activity Detection handled by Deepslate with configurable sensitivity
• Function Tools — Define and invoke tools using LiveKit's llm.tool() helper
• Flexible TTS — Server-side TTS via Deepslate-hosted (cloned) voices or ElevenLabs, with automatic context truncation on interruption
• Automatic Interruption Handling — Truncates the in-flight response when users interrupt

Installation

npm install @deepslate-labs/livekit

Requirements

• Node.js 18 or higher

Peer dependencies

• @livekit/agents ^1.0.7 — LiveKit Agents framework (Node)
• @livekit/rtc-node ^0.13.27 — LiveKit realtime audio frames

npm install @livekit/agents @livekit/rtc-node

@deepslate-labs/core is pulled in automatically.

Prerequisites

Deepslate Account

Sign up at deepslate.eu and set the following environment variables:

DEEPSLATE_VENDOR_ID=your_vendor_id
DEEPSLATE_ORGANIZATION_ID=your_organization_id
DEEPSLATE_API_KEY=your_api_key

ElevenLabs TTS (optional)

For server-side text-to-speech with automatic interruption handling:

ELEVENLABS_API_KEY=your_elevenlabs_api_key
ELEVENLABS_VOICE_ID=your_voice_id
ELEVENLABS_MODEL_ID=eleven_turbo_v2  # optional

Note: You can alternatively use LiveKit's built-in client-side TTS. However, context truncation on interruption only works with server-side TTS configured via ElevenLabsTtsConfig / HostedTtsConfig.

Quick Start

import { fileURLToPath } from "node:url";
import { type JobContext, ServerOptions, cli, defineAgent, voice } from "@livekit/agents";
import { RealtimeModel, elevenLabsConfigFromEnv } from "@deepslate-labs/livekit";

export default defineAgent({
  entry: async (ctx: JobContext) => {
    await ctx.connect();

    const session = new voice.AgentSession({
      llm: new RealtimeModel({
        ttsConfig: elevenLabsConfigFromEnv(),
      }),
    });

    await session.start({
      agent: new voice.Agent({ instructions: "You are a helpful voice AI assistant." }),
      room: ctx.room,
    });

    session.generateReply({ instructions: "Greet the user and offer your assistance." });
  },
});

cli.runApp(new ServerOptions({ agent: fileURLToPath(import.meta.url) }));

Configuration

RealtimeModel

The constructor takes a single options object (RealtimeModelOptions):

| Field | Type | Default | Description | |---|---|---|---| | vendorId | string | env: DEEPSLATE_VENDOR_ID | Deepslate vendor ID | | organizationId | string | env: DEEPSLATE_ORGANIZATION_ID | Deepslate organization ID | | apiKey | string | env: DEEPSLATE_API_KEY | Deepslate API key | | baseUrl | string | "https://app.deepslate.eu" | Base URL for Deepslate API | | systemPrompt | string | "You are a helpful assistant." | System prompt for the model | | temperature | number | 1.0 | Sampling temperature (0.0–2.0) | | generateReplyTimeout | number | 30.0 | Timeout in seconds for generateReply (0 = no limit) | | vad | VadConfig | defaults | Voice activity detection tuning | | ttsConfig | TtsConfig | undefined | TTS configuration (enables server-side audio output) | | wsUrl | string | undefined | Direct WebSocket URL (for local dev/testing) |

VAD Configuration

import { RealtimeModel } from "@deepslate-labs/livekit";

const model = new RealtimeModel({
  vad: {
    confidenceThreshold: 0.5,   // 0.0–1.0: minimum confidence to classify as speech
    minVolume: 0.01,            // 0.0–1.0: minimum volume to classify as speech
    startDurationMs: 200,       // ms of speech required to trigger start
    stopDurationMs: 500,        // ms of silence required to trigger stop
    backbufferDurationMs: 1000, // ms of audio buffered before detection triggers
  },
});

Tuning tips:

• Noisy environments: increase confidenceThreshold (0.6–0.8) and minVolume (0.02–0.05)
• Lower latency: decrease startDurationMs (100–150) and stopDurationMs (200–300)
• Natural pacing: slightly increase stopDurationMs (600–800)

HostedTtsConfig

Use a voice cloned and hosted within Deepslate. No external TTS credentials required.

import { RealtimeModel, HostedTtsMode } from "@deepslate-labs/livekit";

const model = new RealtimeModel({
  ttsConfig: {
    provider: "hosted",
    voiceId: "c3dfa73f-a1ab-4aad-b48a-0e9b9fe4a69f",
    mode: HostedTtsMode.HIGH_QUALITY, // or LOW_LATENCY
  },
});

ElevenLabsTtsConfig

import { RealtimeModel, ElevenLabsLocation, elevenLabsConfigFromEnv } from "@deepslate-labs/livekit";

// Load from environment variables
const model = new RealtimeModel({ ttsConfig: elevenLabsConfigFromEnv() });

// Or configure manually
const model = new RealtimeModel({
  ttsConfig: {
    provider: "eleven_labs",
    apiKey: "your_elevenlabs_key",
    voiceId: "21m00Tcm4TlvDq8ikWAM",
    modelId: "eleven_turbo_v2",
    location: ElevenLabsLocation.US, // US (default), EU, or INDIA
  },
});

Function Tools

Use LiveKit's llm.tool() helper to expose tools to the model. Tool parameters are described with a zod schema:

import { llm, voice } from "@livekit/agents";
import { z } from "zod";

const lookupWeather = llm.tool({
  description: "Get the current weather for a given location.",
  parameters: z.object({
    location: z.string().describe("The city or location to look up weather for."),
  }),
  execute: async ({ location }) => `It's sunny and 22°C in ${location}.`,
});

const agent = new voice.Agent({
  instructions: "You are a helpful assistant.",
  tools: { lookupWeather },
});

Sending a Welcome Message

DeepslateRealtimeSession emits a "session_initialized" event once the WebSocket session is fully initialized and ready to accept messages. Listen for it (and use the speakDirect() helper) to send a welcome message instead of relying on a fixed delay:

const model = new RealtimeModel({ ttsConfig: elevenLabsConfigFromEnv() });
const session = model.session();

session.on("session_initialized", () => {
  void session.speakDirect("Hello! How can I help you today?");
});

Examples

The examples/ directory contains a ready-to-run agent you can use as a starting point.

chat-agent.ts — Voice assistant with function tools

A fully working LiveKit agent that demonstrates:

• Connecting to a LiveKit room
• Server-side ElevenLabs TTS with interruption handling
• Two example function tools: lookupWeather and getCurrentLocation

packages/livekit/examples/
├── chat-agent.ts      # The agent
└── .env.example       # Required environment variables

Setup:

# 1. Configure credentials
cd packages/livekit/examples
cp .env.example .env
# Edit .env and fill in your credentials

# 2. Run (tsx is included as a workspace dev dependency)
pnpm tsx examples/chat-agent.ts dev

Documentation

• Deepslate Documentation
• LiveKit Agents Documentation
• API Reference

License

Apache License 2.0 — see LICENSE for details.