Microsoft Teams Bridge for ElevenLabs Agents

@komaa/elevenlabs-msteams-bridge puts an ElevenLabs Agent on a real Microsoft Teams call. The hosted StandIn media bridge (standin.komaa.com) joins the Teams call and dials into this bridge over an HMAC-authenticated WebSocket; the bridge opens one ElevenLabs Agent conversation per call and relays between them.

Microsoft Teams call
       |
       v
StandIn media bridge      (hosted; joins the call)
       |   HMAC WebSocket, PCM 16 kHz
       v
this bridge               (you run it)
       |   WebSocket
       v
ElevenLabs Agent          (STT + LLM + TTS + turn-taking)

The hot path is copy-only: both sides speak base64 PCM 16 kHz mono (pcm_16000), so caller audio and agent audio are relayed verbatim in both directions. No resampling, no re-encoding, no transcoding.

Features

• Realtime voice, end to end - the caller talks to your ElevenLabs agent and hears it reply. Turn-taking, VAD and interruption are the agent's own (server-side); the bridge adds nothing to the latency budget beyond a relay hop.
• Barge-in done right - when the caller interrupts, the bridge cancels playback on the Teams side and drops stale in-flight agent audio by event_id, so no "audio ghosts" play after the cut.
• Per-call personalization - caller name, tenant and call direction are injected as dynamic_variables at conversation start; an optional localized greeting or spoken disclosure rides first_message; per-caller memory uses the caller's AAD id as user_id (guests get none, never a shared identity).
• Vision on demand - a look client tool lets the agent see the caller's camera or screen-share: describe-then-answer via any OpenAI-compatible vision endpoint, or native multimodal upload (recording-gated). See Vision.
• Agent client tools - end_call, express (avatar emotion), show_image (image on the bot's video tile, SSRF-guarded), look.
• Two call governors - a StandIn-side cutoff the bridge speaks a goodbye for, and a bridge-side MAX_CALL_MINUTES hard cap with a deterministic TTS goodbye.
• Observability - GET /healthz for liveness and GET /metrics (Prometheus text format): calls, durations, rejects, relay/drop counters.
• Hardened transport - replay-proof HMAC upgrade, single-use handshake guard, connection caps, payload caps, backpressure bounds, pre-start timeout, duplicate-call rejection, graceful SIGTERM drain, and an EL_HOST allowlist so your API key can only be sent to ElevenLabs.
• Group-call awareness - participant counts and DTMF digits are fed to the agent as contextual updates ("3 humans on the call, stay quiet unless addressed").

Install

Run it directly:

npx @komaa/elevenlabs-msteams-bridge

Or add it to your project:

npm install @komaa/elevenlabs-msteams-bridge

Node.js >= 20. One runtime dependency (ws).

Quick start

1. As a CLI (env-configured)

Set the three required variables, then run it:

export ELEVENLABS_API_KEY=sk_...
export ELEVENLABS_AGENT_ID=agent_...
export WORKER_SHARED_SECRET=...
npx @komaa/elevenlabs-msteams-bridge

Or keep them in a .env file (copy .env.example, which ships with the package) and load it:

node --env-file=.env node_modules/.bin/elevenlabs-msteams-bridge

Every option is an environment variable; see the Configuration Reference.

2. As a library

import { loadConfig, startServer } from "@komaa/elevenlabs-msteams-bridge";

// env-configured, same variables as the CLI
startServer(loadConfig());

With a custom vision hook (path-2 look: your model, your prompt, the raw frame never leaves your process). This example sends the frame to OpenAI's vision model and returns the description to the agent. Install the SDK alongside the bridge: npm i openai.

import OpenAI from "openai";
import { loadConfig, startServer, type VisionDescriber } from "@komaa/elevenlabs-msteams-bridge";

const openai = new OpenAI(); // reads OPENAI_API_KEY

// frame: { source: "camera" | "screenshare", mime, dataBase64, width, height, participantName?, ... }
const describeWithOpenAI: VisionDescriber = async (frame, question) => {
  const who = frame.source === "screenshare" ? "the caller's shared screen" : "the caller's camera";
  const res = await openai.chat.completions.create({
    model: "gpt-4o", // any vision-capable model
    max_tokens: 300,
    messages: [
      {
        role: "user",
        content: [
          { type: "text", text: `This is ${who}. ${question}` },
          {
            type: "image_url",
            image_url: {
              url: `data:${frame.mime};base64,${frame.dataBase64}`,
              detail: "low", // keep it fast and cheap for a live call
            },
          },
        ],
      },
    ],
  });
  // returned to the agent as the `look` tool result
  return res.choices[0]?.message?.content ?? "I could not make out the image.";
};

startServer(loadConfig(), undefined, describeWithOpenAI);

Azure OpenAI is the same call with a different client (use your deployment name as the model):

import { AzureOpenAI } from "openai";

const openai = new AzureOpenAI({
  endpoint: process.env.AZURE_OPENAI_ENDPOINT,   // https://<resource>.openai.azure.com
  apiKey: process.env.AZURE_OPENAI_API_KEY,
  apiVersion: "2024-10-21",
  deployment: "gpt-4o",                          // your Azure deployment name
});
// ...then call openai.chat.completions.create({ model: "gpt-4o", ... }) exactly as above.

The bridge's built-in path 2 (set VISION_API_URL / VISION_MODEL / VISION_API_KEY) already calls any OpenAI-compatible /chat/completions endpoint for you, no code required. Write a custom VisionDescriber only when you need a different model, provider, or prompt.

A complete runnable project lives in examples/basic-bridge/. The full programmatic surface (custom agent transports, HMAC helpers, protocol types) is documented in the Library API page.

3. Connect it to StandIn

StandIn is the hosted service that joins the Teams call and dials into this bridge. Pick a tier at standin.komaa.com (instant sandbox, a free developer tier with your own Teams bot, or a subscription for production), pair your identity, and then:

1. Set the identity's agent WebSocket URL to where this bridge listens, e.g. wss://el-bridge.example.com:8080/voice/msteams/stream (StandIn appends /{callId} per call).
2. Set WORKER_SHARED_SECRET to the shared secret from pairing - the two sides must match exactly, or the handshake is rejected with 401.
3. Call your Teams bot (or the sandbox meeting). StandIn joins, dials the bridge, and your ElevenLabs agent answers.

Expose it (make it reachable by StandIn)

StandIn is a hosted service, so it dials your bridge from the internet. If you run the bridge on your laptop or a private host, put a tunnel in front of it: the tunnel gives you a public https:///wss:// URL and terminates TLS for you (so you also satisfy the wss:// requirement below). Point it at the bridge's port (default 8080), and give StandIn the resulting wss://<public-host>/voice/msteams/stream URL.

Pick whichever you already use:

Tailscale Funnel (free, no account juggling if you have Tailscale):

tailscale funnel --bg --https=8080 8080

Your URL is then wss://<machine>.<tailnet>.ts.net:8080/voice/msteams/stream.

Cloudflare Tunnel (free, custom or trycloudflare.com hostname):

cloudflared tunnel --url http://localhost:8080

ngrok:

ngrok http 8080

VS Code / dev tunnels (devtunnel):

devtunnel host -p 8080 --allow-anonymous

Each prints a public https://… host; give StandIn the wss://…/voice/msteams/stream form of it. For a fixed production host, run the bridge behind an ingress or load balancer instead, or serve TLS natively with TLS_CERT_PATH + TLS_KEY_PATH.

Always use TLS (wss). Without a tunnel or terminator in front, the bridge serves plain WS and caller audio/video cross the wire in plaintext. The tunnels above terminate TLS for you; a raw ws:// URL should never be given to StandIn outside local testing.

Details and the tier walk-through: Connecting to StandIn.

Configuration

The most important variables (full list in .env.example and the reference):

| Env | Required | Meaning | |---|---|---| | ELEVENLABS_API_KEY | yes | Server-side key; mints signed URLs, uploads files, calls TTS. Never sent to the Teams side. | | ELEVENLABS_AGENT_ID | yes | The agent that answers calls. | | WORKER_SHARED_SECRET | yes | The shared secret from StandIn pairing (HMAC upgrade check). | | PORT / BIND | no | Listen address, default 0.0.0.0:8080. | | EL_HOST | no | Regional pin: api.us.elevenlabs.io, api.eu.residency.elevenlabs.io, api.in.residency.elevenlabs.io, api.sg.residency.elevenlabs.io. Restricted to *.elevenlabs.io so the API key cannot be sent elsewhere. | | EL_TTS_VOICE_ID | no | Enables the deterministic governor goodbye (exact text via standalone TTS). Without it, the goodbye is delegated to the agent. | | EL_FIRST_MESSAGE | no | Localized greeting / spoken AI disclosure (first_message override; must be allowlisted in the agent's security settings). | | MAX_CALL_MINUTES | no | Bridge-side hard cap per call (fractional ok, 0 = off). | | VISION_API_URL / VISION_API_KEY / VISION_MODEL | no | Path-2 vision: any OpenAI-compatible chat-completions endpoint with image input. |

Notes that save debugging time:

• The agent's audio in/out format must be pcm_16000 (agent settings). The bridge validates the conversation metadata at call start and logs an error on mismatch - anything else means garbled audio.
• conversation_config_override fields (first message, prompt, voice) are rejected by ElevenLabs unless allowlisted in the agent's security settings.
• Numeric env vars fail loud: MAX_CALL_MINUTES=abc stops startup with a clear error instead of silently disabling the governor.

Vision (look client tool)

Define a client tool named look on the agent (parameters: optional source = camera|screenshare, optional question). When the agent calls it, the bridge grabs the latest buffered frame and answers one of two ways:

1. Path 2 (preferred, if VISION_API_URL/VISION_MODEL are set): the frame goes to your OpenAI-compatible vision endpoint and the description comes back as the tool result. The raw frame never leaves the bridge (only the text description does), and it works regardless of recording state. Note the data flow, though: the description becomes ElevenLabs conversation content, which ElevenLabs persists by default - so a description of the caller's screen/camera can be stored by a third party even when Teams recording is off. This is a deliberate choice (vision stays usable without recording). If your deployment needs "no vision until recording is on," enable ElevenLabs zero-retention on the agent, or leave VISION_API_URL unset so only the recording-gated path 1 is available.
2. Path 1 (fallback): the frame is uploaded to the live ElevenLabs conversation and injected as a multimodal_message (the agent's LLM must be multimodal). Because this persists the raw frame with a third party, it is refused unless Teams recording is active.

The other client tools the bridge maps: end_call, express ({emotion}), show_image ({dataBase64, mime} or {url}, jpeg/png; URLs are SSRF-guarded - public hosts only, no redirects, bounded time and size).

Call governors

Two governors can end a call gracefully; both speak before hanging up:

• StandIn-side: when a tier limit is reached, StandIn sends assistant.say with the goodbye text; the bridge speaks it (exact text via standalone TTS when EL_TTS_VOICE_ID is set, otherwise the agent is asked to say it) and the call is torn down.
• Bridge-side (MAX_CALL_MINUTES > 0): the bridge arms a timer at call start. On expiry it flushes playback, speaks GOODBYE_TEXT, waits for the audio to play out (real TTS duration, or GOODBYE_GRACE_MS when unknown, always hard-bounded), then ends the call with reason time-limit. Use this when the billing limit lives with you, since ElevenLabs knows nothing about your budget.

Privacy / recording gate

StandIn reports the Teams recording state (recording.status). The bridge never logs or persists transcripts unless LOG_TRANSCRIPTS=true and recording is active. Video frames are buffered in memory only and dropped at teardown.

Vision and recording (know the trade-off): the recording gate blocks path-1 frame uploads to ElevenLabs when recording is off, but path-2 vision descriptions (see above) are ungated by design. In both cases caller audio and any vision descriptions transit ElevenLabs' cloud and are retained per the agent's retention settings. For deployments that must not retain caller data with a third party, enable ElevenLabs zero-retention on the agent and disclose in a spoken EL_FIRST_MESSAGE that an AI is on the call.

Documentation

• Docs site: komaa-com.github.io/elevenlabs-msteams-bridge - getting started, architecture, configuration and library API reference, wire protocol, troubleshooting.
• Example project: examples/basic-bridge/ - a runnable embedding with a custom vision hook and a documented .env.
• StandIn (the hosted service): standin.komaa.com · docs.komaa.com.

Repository layout

src/
  server.ts      HTTP + WS upgrade, HMAC validation, connection guards, session registry
  session.ts     per-call relay: StandIn WS ⇄ ElevenLabs Agent WS, tools, governors
  elevenlabs.ts  ElevenLabs Agent socket, signed-URL mint, standalone TTS, file upload
  protocol.ts    wire message types (JSON, camelCase, discriminated on "type")
  hmac.ts        HMAC-SHA256("{timestampMs}.{callId}") hex, constant-time verify
  ssrf.ts        public-URL guard for agent-supplied fetches
  vision.ts      path-2 describe-then-answer vision hook
  config.ts      env config (fail-loud numeric parsing)
examples/        runnable example projects
website/         docs site (Astro Starlight), deployed to GitHub Pages
test/            node:test suites (run with tsx)

Contributing

PRs welcome - see CONTRIBUTING.md for local setup, conventions, the release flow, and the documentation policy.

License

MIT