MoQ (Media over QUIC) Transport

Media-over-QUIC transport package for use with @pipecat-ai/client-js.

Installation

npm install \
@pipecat-ai/client-js \
@pipecat-ai/moq-transport

Overview

The MoqTransport class connects a PipecatClient to a Pipecat MoQ bot, either through a MoQ relay or directly to a bot running in serve mode. It publishes the local microphone as an Opus broadcast and consumes the bot's broadcast — both audio and a bot→client RTVI message track — using catalog discovery, so codec and sample rate are negotiated at connect time rather than pinned in code. RTVI messages also flow client→bot over a matching transcript track on the client's own broadcast, carrying client-ready, typed text input, function-call results, and other client→server RTVI traffic.

Connection management uses WebTransport with a WebSocket fallback (raced by @moq/net), with auto-reconnect via Connection.Reload.

Transcript records

Each transcript track is a single group that a subscriber always reads from its first record, so a reconnect on either side replays the whole log. To keep a replay from redelivering messages such as client-ready, every record is the RTVI message plus two fields: seq, the record's position in the publisher's log, and epoch, an opaque string identifying that log. The transport numbers what it sends and drops incoming records at or below the last seq it accepted for the current epoch; a different epoch is a new bot, whose count starts over. Both fields are removed before the message reaches PipecatClient. A record without seq is delivered unchanged, so a bot that predates the fields keeps working. acceptTranscriptRecord and the TranscriptRecord type are exported for other implementations of the same stream.

Each side also appends a { label: "moq-transport", type: "session-ending" } record before it leaves. The bot's tracks ending after that marker is a hangup, and the transport disconnects. Ending without it may be a relay between the peers failing, which looks the same on the wire, so the transport redials: the bot either reappears on the new session or is never announced on it.

Features

• 🎤 Microphone capture and Opus publish (@moq/publish)
• 📡 WebTransport with WebSocket fallback and auto-reconnect (@moq/net)
• 🎧 Catalog-driven bot audio playback with bounded-latency jitter buffering (@moq/watch)
• 💬 Bidirectional RTVI messages over dedicated transcript tracks (@moq/json) — bot→client events/transcripts, and client→bot messages like client-ready
• 🔐 serverCertificateHashes pinning for self-signed dev relays

Usage

Basic Setup

import { PipecatClient } from "@pipecat-ai/client-js";
import { MoqTransport } from "@pipecat-ai/moq-transport";

const transport = new MoqTransport({
  relayUrl: "https://relay.example.com:4080/moq",
});

const pcClient = new PipecatClient({
  transport,
  callbacks: {
    // Event handlers
  },
});

await pcClient.connect();

Self-signed dev relay

const transport = new MoqTransport({
  relayUrl: "https://localhost:4080/moq",
  serverCertificateHashes: [
    { algorithm: "sha-256", value: certHashBytes },
  ],
});

Configuration Options

interface MoqTransportOptions {
  relayUrl: string;                              // Required: full URL of the MoQ peer
  serverCertificateHashes?: WebTransportHash[];  // Optional: pinned cert hashes for self-signed dev setups
  clientId?: string;                             // Optional: this client's participant id (default "client0")
  botId?: string;                                // Optional: peer (bot) participant id to consume (default "bot0")
  namespace?: string;                            // Optional: top-level namespace / room name (default "pipecat")
  transcriptTrack?: string;                      // Optional: track name for the bidirectional RTVI transcript channel (default "transcript.json.z")
  audioLatencyMs?: number;                       // Optional: jitter buffer floor latency in ms (default 80)
  audioBufferMaxMs?: number | "real-time";       // Optional: buffered-playback latency ceiling in ms, or "real-time" to collapse to the floor (default 30000)
  audioSampleRate?: number;                      // Optional: mic publish sample rate in Hz; one of 8000/12000/16000/24000/48000 (default 48000)
  relayRetryTimeoutMs?: number;                  // Optional: how long to keep trying to reach the relay before giving up, in ms; finite and positive (default 60000)
}

Broadcast paths are derived as <namespace>/<clientId> (publish) and <namespace>/<botId> (subscribe). Audio track names inside each broadcast are discovered from the bot's catalog, so they aren't configured directly.

Connecting via a bot /start endpoint

If your /start endpoint returns the bot's MoQ config nested under a moq key (the shape the bot's pipecat.transports.moq.transport returns), pass that response straight to connect() / PipecatClient.startBotAndConnect() — MoqTransport unwraps it (including base64-decoding certHash into serverCertificateHashes) in _validateConnectionParams, no app-side transform needed:

{
  "moq": {
    "relayUrl": "https://relay.example.com:4080/moq",
    "certHash": "base64-encoded-sha-256-or-null",
    "namespace": "pipecat",
    "clientId": "client0",
    "botId": "bot0",
    "transcriptTrack": "transcript.json.z"
  }
}

Handling Events

The transport implements the various Pipecat event handlers. Check out the docs or samples for more info.

API Reference

States

The transport can be in one of these states:

• "disconnected"
• "initialized"
• "connecting"
• "connected"
• "ready"
• "disconnecting"
• "error"

Once the session is ready, a relay reconnect does not change the state. The connection redials in the background. A message sent meanwhile is kept in the transcript log and replayed to the bot when it resubscribes; audio from the gap is not. If the relay cannot be reached for the whole retry window (relayRetryTimeoutMs, 60 seconds by default), the transport moves to error and reports a fatal error through onError, and PipecatClient disconnects. The same window bounds the first connection. Behind a layer-4 load balancer with source-IP stickiness, a lost relay keeps receiving the client's reconnects until the balancer takes it out of rotation, so the window has to outlast that.

Error Handling

The transport includes error handling for:

• Microphone acquisition failures (initDevices, _connect)
• Invalid relayUrl
• WebTransport / WebSocket connection failures (retried by @moq/net auto-reconnect; fatal once its retry window runs out)
• Catalog decode and audio decode errors (logged; the consume loop continues)

License

BSD-2 Clause