Patter is the open-source SDK that gives your AI agent a phone number. Point it at any function that returns a string, and Patter handles the rest: telephony, speech-to-text, text-to-speech, and real-time audio streaming. You build the agent — we connect it to the phone.

Quickstart

npm install getpatter

Set the env vars your carrier and engine need:

export TWILIO_ACCOUNT_SID=ACxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
export TWILIO_AUTH_TOKEN=your_auth_token
export OPENAI_API_KEY=sk-xxxxxxxxxxxxxxxxxxxxxxxx

Four lines of TypeScript:

import { Patter, Twilio, OpenAIRealtime } from "getpatter";

const phone = new Patter({ carrier: new Twilio(), phoneNumber: "+15550001234" });
const agent = phone.agent({ engine: new OpenAIRealtime(), systemPrompt: "You are a friendly receptionist for Acme Corp.", firstMessage: "Hello! How can I help?" });
await phone.serve({ agent, tunnel: true });

tunnel: true spawns a Cloudflare tunnel and points your Twilio number at it. In production, pass webhookUrl: "api.prod.example.com" to the constructor instead.

Features

| Feature | Method | Example | |---|---|---| | Inbound calls | phone.serve({ agent }) | Answer calls as an AI | | Outbound calls + AMD | phone.call({ to, machineDetection: true }) | Place calls with voicemail detection | | Tool calling | agent({ tools: [tool(...)] }) | Agent calls external APIs mid-conversation | | Custom STT + TTS | agent({ stt: new DeepgramSTT(), tts: new ElevenLabsTTS() }) | Bring your own voice providers | | Dynamic variables | agent({ variables: {...} }) | Personalize prompts per caller | | Pluggable LLM | agent({ llm: new AnthropicLLM() }) | 5 built-in providers: OpenAI, Anthropic, Groq, Cerebras, Google | | Custom LLM (any model) | serve({ onMessage }) | Route to anything — local inference, internal gateways, etc. | | Call recording | serve({ recording: true }) | Record all calls | | Call transfer | transfer_call (auto-injected) | Transfer to a human | | Voicemail drop | call({ voicemailMessage: "..." }) | Play message on voicemail | | Phone-as-a-tool (external agents) | new PatterTool({ phone, agent }).execute(...) | Drop into LangChain / OpenAI Assistants / Hermes / MCP |

Configuration

Environment variables

Every provider reads its credentials from the environment by default. Pass apiKey: "..." to any constructor to override.

| Variable | Used by | |---|---| | TWILIO_ACCOUNT_SID, TWILIO_AUTH_TOKEN | new Twilio() carrier | | TELNYX_API_KEY, TELNYX_CONNECTION_ID, TELNYX_PUBLIC_KEY (optional) | new Telnyx() carrier | | PLIVO_AUTH_ID, PLIVO_AUTH_TOKEN | new Plivo() carrier — Auth Token doubles as the V3 webhook signature key | | OPENAI_API_KEY | OpenAIRealtime, WhisperSTT, OpenAITTS | | ELEVENLABS_API_KEY, ELEVENLABS_AGENT_ID | ElevenLabsConvAI, ElevenLabsTTS | | DEEPGRAM_API_KEY | DeepgramSTT | | CARTESIA_API_KEY | CartesiaSTT, CartesiaTTS | | RIME_API_KEY | RimeTTS | | LMNT_API_KEY | LMNTTTS | | SONIOX_API_KEY | SonioxSTT | | ASSEMBLYAI_API_KEY | AssemblyAISTT | | ANTHROPIC_API_KEY | AnthropicLLM | | GROQ_API_KEY | GroqLLM | | CEREBRAS_API_KEY | CerebrasLLM | | GEMINI_API_KEY (or GOOGLE_API_KEY) | GoogleLLM |

cp .env.example .env
# Edit .env with your API keys

Other carriers: Telnyx and Plivo are both fully supported alternatives to Twilio. All three carriers receive equal support for inbound DTMF, transfer, AMD, status callbacks, recording, voicemail drop, and metrics. Plivo additionally supports native DTMF send over the media WebSocket — a capability Twilio Media Streams lacks. Plivo's Auth Token doubles as the V3 webhook signature key (no separate public key, unlike Telnyx Ed25519).

Voice Modes

| Mode | Latency | Quality | Best For | |---|---|---|---| | OpenAI Realtime | Lowest | High | Fluid, low-latency conversations | | Pipeline (STT + LLM + TTS) | Low | High | Independent control over STT and TTS | | ElevenLabs ConvAI | Low | High | ElevenLabs-managed conversation flow |

API Reference

Patter constructor

new Patter({
  carrier: Twilio | Telnyx | Plivo;
  phoneNumber: string;
  webhookUrl?: string;                              // Public hostname. Mutually exclusive with tunnel.
  tunnel?: CloudflareTunnel | StaticTunnel | boolean;  // `true` is shorthand for new CloudflareTunnel().
})

| Parameter | Type | Description | |---|---|---| | carrier | Twilio / Telnyx / Plivo | Carrier instance. Reads env vars by default. | | phoneNumber | string | Your phone number in E.164 format. | | webhookUrl | string | Public hostname your local server is reachable on. | | tunnel | CloudflareTunnel \| StaticTunnel \| boolean | new CloudflareTunnel(), new StaticTunnel({ hostname: ... }), or true (shorthand for new CloudflareTunnel()). |

phone.agent()

phone.agent({
  systemPrompt: string;
  engine?: OpenAIRealtime | ElevenLabsConvAI;        // default: new OpenAIRealtime()
  stt?: STTProvider;                                 // e.g. new DeepgramSTT()
  tts?: TTSProvider;                                 // e.g. new ElevenLabsTTS()
  voice?: string;
  model?: string;
  language?: string;
  firstMessage?: string;
  tools?: Tool[];
  guardrails?: Guardrail[];
  variables?: Record<string, string>;
})

Pass engine for end-to-end mode, stt + tts for pipeline mode. Both arguments may take plain adapter instances (e.g. new DeepgramSTT()) that read their API key from the environment.

phone.serve()

await phone.serve({
  agent: Agent;
  port?: number;
  tunnel?: boolean;                 // shortcut for Patter({ tunnel: new CloudflareTunnel() })
  dashboard?: boolean;
  recording?: boolean;
  onCallStart?: (data) => Promise<void>;
  onCallEnd?: (data) => Promise<void>;
  onTranscript?: (data) => Promise<void>;
  onMessage?: (data) => Promise<string> | string;
  voicemailMessage?: string;
  dashboardToken?: string;
});

phone.call()

await phone.call({
  to: string;
  agent?: Agent;
  from?: string;
  firstMessage?: string;
  machineDetection?: boolean;
  voicemailMessage?: string;
  ringTimeout?: number;
});

STT / TTS catalog

import {
  // Carriers
  Twilio, Telnyx, Plivo,
  // Engines
  OpenAIRealtime, ElevenLabsConvAI,
  // STT
  DeepgramSTT, WhisperSTT, OpenAITranscribeSTT, CartesiaSTT, SonioxSTT, AssemblyAISTT,
  // TTS
  ElevenLabsTTS, OpenAITTS, CartesiaTTS, RimeTTS, LMNTTTS,
  // LLM
  OpenAILLM, AnthropicLLM, GroqLLM, CerebrasLLM, GoogleLLM,
  // Tunnels
  CloudflareTunnel, StaticTunnel,
  // Primitives
  Tool, Guardrail, tool, guardrail,
  // Integrations
  PatterTool,
} from "getpatter";

Every class reads its API key from the environment by default, so new DeepgramSTT() / new ElevenLabsTTS() work out of the box when the corresponding env var is set.

Examples

Inbound calls — default engine

import { Patter, Twilio, OpenAIRealtime } from "getpatter";

const phone = new Patter({ carrier: new Twilio(), phoneNumber: "+15550001234" });
const agent = phone.agent({
  engine: new OpenAIRealtime(),
  systemPrompt: "You are a helpful customer service agent.",
  firstMessage: "Hello! How can I help?",
});

await phone.serve({
  agent,
  tunnel: true,
  onCallStart: (data) => console.log(`Call from ${data.caller}`),
  onCallEnd: () => console.log("Call ended"),
});

Custom voice — Deepgram STT + ElevenLabs TTS

import { Patter, Twilio, DeepgramSTT, ElevenLabsTTS } from "getpatter";

const phone = new Patter({ carrier: new Twilio(), phoneNumber: "+15550001234" });
const agent = phone.agent({
  stt: new DeepgramSTT(),                         // reads DEEPGRAM_API_KEY
  tts: new ElevenLabsTTS({ voice: "sarah" }),     // reads ELEVENLABS_API_KEY
  systemPrompt: "You are a helpful voice assistant.",
});
await phone.serve({ agent, tunnel: true });

Pipeline mode — pick STT, LLM, TTS independently

import { Patter, Twilio, DeepgramSTT, AnthropicLLM, ElevenLabsTTS } from "getpatter";

const phone = new Patter({ carrier: new Twilio(), phoneNumber: "+15550001234" });
const agent = phone.agent({
  stt: new DeepgramSTT(),                         // reads DEEPGRAM_API_KEY
  llm: new AnthropicLLM(),                        // reads ANTHROPIC_API_KEY
  tts: new ElevenLabsTTS({ voiceId: "sarah" }),   // reads ELEVENLABS_API_KEY
  systemPrompt: "You are a helpful voice assistant.",
});
await phone.serve({ agent, tunnel: true });

Available LLM providers: OpenAILLM, AnthropicLLM, GroqLLM, CerebrasLLM, GoogleLLM. Tool calling works across all five. For fully custom logic, drop llm and pass an onMessage callback to serve() instead.

Tool calling

import { Patter, Twilio, OpenAIRealtime, tool } from "getpatter";

const checkAvailability = tool({
  name: "check_availability",
  description: "Check appointment availability for a given ISO date.",
  parameters: {
    type: "object",
    properties: { date: { type: "string" } },
    required: ["date"],
  },
  handler: async ({ date }) => ({ available: true }),
});

const phone = new Patter({ carrier: new Twilio(), phoneNumber: "+15550001234" });
const agent = phone.agent({
  engine: new OpenAIRealtime(),
  systemPrompt: "You are a booking assistant.",
  tools: [checkAvailability],
});
await phone.serve({ agent, tunnel: true });

Outbound calls

import { Patter, Twilio, OpenAIRealtime } from "getpatter";

const phone = new Patter({ carrier: new Twilio(), phoneNumber: "+15550001234" });
const agent = phone.agent({
  engine: new OpenAIRealtime(),
  systemPrompt: "You are making reminder calls.",
  firstMessage: "Hi, this is a reminder from Acme Corp.",
});

await phone.serve({ agent, tunnel: true });
await phone.call({ to: "+14155551234", agent });

Dynamic variables

const agent = phone.agent({
  engine: new OpenAIRealtime(),
  systemPrompt: "You are helping {customer_name}, account #{account_id}.",
  firstMessage: "Hi {customer_name}! How can I help you today?",
  variables: { customer_name: "Jane", account_id: "A-789" },
});

Contributing

Pull requests are welcome.

cd libraries/typescript && npm install && npm test

Please open an issue before submitting large changes so we can discuss the approach first.

License

MIT — see LICENSE.