@demystify/media — one port for speech, providers behind it as drivers

Speech-to-text and text-to-speech behind a port we own. Sarvam is a driver. So is the mock. So is anything you write.

import { createMedia } from "@demystify/media";

const media = createMedia(); // no key, no network, works right now
await media.transcribe(audio, { languages: ["hi"], mode: "codemix" });

Zero runtime dependencies. sarvamai is an optional peer dependency the Sarvam driver lazy-imports; nothing is installed unless you ask for it.

The property that shapes everything else

The caller never names a vendor. It states constraints, and a selector picks:

const media = createMedia({
  drivers: { sarvam: sarvamDriver({ apiKey: process.env.SARVAM_API_KEY }) },
});

await media.transcribe(audio, { languages: ["hi"], mode: "codemix" });

"sarvam" appears once, in the wiring, and nowhere in the calling code. That is not tidiness — it is the difference between a vendor you can replace and a vendor you cannot. Speech vendors get replaced: prices move, a customer demands on-prem, a better Indic model ships. When it happens you edit one object literal.

The corollary is a rule: nothing vendor-specific may leak into the port. No Sarvam model ids, no BCP-47 dialect codes, no provider response shapes. Sarvam calls Odia od-IN; ISO calls it or; the port says or and the driver translates.

The routing table

| Constraint | Goes to | Why | |---|---|---| | Indian languages, any mode | Saaras v3 | 23 languages: 22 Indian + English | | Code-switched Hinglish | Saaras v3 codemix | A first-class output mode, not a workaround | | Realtime Indian voice | Saaras v3 realtime | WebSocket, WAV/PCM 16kHz, partial transcripts | | Indian TTS | Bulbul v3 | 11 languages — narrower than Saaras, deliberately checked | | onDevice: true | nothing — it refuses | No on-device driver ships in 0.1.0. See below. | | Anything else | whichever driver is registered, else a clear refusal | |

Selection rules, in order:

1. onDevice first. A driver that does not run locally is rejected before anything else is considered.
2. realtime filters to drivers that actually stream.
3. mode filters to drivers that support it. Ask for codemix from a driver that only transcribes and you get a refusal naming the mode, not a silent downgrade.
4. Languages. A driver that names hi beats one that declares "*". Ties break by registration order, so your ordering is a real preference.

media.explain("stt", { languages: ["hi"] }) returns the winner, why it won, and why every other driver lost. Print it at boot and you can see the routing you actually got.

onDevice refuses. It does not fall back.

await media.transcribe(audio, { onDevice: true });
// MediaError: no driver satisfies onDevice — none of the 2 registered stt
// driver(s) runs locally (sarvam: does not run on the device; …). No on-device
// driver ships with @demystify/media; register one, or drop the constraint
// deliberately.

onDevice: true means this audio may not leave the machine. The dangerous failure is not "no driver found" — that is loud. It is a network driver quietly answering, because by the time anyone notices, the audio is already gone. So it is checked first and refused by name.

The mock refuses it too, even though the mock runs in-process. The mock does not recognise speech; answering "yes, on-device" would turn a privacy constraint into a green test and a placeholder transcript.

Streaming

Two shapes. Use whichever fits.

// Drained for you: partials to the callback, resolves with the final.
const final = await media.transcribeStream(micChunks, {
  languages: ["hi"],
  onPartial: (p) => render(p.text),
});

// Or take the events yourself.
for await (const event of media.transcribeEvents(micChunks, { languages: ["hi"] })) {
  if (event.type === "partial") render(event.text);
  else commit(event.transcript);
}

The guarantees, in order of how much damage their absence does:

1. A stream that fails, fails. It never hangs. A stuck voice call reports nothing at all, which is worse than a failed one, because nothing upstream ever learns it stopped working. Errors are thrown into your for await; partials that already arrived stay delivered.
2. A stream that ends without a final says so — stream_ended_without_final. It does not hand back the last partial as if it were the answer. A partial is by definition a guess that was going to be revised.
3. Partials arrive in order and the final is last. A driver may emit several finals (one per utterance is normal on a long call); transcribeStream resolves with the last.

onPartial throwing propagates to you and closes the driver's stream, so a socket is not left open because a UI blew up.

Audio in: either { encoding, sampleRateHz, chunks }, or a bare AsyncIterable<Uint8Array> — which is read as 16kHz pcm_s16le, what realtime Indic ASR wants on the wire. Documented, not guessed.

Sarvam

import { createMedia, sarvamDriver } from "@demystify/media";

const media = createMedia({
  drivers: {
    sarvam: sarvamDriver({
      apiKey: process.env.SARVAM_API_KEY, // or set SARVAM_API_KEY
      baseUrl: "https://sarvam.customer.internal", // VPC / on-prem
      pricePerHour: { amountMinor: 3000, currency: "INR" }, // optional, see below
    }),
  },
});

npm install sarvamai   # optional peer — only if you use this driver

Without it you get one sentence rather than an ERR_MODULE_NOT_FOUND five frames deep in a dynamic import:

MediaError [driver_not_installed]: the Sarvam driver needs the optional peer
dependency `sarvamai`, which is not installed. Run `npm install sarvamai` …

Two Sarvam clients can be registered under different ids — sarvam-vpc and sarvam-cloud — and told apart by result.driver in a log.

Modes map straight through: transcribe · translate · verbatim · translit · codemix.

Money. cost is null unless you tell the driver what you pay. There is no built-in price list and there will not be one — a hardcoded price is wrong the day the vendor changes it and stays wrong silently. Given pricePerHour, cost comes back as integer minor units with the currency stated ({ amountMinor: 1500, currency: "INR" }), never a float.

What it does NOT do

• No on-device driver. onDevice: true refuses. Whisper.cpp, Moonshine and friends are a real gap and this package has the port for them, not the driver.
• No Whisper or other non-Indic driver in 0.1.0. The port is the same; the driver is unwritten.
• No batch API. Sarvam's async batch (≤2h audio, diarization) is not wired. transcribe is the sync REST path — under 30 seconds of audio. Longer audio needs batch, and you will get a provider error, not a helpful one from us.
• No diarization. TranscriptSegment.speaker exists on the port and is always null today, because Sarvam only diarizes on the batch API.
• No retries, no backoff, no circuit breaker. sarvamai has its own retry policy; we do not add a second one on top. Errors are marked retryable so you can decide.
• No metrics. Nothing is emitted anywhere, and nothing is logged.
• No audio conversion. No resampling, no transcoding. Raw PCM at anything other than 16kHz is refused rather than silently resampled.
• Not verified against a live Sarvam key. The request and response shapes are written against [email protected]'s published types and the whole driver is tested offline against a fake client. Nobody has yet run it against the real API. That is the honest state of it as of 0.1.0.

Errors

Every failure is a MediaError with a code you can branch on and a retryable flag, so "should I try again?" is answerable without parsing prose.

| code | retryable | meaning | |---|---|---| | no_driver_registered | no | nothing registered for that capability | | no_driver_satisfies_constraints | no | drivers exist; none matches | | unknown_driver | no | you named a driver id that is not registered | | driver_not_installed | no | an optional peer dependency is missing | | driver_misconfigured | no | no key, or the client is not the shape expected | | driver_failed | yes | the provider was reached and failed | | stream_ended_without_final | yes | the socket dropped before a final arrived | | invalid_input | no | the audio or text cannot be used | | aborted | no | your AbortSignal fired |

Standards

• Agnostic core. No Supabase, no ORM, no HTTP client, no framework. Web globals only (atob, Blob, File), so it runs in Node, Vercel serverless, and edge.
• Zero-config / keyless / offline default. createMedia() needs nothing. The mock honours abort signals, emits ordered partials, and its TTS returns a genuinely playable WAV — a reference implementation of the port, not a stub.
• Money is integer minor units with the currency alongside. No float exists in the code, and a fractional price is rejected at construction.
• ESM, Node ≥22, TypeScript strict with exactOptionalPropertyTypes. No any in the public surface.

Writing a driver

Implement SttDriver, TtsDriver, or both, and register it. Four methods and a capability declaration is the whole contract; src/drivers/mock.ts is a complete worked example, and src/ports.ts documents what each field is for.

const media = createMedia({
  drivers: { whisper: { stt: myWhisperDriver }, sarvam: sarvamDriver({ apiKey }) },
});

Testing

pnpm test   # 135 tests

99% statements, 100% of functions, 91% of branches. Two things are worth knowing about how they are written. The streaming guarantees are tested against a driver that misbehaves on purpose — drops mid-utterance, ends without a final, never streams at all — because those are the paths that matter and they do not occur naturally. And the whole Sarvam driver runs offline against a fake client, including sarvamai being absent, which is real here rather than simulated: .npmrc turns off pnpm's peer auto-install so the not-installed path is genuinely exercised.

MIT.