@bachstudio/mcp-asr-diarize

MCP server for speaker-attributed speech-to-text — it tells apart who is speaking and transcribes what they said, in one call.

Point it at a meeting recording and you get back:

模式=diarize  时长=1832.4s  语种=zh  说话人=张三、李四、说话人3
耗时=41.7s（decode:1.2s asr:33.8s diar:6.7s）  加速比=43.9x 实时
--------------------------------------------------------
[00:00:03] 张三: 各位好，今天的会议主要讨论第三季度的产品排期。
[00:00:11] 李四: 我这边的结论是，视频生成模块要往后推两周。
[00:00:19] 说话人3: 预算方面没有问题，人力是主要瓶颈。

Speakers whose voiceprint you have enrolled come back with their real name; everyone else gets 说话人1 / 2 / 3.

Install

Backend on the same machine (default http://127.0.0.1:8811):

claude mcp add asr -- npx -y @bachstudio/mcp-asr-diarize

Backend somewhere else — pass --url:

claude mcp add asr -- npx -y @bachstudio/mcp-asr-diarize --url https://your-host/asr

Or in any MCP client's config:

{
  "mcpServers": {
    "asr": {
      "command": "npx",
      "args": ["-y", "@bachstudio/mcp-asr-diarize", "--url", "https://your-host/asr"]
    }
  }
}

Zero dependencies, Node >= 18. --help prints every flag.

| Flag | Env | Default | Meaning | | --- | --- | --- | --- | | --url | ASR_BASE_URL | http://127.0.0.1:8811 | Backend address. A path prefix is fine (https://host/asr) | | --token | ASR_TOKEN | — | Bearer token, if the backend requires one | | --timeout | ASR_TIMEOUT | 1800 | Per-request timeout, seconds |

When the backend is on 127.0.0.1, files are handed over by absolute path (no copy). For any other address the file is uploaded over POST /upload first, so a remote GPU box behind a tunnel works with no shared filesystem.

Requires a backend

This package is the MCP front end only. It talks to an HTTP backend that owns the models. The reference backend runs faster-whisper (CTranslate2, fp16) on the GPU and sherpa-onnx (pyannote segmentation-3.0 + 3D-Speaker CAM++ embeddings) on the CPU, in parallel — on an RTX 4090 that lands around 40x realtime for a Chinese meeting, diarization included.

Backend HTTP contract

Any server implementing these four routes works:

| Route | Body | Returns | | --- | --- | --- | | GET /health | — | {ok, loaded[], gpu{}, voiceprints[]} | | POST /transcribe | {audio_id \| path, preset, language, mode, num_speakers, channel_names[], hotwords, identify} | {ok, mode, duration, language, speakers[], segments[{start,end,speaker,text}], text, timing{}, speedup} | | POST /enroll | {name, path} | {ok, name, seconds, enrolled[]} | | POST /upload?name=&audio_id=&final= | raw bytes | {ok, audio_id, path, bytes, complete, audio{}} | | POST /fetch | {url, name} | same as /upload | | POST /audio/delete | {audio_id} | {ok} |

Tools

| Tool | What it does | | --- | --- | | upload_audio | Hand the backend an audio file and get an audio_id | | transcribe_meeting | Audio/video → speaker-labelled transcript | | delete_audio | Drop an uploaded file early (they expire after 24h anyway) | | enroll_voiceprint | Register one person's voice so they get named | | list_voiceprints | List enrolled speakers | | delete_voiceprint | Remove one | | asr_health | Backend status, loaded models, free VRAM | | start_live_session | Open a live (as-you-speak) transcription session | | get_live_transcript | Pull what has been said so far, incrementally | | stop_live_session | Close it, and get a cleaned-up offline pass over the whole recording | | list_live_sessions | Sessions still open |

Live transcription

MCP is request/response — it cannot stream tokens back. So the live part happens between your client and the backend, and these tools let the model look in on it at any moment:

audio:  your page ──16kHz int16 PCM, chunked POST──▶ /live/push
MCP:    start_live_session()            → session_id
        get_live_transcript(session_id, since)  → only what is new
        stop_live_session(session_id)   → live transcript + a polished offline re-run

Measured on an RTX 4090 over a public tunnel: text appears ~1.5s after the speaker stops (0.35s end-of-utterance detection + polling interval + inference + round trip).

Someone who talks for two minutes without pausing would otherwise produce nothing until the forced cut, so get_live_transcript also returns partials: the sentence currently being spoken, re-transcribed roughly every 1.5s. On a 40-second stretch of unbroken speech the first words showed up at 2.4s instead of waiting 18s for the cut. Partials are provisional — they are replaced by the committed version once the sentence ends, and they never enter the speaker clustering.

Nobody has to enrol first. Strangers are separated automatically into 说话人1 / 2 / 3. After every sentence the backend re-clusters all voiceprints collected so far with agglomerative clustering, and reads the speaker count off the dendrogram's largest gap — so it never depends on an absolute similarity threshold, which drifts with recording quality. Because every sentence is re-judged against the full picture, earlier labels can be rewritten as evidence accumulates (two clusters turning out to be one person, say). The rev field increments whenever that happens: when it changes, re-fetch with since: 0 instead of taking the incremental slice. Pass num_speakers when you know it — it beats any auto-detection.

Anyone who is enrolled gets their real name from their first sentence instead of a number.

stop_live_session re-runs the whole recording through the offline pipeline. That second pass has better punctuation and speaker boundaries than anything produced sentence-by-sentence — keep that one. The recording is also saved as an audio_id you can re-run with other settings.

Pass language only when the meeting really is in one language. Sentence-at-a-time transcription is far more sensitive to a wrong language hint than whole-file transcription is: forcing zh onto an English sentence makes Whisper hallucinate fluent Chinese. Leave it empty and each utterance is detected on its own.

upload_audio

Three ways in — pick whichever you already have:

| Param | When to use | | --- | --- | | path | You have the file on the machine running the MCP server | | content | Base64 bytes inline. No address of any kind is needed, so CORS and auth on wherever the audio came from stop mattering | | url | The backend fetches it itself — useful when the client is a browser blocked by CORS |

Returns an audio_id. Uploads are chunked at 8MB, so a two-hour recording goes through reverse proxies that cap request bodies (Cloudflare's free tier stops at 100MB). The file is decoded once on arrival, so a corrupt or unsupported file fails right there instead of halfway through a transcription.

Pass the audio_id to transcribe_meeting. Re-running with different settings (another preset, a different speaker count) reuses the same upload — no second transfer.

transcribe_meeting

| Param | Default | Notes | | --- | --- | --- | | audio_id | — | From upload_audio. Either this or path | | path | — | Local file, any container: webm/opus/m4a/mp3/wav/mp4 | | preset | balanced | fast / balanced / accurate | | language | auto | Pass zh / en when known — faster and more accurate | | mode | auto | split / diarize / single | | num_speakers | auto | Pass it when known; materially better clustering | | speaker_names | ["对方","我"] | Channel labels for split mode | | hotwords | — | Proper nouns, space separated | | identify | true | Match clusters against enrolled voiceprints |

The split shortcut, and why it isn't the whole story

If the file is stereo and the two channels differ, mode: auto transcribes each channel separately. That is the fast path for recordings captured as left = them, right = me — the "me vs them" split is physical, so no clustering error is possible there.

But "them" is usually several people. So the far-end channel is also run through voiceprint clustering, and comes back as 对方1 / 对方2 / … (just 对方 when there really is only one). The near-end channel is your own microphone, assumed to be one person — set mic_multi if a room full of people shares it. Turn the whole thing off with split_diarize: false.

Measured on a 45s stereo scene with three distinct voices on the far end and one on the near end: 11 of 12 utterances labelled correctly, speaker count auto-detected exactly. The single miss was a 1.4-second "Thanks." — utterances under about two seconds carry too little voice for a reliable embedding.

License

MIT © bachstudio