musicodec

A perception channel for music, for text-only LLMs. Like OCR for documents or captioning for images: audio goes in, structured arrangement facts come out — sections, dominant stems, texture forms (facture), energy, plus timestamped ASR lyrics and per-song upstream confidence. Perception, not conclusions.

What the LLM gets

{
  "song": { "key": "Eb", "scale": "minor", "bpm": 161.6, "n_sections": 19 },
  "sections": [
    { "t_start": 32.7, "t_end": 44.1, "cluster": "C",
      "stem1": "bass", "phase1": "sustained",
      "stem2": "vocals", "phase2": "sustained",
      "stem3": "drums", "phase3": "sustained",
      "n_active": 4, "energy": "loud", "chord_density": "dense",
      "lyrics": ["昨日今朝以后", "长夜无尽头"] }
  ],
  "lyrics": { "lines": [ { "t_start": 33.7, "t_end": 35.8, "text": "昨日今朝以后" } ] },
  "confidence": { "upstream": { "key_strength": 0.921, "beats_confidence": 1.74,
    "separation": { "residual_ratio": 0.058, "fell_back_to_4stem": false } } }
}

• cluster is an anonymous structural label (A–H), not verse/chorus — the tool does not claim semantics it cannot measure.
• phase* is the facture of each stem (Schaeffer 1966): impulse / iterative / sustained.
• null always means unmeasured, never zero — the tool refuses to guess.
• Lyrics are transcribed from this very audio (separated vocals), so timestamps align with sections; fetch canonical lyrics yourself for text corroboration.

Setup

The MCP server is a thin client; analysis runs on a GPU backend service.

# 1. reach the backend (if remote):
ssh -N -L 8643:localhost:8643 <gpu-host>

# 2. register the MCP server (Claude Code example):
claude mcp add musicodec -- npx -y musicodec@latest
# or with an explicit backend:
MUSICODEC_API=http://127.0.0.1:8643 npx -y musicodec@latest

@latest matters during the beta: plain npx -y musicodec reuses a cached copy and will not pick up new releases.

Optional timeouts (ms) — the client aborts rather than hanging if the backend stops responding: MUSICODEC_TIMEOUT_MS (polling, default 30000), MUSICODEC_TIMEOUT_ANALYZE_MS (upload, default 180000).

Tools

• musicodec_analyze { file_path, force_recompute? } — submit a local audio file (wav/mp3/m4a/flac). Returns a job_id; identical audio returns instantly from cache (byte-level and perceptual-fingerprint caches, always disclosed via result.source; force_recompute bypasses both). Typical flow: yt-dlp -x --audio-format wav <url> first, then submit the clean file.
• musicodec_result { job_id } — poll (~2 min per song). Full field semantics are in the tool description.
• musicodec_appreciate_song { file_path? } — returns the music-critic working instructions (persona switch + evidence discipline for consuming the analysis). Call it first when asked to review a song. The same content is exposed as the appreciate_song MCP prompt for harnesses that surface prompts; the tool form exists because many agent harnesses never show prompts to the model.
• musicodec_feedback { job_id, usefulness, ... } — optional structured judgement after you have used an analysis (fields consumed, contradictions with corroborating evidence, human-confirmed flag). Feeds the per-field contradiction scoreboard; never send reasoning or review text.

Design

Design doc: the codec layer is a lossless, round-trip-verified bijection; perception error stays attributed to the upstream detectors (Demucs / librosa / Essentia / FireRedASR2), whose self-reported confidences are passed through untouched. Upstream improves → results improve, schema unchanged.

MIT