@musicbento/audio-to-midi

Browser-first Audio to MIDI conversion for JavaScript and TypeScript using Spotify Basic Pitch.

Built by MusicBento.com, with an interactive Audio to MIDI Converter.

Install

npm install @musicbento/audio-to-midi

Quick start

import {
  audioToMidiDefaults,
  convertAudioToMidi,
} from '@musicbento/audio-to-midi';

const file = fileInput.files?.[0];
if (!file) throw new Error('Select an audio file first');

const result = await convertAudioToMidi(file, audioToMidiDefaults, {
  onProgress(progress) {
    console.log(Math.round(progress * 100));
  },
});

const url = URL.createObjectURL(result.midiBlob);

The result contains:

{
  title: string;
  filename: string;
  noteCount: number;
  midiBytes: Uint8Array;
  midiBlob: Blob;
}

Reuse the loaded model

For multiple conversions, create one converter instance so the Basic Pitch model stays loaded:

import {
  AudioToMidiConverter,
  audioToMidiDefaults,
} from '@musicbento/audio-to-midi';

const converter = new AudioToMidiConverter(audioToMidiDefaults);

const first = await converter.convert(firstFile);
const second = await converter.convert(secondFile);

Supported input

The package accepts browser File objects with these extensions:

• MP3
• WAV
• M4A
• AAC
• OGG
• FLAC
• WebM

Actual decoding support depends on the browser's Web Audio implementation.

The default limits match MusicBento's browser converter:

• Maximum file size: 100 MB
• Maximum duration: 8 minutes
• Transcription sample rate: 22,050 Hz

Model loading

audioToMidiDefaults uses the model shipped with this npm package through jsDelivr:

import { bundledModelUrl } from '@musicbento/audio-to-midi';

console.log(bundledModelUrl);

Audio data stays in the browser. Using bundledModelUrl makes a network request only for the Basic Pitch model files.

Self-host the model

The published package includes:

model/model.json
model/group1-shard1of1.bin

Copy the model directory from node_modules/@musicbento/audio-to-midi/ into your application's public assets, then provide your own URL:

import {
  AudioToMidiConverter,
  audioToMidiDefaults,
} from '@musicbento/audio-to-midi';

const converter = new AudioToMidiConverter({
  ...audioToMidiDefaults,
  modelUrl: '/models/basic-pitch/model.json',
});

This keeps both the audio and model delivery under your own application origin.

TensorFlow backend

The default backend is webgl. To use the CPU backend:

const converter = new AudioToMidiConverter({
  ...audioToMidiDefaults,
  backend: 'cpu',
});

Progress and status

const result = await converter.convert(file, {
  onStatus(status) {
    console.log(status);
  },
  onProgress(progress) {
    console.log(progress);
  },
});

Status values are:

loading-model
decoding-audio
transcribing
preparing-midi

Errors

Conversion failures use AudioToMidiError with a machine-readable code:

import {
  AudioToMidiError,
  audioToMidiDefaults,
  convertAudioToMidi,
} from '@musicbento/audio-to-midi';

try {
  await convertAudioToMidi(file, audioToMidiDefaults);
} catch (error) {
  if (error instanceof AudioToMidiError) {
    console.error(error.code);
  }
}

Privacy

Audio decoding, transcription, and MIDI generation run locally in the browser. The package does not upload audio content to MusicBento.com.

For a ready-to-use browser tool, visit MusicBento.com Audio to MIDI Converter.

License

MIT

The bundled Basic Pitch model is distributed under Spotify Basic Pitch's Apache-2.0 license. Its license text is included in THIRD_PARTY_LICENSES/spotify-basic-pitch-LICENSE.