iRewind Web MediaInfo

Reads what a media file is, in the browser: kind, container, duration, size, codecs, creation date, custom metadata. Works on a File/Blob or on a URL (HTTP range requests), for video, audio and images.

import { probeMedia, getVideoThumbnail } from '@irewind/web-mediainfo'

const info = await probeMedia(fileOrUrl, {
  signal, // AbortSignal
  analyzer, // 'js' | 'element' | 'wasm', or an array of them
  expect, // 'video' | 'audio' | 'image', or an array of them
  showMessage // (message) => void
})

// every kind
info.analyzer // 'js' | 'element' | 'wasm': the one that answered
info.kind // 'video' | 'audio' | 'image'
info.container // 'mp4' | 'mov' | 'm4a' | 'webm' | 'mkv' | 'mxf' | 'wav' | 'mp3' | 'jpeg' | 'png' | ...
info.size // bytes
info.createdAt // epoch ms

// video and audio
info.duration // ms, a multiple of 40
info.fastStart // mp4/mov/m4a, webm/mkv: the index comes before the media data
info.audio // { codec, profile, bitDepth, mime, duration, bitrate, channels, sampleRate }
info.metadata // { name: value } of the custom tags

// video and image
info.width // display size, rotation applied
info.height
info.aspectRatio // width / height

// video
info.video // { codec, profile, level, bitDepth, chroma, mime, duration, bitrate, frameRate, rotation, hasAlpha }

// image
info.image // { hasAlpha, isAnimated }

The result only has the fields that apply to the kind of the media, and of those only the ones that have a value: no key ever holds null. A field is left out when the file does not have it or does not say, and when the analyzer that answered cannot tell.

if (info.audio) {
  // the file has sound
}

const bitrate = info.video.bitrate ?? fallback

Always there: analyzer, kind, container, size; duration for a video or an audio file; width, height and aspectRatio for a video or an image; codec and duration in audio and video, and rotation and hasAlpha in video; hasAlpha in image. The element analyzer gives less (see below).

How a stream is encoded

| Field | In | What | | ---------- | ---------------- | ------------------------------------------------------------------------------------------------------- | | profile | video, audio | the profile of the codec, in lower case with dashes (the tables below) | | level | video | '4.1', '5.1'; a name for MPEG-2 ('main', 'high') | | bitDepth | video, audio | bits of a color component (8, 10, 12); bits of a sample for sound stored without loss (PCM, FLAC, ALAC) | | chroma | video | '4:2:0', '4:2:2', '4:4:4', '4:0:0' | | mime | video, audio | the name browsers know the codec by: 'avc1.640028', 'hvc1.1.6.L93.B0', 'mp4a.40.2', 'opus' | | bitrate | video, audio | bits per second |

| Video codec | profile | | ----------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | h264 | baseline, constrained-baseline, main, extended, high, progressive-high, constrained-high, high-10, high-422, high-444, their -intra variants (high-422-intra), cavlc-444-intra | | hevc | main, main-10, main-still-picture, rext | | vp9 | '0' to '3' | | av1 | main, high, professional | | prores | proxy, lt, standard, hq, 4444, 4444-xq | | mpeg2 | simple, main, 422, high | | mpeg4 | simple, advanced-simple |

| Audio codec | profile | | ----------- | ------------------------------------------------------------------------------------- | | aac | lc, he, he-v2, main, ltp, ssr, ld, eld, xhe | | pcm | float for floating point samples (which is what its bitDepth of 32 or 64 then is) |

mime is what goes in the codecs parameter of a MIME type, to ask the browser whether it plays the stream:

const { video, audio } = await probeMedia(file)

if (video.mime) {
  const codecs = [video.mime, audio?.mime].filter(Boolean).join(', ')
  MediaSource.isTypeSupported(`video/mp4; codecs="${codecs}"`)
}

These fields come from the configuration the container keeps for the decoder, so the js analyzer has them without reading any media data. What it has depends on the container: a WebM file seldom says more about its VP9 stream than its name (mime: 'vp9'), an MXF names the profile of its H.264 but not always the level. The wasm analyzer has what MediaInfo reports, which is sometimes more (it decodes a frame) and sometimes less (no bit depth for ProRes, no bitrate for Opus and Vorbis). Its mime is put together from the profile and the level: the flags MediaInfo does not report (the constraints of H.264 and HEVC beyond what the profile implies) are the usual ones, not necessarily those of the file.

Analyzers

analyzer lists the analyzers to try, in order. An analyzer that cannot answer (or fails) hands the file over to the next one; a single name means only that analyzer. The default is ['js', 'wasm'].

| Analyzer | Reads | Fills | | --------- | -------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------- | | js | mp4, mov, m4a, webm, mkv, mxf, mp3, flac, ogg (Opus, Vorbis), wav (PCM), jpeg, png, gif, webp, bmp, heic, avif | everything | | wasm | everything above, plus avi, raw aac and whatever the js one declines | everything | | element | what the browser can load in a <video> or an <img> | kind, duration, width, height, aspectRatio |

• js only reads the headers of the file: a few kilobytes, no startup cost. It fails closed: anything it does not fully recognize (a fragmented mp4, an unknown codec, a variable-bitrate mp3 that does not say how many frames it has...) is declined, not guessed. An MXF is read from its header metadata, and from its footer when the header was written before the recording ended (camera files): H.264, MPEG-1/2, MPEG-4 Visual or ProRes video with PCM audio; what is reported is what the file stores, not the cut of it that its material package plays.
• wasm is our build of MediaInfoLib (see the features below), downloaded on first use. It is also the reference the js readers are tested against. It parses the least that tells what a file is, which is still megabytes of it. A file that stores no duration (a recording that was never finalized, such as what MediaRecorder gives, raw AAC, a variable-bitrate mp3 without an info frame) is then gone through in full to time it: over HTTP, that downloads all of it.
• element never loads the wasm and covers the browser-playable files the js analyzer declines (a fragmented mp4, raw aac). It needs a DOM and, for URLs, CORS. It picks <img> or <video> from expect, then the file extension, then the first bytes. It also gives size when that takes no request (always for a File). Its kind is what the browser can decode: a video whose codec it cannot play (ProRes, for one) but whose audio it can comes out as audio.

// full info: JS first, wasm when needed
await probeMedia(file)

// basics only, never loads the wasm
await probeMedia(url, { analyzer: ['js', 'element'], expect: 'video' })

// only the wasm analyzer
await probeMedia(file, { analyzer: 'wasm' })

Options

• expect rejects when the media is of another kind. When the first bytes already tell (an image where a video is expected), that happens before any parsing.
• showMessage is called with a status while working ('Loading content analyzer', 'Analyzing content', 'Measuring duration' when the whole file has to be gone through) and without arguments when done.
• signal aborts the probe; the promise rejects with the signal's reason (an AbortError by default).

Errors

probeMedia rejects with a plain Error whose message can be shown as it is:

• no analyzer in the chain could read the media: Cannot read clip.xyz: it is not a media file, it is damaged, or its format is not supported. Its cause is the error of the last analyzer that failed, when one did.
• the kind is not one of the expected ones: Expected a video, but photo.jpg is an image.

An aborted probe rejects with the signal's reason (an AbortError by default), and an invalid option throws a TypeError.

Notes on the result

• A file with a video track is a video, otherwise one with an audio track is an audio. A video without sound has no audio. With the element analyzer, which never gives audio, that says nothing about the sound.
• Durations (of the file and of its video and audio) are floored to a multiple of 40 ms, a frame at 25 fps.
• metadata holds the free-form tags of the file (what ffmpeg writes with -metadata name=value; in mp4/mov that needs -movflags use_metadata_tags): the key/value store of mp4/mov, Matroska tags, ID3v2 user text frames, Vorbis comments. Matroska upper-cases tag names. Well-known tags such as the encoder may be listed by one analyzer and not by the other; look up your own names.
• createdAt comes from the container for mp4/mov, Matroska and MXF. For a picture (jpeg, heic, avif, png, webp) it is the Exif capture time, failing that the Exif "last written" time; in a PNG the Exif chunk is only looked for in front of the image data. For a broadcast wav it is the origination date and time. A date stored without a time zone (MXF, most Exif data, broadcast wav) is taken as UTC, so a file gives the same createdAt wherever it is read; a time zone the file does give (an Exif offset) is applied.
• video.frameRate is in frames per second; for a variable frame rate it is the average. Matroska and WebM files often do not store one, nor a bitrate.
• audio.bitrate is exact for PCM and for an mp3 of constant bitrate; for the rest it is the size of the audio over its duration. For an Ogg file it is the size of the file over its duration.
• fastStart tells whether the index of the file comes before its media data. For an mp4/mov/m4a, that is whether it can start playing over HTTP before it has fully downloaded. A WebM or Matroska file always can; there it tells whether its seek index (the Cues) is up front, so that seeking takes no fetch from the end of the file (false when the file has none, as a recording that was never finalized). Other containers have no such choice, and no fastStart.
• image.isAnimated covers animated GIF, WebP and PNG, which stay kind: 'image' (and so have no duration). For a GIF it is left out when the first 64 KB of the file do not tell (a large picture without the loop marker of an animation).
• video.rotation is in degrees clockwise (0, 90, 180, 270); width and height already account for it, and for the Exif orientation of a JPEG.

getVideoThumbnail

const { thumbnailFile, thumbnailUrl, cleanupFn } = await getVideoThumbnail(
  fileOrUrl,
  { offset: 1, returnThumbnailUrl: true }
)

Grabs the frame at offset seconds as a 480px wide PNG File. With returnThumbnailUrl it also returns an object URL and the cleanupFn that revokes it.

Imports

The two functions are separate files, and the package has no side effects: a bundler only takes the one an app uses, and a page that only makes thumbnails does not load the analyzers (1.5 KB against 62 KB).

// both work the same
import { probeMedia, getVideoThumbnail } from '@irewind/web-mediainfo'

import probeMedia from '@irewind/web-mediainfo/probeMedia'
import getVideoThumbnail from '@irewind/web-mediainfo/getVideoThumbnail'

Development

yarn build              # wasm (Docker) + JS, into dist/
yarn build:wasm         # only dist/mediainfo.{js,wasm}; also refreshes the features below
yarn build:js           # only the JS: dist/probeMedia.js, dist/getVideoThumbnail.js and dist/index.js
yarn build:test-media   # generates the samples in test/media (needs ffmpeg; Pillow for WebP and AVIF, macOS for HEIC)
yarn test               # compares the js analyzer against the wasm analyzer on every sample
yarn dev                # playground: both analyzers side by side, in the browser

The parser allowlist of the wasm build is in src/wasm/build/flags.sh; the source patches it needs are in src/wasm/patches.

Features of the wasm analyzer

wasm: 1.59 MB (1,590,110 bytes)

MediaInfoLib: v26.05 (ZenLib v0.4.41, emscripten 6.0.11)

Containers (6): adm, mk, mpeg4, mxf, ogg, riff

Video (9): afdbardata, av1, avc, hevc, mpeg4v, mpegv, prores, vp8, vp9

Audio (8): aac, ac3, flac, iab, mpega, opus, pcm, vorbis

Images (5): bmp, gif, jpeg, png, webp

Tags (7): exif, icc, id3, id3v2, tag, vorbiscom, xmp