Synthesised & Extended Audio Description (SEAD)

Experimental: I have not (yet much) researched how acceptable synthesised speech is to end-users for audio description on the Web.

• Demo: nfreear.github.io/audio-describe/demo

Usage

npm install audio-describe

CDNs

• Unpkg.com: unpkg.com/audio-describe
• esm.sh: esm.sh/audio-describe

Javascript

JavaScript to import and register the audio-describe-controller custom element:

import { AudioDescribeElement } from 'audio-describe';

customElements.define('audio-describe-controller', AudioDescribeElement);

HTML

<audio-describe-controller>
  <video src="path/to/video.mp4" controls>
    <track kind="descriptions" srclang="en" src="path/to/ext-audio-description.en.vtt">
  </video>
</audio-describe-controller>

Other elements can be substituted in place of <video>, to support for example, YouTube, Vimeo, and Video.js — see mediaElement property in the API.

Problem

Organisations frequently produce short videos for social media and the Web, with many short cuts and few gaps in the audio track. Even with the benefit of planning, it would be hard to find spaces for standard audio description.

Proposed Solution

The proposed solution combines:

• Extended audio description
• WebVTT tracks for 'descriptions'
• WebVTT files containing JSON metadata
• Speech Synthesis Web API (widely supported)

Documentation

Work-In-Progress!

• Background
• WebVTT extension, for Synthesised & Extended Audio Description
• API

WebVTT

An example of WebVTT containing extended audio description and metadata:

WEBVTT

...

4
00:00:16.000 --> 00:00:17.000
The next line is extended audio description!

5
00:00:18.000 --> 00:00:18.400
Now the yacht is sailing down wind, parallel with the shore.

6
00:00:18.600 --> 00:00:19.100
{
  "pauseMedia": 3000
}

7
00:00:21.000 --> 00:00:22.000
End of extended audio description.

More example VTT.

License

• License: MIT