🎧 synced-audio-guide

A reusable, multilingual, synchronized audio companion for any video exhibit.

Your screens play videos; visitors' phones play the narration in their language, locked to the exact timestamp on screen — and it stays in sync, even across loops. Drop it into any project that plays videos (RFID-triggered, touch kiosks, looping installations, playlists) with a few lines.

• The guide (this package) owns everything sync: a per-video authoritative clock, Socket.IO, the phone UI, the QR, and serving audio.
• Your host app owns video playback (and why the video changes — RFID, buttons, timers). It includes a tiny reporter and calls guide.play(videoId, videoEl) — that's the whole integration.

Node.js + Express + Socket.IO, vanilla front-end. No framework, no database, no build step. Runs fully offline on a LAN.

┌─ your host app (:3000) ────────────┐        ┌─ synced-audio-guide (:4000) ─────────┐
│  something plays a video           │        │  • one authoritative clock per video │
│  <script src=":4000/reporter.js">  │ ─tick─► │  • Socket.IO rooms + 1s sync         │
│  guide.play('neuron', videoEl)     │        │  • phone UI (numbered grid)          │
│  guide.showQr('#corner') ◄─ QR ────┼────────┤  • serves audio to phones            │
└────────────────────────────────────┘        └──────────────┬───────────────────────┘
                                             phones scan QR ──►│  pick number + language → synced

Install

npm install synced-audio-guide

Requires Node 18+.

Quick start

npx synced-audio-guide init

This scaffolds an asset/video/ folder to fill and a starter guide.config.js.

Add your videos + audio

One folder per video; inside it the video file and an audio/ folder with one file per language:

asset/video/
├── video1/
│   ├── video1.mp4          # the video (omit it if your host plays its own)
│   └── audio/
│       ├── english.mp3     # each track = same length as the video, synced to it
│       └── hindi.mp3
└── video2/
    ├── video2.mp4
    └── audio/
        ├── english.mp3
        └── hindi.mp3

The folder name is the videoId (what you pass to guide.play); its grid number comes from the digits in the name.

Configure + run

guide.config.js (the init command writes a starter one):

const path = require('path');
module.exports = {
  museumName: 'Cell Specialization',
  port: 3000,
  contentDir: path.join(__dirname, 'asset', 'video'),   // the folder above
  // languages: [...],   // optional — defaults to English + Hindi
};

Run it as its own service:

npx synced-audio-guide --config guide.config.js

…or start it programmatically from your own Node code:

const { createAudioGuide } = require('synced-audio-guide');
createAudioGuide(require('./guide.config.js')).listen();

Open a screen per video, fullscreen — each plays its video with the QR bottom right; visitors scan, pick a number + language, and hear it in sync:

http://<ip>:3000/display?video=video1
http://<ip>:3000/display?video=video2

The boot log prints these URLs. There's also a live demo at /host-demo.

Embedded in a host app (your app plays the video)

If your project already plays the video (RFID kiosk, touch screen, looping installation…), run the guide as a separate service and let your display report to it. Your contentDir then only needs each video's audio/ folder — the .mp4 is optional, since your app plays its own video.

Report from your display

In your host's display page, include the reporter and tell the guide which video is playing — however your project decides that:

<video id="player"></video>
<div id="qr-corner"></div>

<script src="http://GUIDE_IP:4000/reporter.js"></script>
<script>
  const guide = AudioGuide.connect();          // infers the guide from the script's origin
  const player = document.getElementById('player');
  guide.showQr('#qr-corner');                  // optional: render the guide's QR

  // whenever your app starts a video:
  function playVideo(videoId, file) {
    player.src = file;                          // your own video source
    player.loop = true;
    player.play();
    guide.play(videoId, player);               // ← the whole integration
  }
</script>

Switching video is just another play() — the previous video greys out on phones instantly. guide.stop() when nothing is playing.

Worked example — RFID (Cell Specialization)

const TAG_TO_VIDEO = { '04A1B2': 'neuron', '04C3D4': 'muscle' };   // your RFID map

function onRfidScan(tag) {
  const videoId = TAG_TO_VIDEO[tag];
  if (!videoId) return;
  playVideo(videoId, `/videos/${videoId}.mp4`);                    // your app's own video path
}

The videoId (neuron, muscle) must match a folder under the guide's contentDir that has an audio/ folder — e.g. asset/video/neuron/audio/.

A visitor scans the QR → the phone shows the numbered grid with the currently-playing video's number enabled (others greyed) → they pick the number + a language → the narration plays in sync. Scan a new tag and the phone follows within the drift-correction window.

Only ever show one looping video? Call guide.play(id, el) once on load and never again — the grid simply has a single button.

Pose / RFID kiosks → use phoneMode: 'follow'

When the clip is triggered (a pose match, an RFID scan) and plays for a few seconds, the numbered grid doesn't fit. Set phoneMode: 'follow' in your config: the visitor just picks a language, and their phone auto-plays whatever clip is live and stops when it ends — no number-pressing. Your kiosk still only calls guide.play(videoId, el) when a clip starts and guide.stop() when it ends; the phone follows. Audio for every clip is prefetched so playback starts instantly.

Standalone vs embedded

Both modes use the same colocated contentDir — the only difference is who plays the video:

• Standalone — put the .mp4 in each video folder and open the guide's own screen page (/display?video=<id>). This repo's example/ + asset/ is exactly this; run it with npm start.
• Embedded — omit the .mp4, keep only each video's audio/ folder, and let your host app play the video + report via reporter.js (above).

Configuration reference

createAudioGuide(config) / your guide.config.js — all optional except contentDir:

| Option | Default | Meaning | |----------------------|----------------------|---------| | contentDir | — | Folder with one subfolder per video (each holding the video file + an audio/ folder). The one you configure. | | museumName | 'Museum Audio Guide' | Title shown on the phone header. | | port | 3000 | Port the service listens on. | | phoneMode | 'grid' | 'grid' (visitor presses a number) or 'follow' (visitor picks a language; the phone auto-plays whatever clip is live — best for pose/RFID-triggered single screens). | | languages | English + Hindi | Array of { code, label, nativeLabel, file, flag }. | | videos | auto | Optional [{ id, number, title }] to set grid order/labels (title shows on the phone). | | hostIp | auto-detected | LAN IP for the QR (override for VPN/virtual adapters). | | wifi | off | { ssid, password, security, hidden } for a Wi-Fi auto-join QR. | | broadcastIntervalMs| 1000 | How often each video's timestamp is pushed. | | reanchorThresholdS | 0.5 | Re-lock a clock if its display diverges by more than this. | | quiet | false | Silence the console banner/logs (useful when embedding). |

Env overrides: PORT, MUSEUM_NAME, HOST_IP, CONTENT_DIR, PHONE_MODE, WIFI_SSID, WIFI_PASSWORD, WIFI_SECURITY, WIFI_HIDDEN.

Adding a language

The language system is dynamic — you edit one array (in your config, or the package default config/languages.default.js):

languages: [
  { code: 'english', label: 'English', nativeLabel: 'English',  file: 'english.mp3', flag: '🇬🇧' },
  { code: 'french',  label: 'French',  nativeLabel: 'Français', file: 'french.mp3',  flag: '🇫🇷' },
]

Then drop french.mp3 into every video's audio folder (asset/video/<videoId>/audio/french.mp3). Restart. The dropdown updates automatically; for any video missing that file, the language is hidden just for that video (and a warning is logged) rather than crashing.

| Field | Meaning | |---------------|------------------------------------------------------| | code | Unique id (lowercase, no spaces) | | label | Name in English (secondary line) | | nativeLabel | Name in the language itself (shown in the dropdown) | | file | Filename used inside every <videoId>/audio/ | | flag | Optional emoji |

The phone experience

• Header: museum name + a scrollable language dropdown.
• Numbered grid: one circle per video; live screens enabled, offline ones greyed. Tap the number of the screen in front of you.
• Player bar: play/pause, progress, current/remaining time, volume, and a live sync badge.
• Changing language swaps the track in place at the current position.

How the sync works (briefly)

The server keeps a smooth, free-running clock per video:

position(now) = ((now - anchorEpochMs) / 1000)  modulo  videoDuration

Reports from the display/reporter only re-anchor the clock when the video genuinely diverges (buffering, loop, seek). Each phone does an NTP-style handshake to learn the server clock, receives its video's sync once a second, interpolates locally, and corrects drift:

| Difference | Action | |-------------------|--------------------------------------| | < 200 ms | do nothing (in sync) | | 200 – 800 ms | nudge playback speed (1.03× / 0.97×) | | > 800 ms | hard-seek to the server position |

Everything is computed modulo the track length, so a loop is never mistaken for a large gap. Timestamps are real (derived from Date.now()), never faked. Measured drift on a LAN: single-digit milliseconds, with full isolation between videos.

Non-browser hosts (Unity / native / hardware)

No HTML <video> to observe? Report over REST instead of the reporter SDK:

POST http://GUIDE_IP:4000/report
Content-Type: application/json

{ "videoId": "neuron", "currentTime": 12.4, "duration": 205.6 }

Send it a few times a second while a video plays. Extra fields: "loop": true on a wrap, "stop": true when the video ends. Everything else (phones, sync) works identically.

API reference

createAudioGuide(config) → guide

Returns { app, server, io, config, videos, clocks, getClock, listen, stop, url, serverIp }.

• listen(cb?) → Promise — precomputes the QR, starts the loops, binds the port.
• stop() → Promise — closes the server + sockets and clears all timers (clean for tests/embedding).
• app is the Express app and io the Socket.IO server, if you need to extend them.

Reporter — window.AudioGuide (from /reporter.js)

• AudioGuide.connect({ server?, reportIntervalMs? }) → guide (server defaults to the script's origin; reportIntervalMs defaults to 500).
• guide.play(videoId, videoElement) — start / hot-switch reporting.
• guide.stop() — stop reporting.
• guide.showQr(target, { size?, showUrl? }) — render the guide QR into an element/selector.
• guide.ready() → Promise that resolves once the socket is connected.

HTTP

• GET / — phone page · GET /display?video=<id> — built-in screen · GET /host-demo — integration demo
• GET /reporter.js — the reporter SDK
• GET /timestamp?video=<id> → { videoId, timestamp, duration, serverTime, ready, loopCount }
• GET /api/config → { museumName, phoneUrl, serverIp, port, qrDataUrl, wifi, languages, videos }
• POST /report → { videoId, currentTime, duration, loop?, stop? } (REST reporting seam)
• All responses send Access-Control-Allow-Origin: * for cross-origin hosts.

Socket.IO

• Connect ?role=display (a screen/reporter) or ?role=phone (a visitor).
• sync (→ a video's room, 1 Hz): { videoId, timestamp, duration, serverTime, ready, loopCount }
• videos:status (→ all, on change): [{ videoId, number, ready }]
• phone:join (phone→server, ack): tune into a video's room; ack returns its sync.
• time:ping (client→server, ack): clock-offset handshake.
• display:tick / display:loop / display:stop (display→server): { videoId, currentTime, duration }.
• display:status (→ a video's room): { videoId, connected }.

Wi-Fi auto-join QR (optional)

Since phones must share the server's LAN, the display can show a second "Join Wi-Fi" QR that iOS 11+/Android 10+ join with one tap. Off by default — set wifi: { ssid, password } in your config (or WIFI_SSID/WIFI_PASSWORD) and render it. One QR can't both join Wi-Fi and open a URL; for a true one-step join, configure your router as a captive portal to http://SERVER_IP:PORT.

Best museum setup: a small dedicated router at the exhibit — fully offline, unlimited phones.

Troubleshooting

• A number is greyed out — that screen isn't reporting. Open/refresh its display, or check your guide.play(...) call.
• QR won't scan / page won't load — the phone isn't on the server's Wi-Fi, or the auto-detected IP is wrong (set hostIp/HOST_IP). Allow Node through the firewall on Private networks.
• A language missing for one video — its .mp3 isn't in that video's <videoId>/audio/ folder.
• Audio slightly off, then snaps — expected on a big gap (>800 ms): it hard-seeks to the authoritative position, then resumes at 1×.

Publishing / offline reuse

npm test                 # smoke test
npm publish              # public registry (npm requires 2FA: npm publish --otp=<code>)

Offline (museum machine, no registry): npm pack → copy the .tgz → npm install ./synced-audio-guide-1.2.3.tgz.

License

MIT — see LICENSE.