NYHMAS

Audio, made visible. Upload any track, watch it become a living visual, and export it as video — entirely in the browser. Nothing is uploaded; your audio never leaves your device.

Built as a production-grade platform designed to scale to 100+ visual effects while staying smooth on mid-range hardware.

Highlights

• Upload → visualize → export — drag in any audio (MP3/WAV/OGG/FLAC/M4A/AAC), pick an effect, record a video.
• Fully client-side — WebAudio + WebGL + WebCodecs. No server, no cost per user, private by default.
• Deterministic engine — effects read time and audio from swappable sources, so frame-perfect export "just works" for every effect without per-effect code.
• Adaptive quality — resolution scales to hold ~60 fps on the device you're on.
• Liquid Glass UI — true-black OLED stage, frosted controls, SF-style type, one restrained accent.
• Two export tiers — real-time capture (universal) and deterministic WebCodecs render (frame-perfect MP4).

Getting started

npm install
npm run dev      # http://localhost:4321

Other scripts:

npm run build    # production build → dist/
npm run preview  # preview the build
npm run test     # unit tests (Vitest)
npm run lint     # Biome check (lint:fix to auto-fix)
npx astro check  # type-check

A demo track ships in public/demo/track.mp3 (the "Try the demo" button).

Use it as a package

The engine, effects, overlays, and export pipeline ship as the nyhmas npm package (ESM + types; three is a peer dependency):

npm install nyhmas three

import { createVisualizer } from 'nyhmas';

const viz = await createVisualizer(document.getElementById('stage')!, {
  effect: 'aura', // 'gold-particles' | 'orb' | 'galaxy' | 'aura'
});
viz.start();                      // run the render loop (idles silently)
await viz.loadTrack(file);        // File, Blob, or ArrayBuffer
await viz.play();

The container needs a CSS size and non-static positioning (the canvas mounts absolutely inside it). For custom work, the lower-level pieces are exported too: Engine, AudioEngine, Scene3DEffect / ShaderEffect / ParticleEffect base classes, OverlayLayer, the EFFECTS registry — and the full export pipeline (OfflineRenderer for deterministic WebCodecs MP4 renders, Recorder for real-time capture, offlineExportSupported for feature detection).

Deployment

The build is fully static (dist/) — any static host works (Netlify, Vercel, S3 + CDN, GitHub Pages). Asset URLs are absolute by default; when deploying under a subpath (e.g. a GitHub Pages project site), set site and base in astro.config.mjs so URLs resolve correctly.

Architecture

Two independent layers. The app shell (Astro) renders the UI once and never touches the frame loop; the engine (Three.js) owns rendering and is entirely framework-agnostic.

src/
  engine/
    Engine.ts            # owns renderer, the single rAF loop, quality, post-FX
    Renderer.ts          # WebGL2 wrapper (WebGPU-ready seam)
    Clock.ts             # SWAPPABLE time: LiveClock (playback) | FrameClock (export)
    Renderable.ts        # what the loop drives (scene + camera + update)
    smoothing.ts         # frame-rate-independent exponential smoothing
    quality/             # Capabilities + QualityManager (adaptive resolution)
    postfx/PostFX.ts     # shared composer: bloom + tonemap (OutputPass)
    audio/
      AudioEngine.ts     # WebAudio graph + play/pause/seek transport
      AudioFrame.ts      # shared per-frame payload: spectrum/waveform textures + bands
      AudioSource.ts     # interface: LiveAudioSource | OfflineAudioSource
      fft.ts             # radix-2 FFT for deterministic offline analysis
    effects/
      Effect.ts          # the effect contract + EngineContext
      Scene3DEffect.ts   # base class for 3D effects (the "3D host")
      ShaderEffect.ts    # base class for 2D fullscreen-shader effects (the "2D host")
      ParticleEffect.ts  # shared base for the point-cloud signature look
  effects/               # the library — one folder per effect (lazy-loaded)
    goldParticles/  orb/  galaxy/  aura/
  export/
    Recorder.ts          # Tier 1: captureStream + MediaRecorder
    OfflineRenderer.ts   # Tier 2: FrameClock + WebCodecs + mediabunny
    Compositor.ts        # burns overlay pixels into captured frames
  components/            # Astro UI (Liquid Glass): TopBar, Dropzone, TransportBar, …
  lib/
    registry.ts          # effect registry (dynamic imports → per-effect chunks)
    dialog.ts            # shared modal show/hide lifecycle
    AppController.ts     # wires engine + audio + DOM
  styles/tokens.css      # the design system

The engine loop

Engine runs one requestAnimationFrame loop. Each frame it: ticks the Clock, refreshes the AudioSource, calls effect.update(dt, t, audioFrame), renders through the shared PostFX chain, and feeds the frame interval to the QualityManager, which nudges render resolution to hold the frame budget.

Why "deterministic"?

Effects never call performance.now() or read a live analyser directly — time and audio arrive from the engine. That single rule is what makes high-quality export possible: for export we swap the LiveClock for a FrameClock (advances a fixed 1/fps per frame) and the LiveAudioSource for an OfflineAudioSource (computes the spectrum for any timestamp via FFT). Every effect then renders identically and reproducibly, with no dropped frames — no per-effect changes required.

All smoothing (audio bands, particle easing, logo motion) is driven by frame-time deltas, not per-frame factors (engine/smoothing.ts), so motion looks the same on 120Hz displays, on slow frames, and in the fixed-step export.

Adding an effect

Each effect is a folder under src/effects/ exporting a class, plus one entry in src/lib/registry.ts. Point-cloud effects should extend ParticleEffect — it supplies the shared palette, uniforms, reaction smoothing, idle breathing, and beat response, so a new effect is just shaders + geometry + tuning (see src/effects/orb/ for a minimal example).

3D effect — extend Scene3DEffect, implement build (create meshes) and onUpdate (react to the AudioFrame):

export class MyEffect extends Scene3DEffect {
  readonly meta = { id: 'my-effect', title: 'Mine', kind: '3d' as const };
  protected build(ctx) { /* add objects to this.scene */ }
  protected onUpdate(dt, t, audio) { /* react to audio.bands / audio.spectrum */ }
}

2D effect — extend ShaderEffect with a fragment shader; the shared audio (uSpectrum, uWaveform, uBass, uMid, uTreble, uEnergy, uBeat, uTime, uResolution) is wired automatically:

export class MyShader extends ShaderEffect {
  readonly meta = { id: 'my-shader', title: 'Mine', kind: '2d' as const };
  constructor(w: number, h: number) {
    super(w, h, /* glsl */ `... void main(){ ... }`);
    this.bloom = { strength: 1.0, radius: 0.9, threshold: 0.3, enabled: true };
  }
}

Register it with a dynamic import so it ships as its own chunk:

{
  meta: { id: 'my-effect', title: 'Mine', kind: '3d' },
  load: async () => {
    const { MyEffect } = await import('../effects/myEffect');
    return (w, h) => new MyEffect(w, h);
  },
}

Video export

• Record (Tier 1) — canvas.captureStream + the WebAudio stream → MediaRecorder. Universal; quality follows real-time performance. The captured canvas is rAF-driven, so keep the tab visible while recording (background tabs freeze the video); the app warns if you switch away.
• Fast render (Tier 2) — renders every frame off the FrameClock, encodes H.264 + AAC via WebCodecs, and muxes an MP4 with mediabunny. No dropped frames, exact A/V sync, full resolution regardless of the live adaptive scale. Offered automatically where supported; Record is the fallback.

Tech

Astro · TypeScript · Three.js · Tailwind v4 · WebAudio · WebCodecs · Vitest · Biome.

Roadmap

• Grow the effect library toward 100+ (2D shader art + 3D scenes).
• WebGPU backend via the isolated Renderer seam (WebGL2 stays the fallback).
• Per-effect parameters and presets; optional "story mode" overlays.