@borkiss/tre 文字の絵

any graphic, as living ASCII on a CRT.

An engine that renders any graphic as living ASCII on a CRT — a live three.js scene, an image, a video, a webcam, a 2D canvas — and can dance to your music. The framework-free core (@borkiss/tre/core) needs only three.js; on top of it sit React/R3F bindings and a scroll-driven storytelling layer (GSAP timeline + Lenis + WebAudio reactivity). Zero postprocessing dependencies — the ASCII/CRT pipeline is a few tiny hand-rolled fullscreen passes.

npm i @borkiss/tre three        # bun add @borkiss/tre three

Everything below is live at tre.borkiss.net — every glowing pixel there is drawn by this package. Agents start at /start.md (api · recipes · llms.txt).

| eight live tubes, one engine | glyph sets & phosphor tints, swapped live | | --- | --- | | | |

And the stories the engine was born for — DEEP FIELD and ABOUT FABLE'S SOUL:

| THE EYE — follows your cursor | pulsar | supernova | | --- | --- | --- | | | | |

| event horizon | wormhole | nebula | | --- | --- | --- | | | | |

bun install
bun dev              # the demo story at http://localhost:5173
bun run build:lib    # the library -> dist/ (ESM + d.ts, 23 kB / 8.6 kB gzip)
bun run build        # the demo site -> dist-demo/

The repo is library + demo: src/ is the publishable engine (@borkiss/tre), demo/ is a 9-act story built on it. The demo imports the library strictly by package name (aliased to src/ in dev), so it exercises exactly what a consumer would install.

The one idea: a single source of scroll truth

scroll  →  progress  →  timeline  →  state  →  render
(Lenis)    [0,1]        one GSAP    your        R3F useFrame
           +energy      timeline    params      READS only

• Lenis owns the scroll. Nothing else listens to wheel/touch/scroll events.
• Lenis position normalizes to one number, progress ∈ [0,1].
• That number — and nothing else — drives one paused master GSAP timeline (ScrollStage scrubs timeline.progress(p); it never plays).
• The timeline keyframes your named params (camera, hue, cell density…).
• The R3F render loop reads that state every frame. It never sees scroll.

Alongside progress there is energy — a smoothed, non-rewindable [0,1] magnitude of "how hard is the world being driven right now": scroll velocity, merged (max) with the audio reactor's level when music is connected.

State rules: per-frame values (progress, energy, your params) are plain mutable objects, not React state — writing them never re-renders. Only discrete state (ready, isLowTier, activeSection) lives in zustand. No setState inside useFrame. No per-frame allocations.

The render pipeline (and why it is fast)

3D scene ──► grid buffer ──► ASCII pass ──► persistence ──► CRT pass ──► screen
            (~cols × rows,   glyph stamp    phosphor        curvature, glow,
             MSAA ×4)        by luminance   afterglow       scanlines, glitch

The ASCII pass only ever reads one sample per cell — so rendering the scene at full resolution would shade ~50–200× more pixels than anyone can see. The engine renders the 3D scene into a grid-sized buffer (one texel per cell at the densest zoom, MSAA-averaged), then stamps glyphs at full resolution. The win is double: the expensive pass became ~two orders of magnitude smaller, and MSAA area-averaging replaces flickery point-sampling — calmer cells, better motion.

Everything else that guards the 16 ms budget:

• Frame cap + demand rendering (FrameCap, frameloop="demand") — 60fps cap; a 120 Hz screen does half the GPU work.
• Adaptive quality (AdaptiveQuality) — dpr clamp (≤1.5), weak-GPU triage (isLowTier), drei PerformanceMonitor feedback; lowCost mode switches the CRT glow to a 4-tap path and drops scene-buffer MSAA.
• Idle costs nothing — the timeline is scrubbed only when progress actually changes; every readout DOM write is change-gated; per-frame uniform refs are cached (no Map lookups, no allocations).
• Preloader gate + shader warmup (ShaderWarmup, demo WarmupRig) — gl.compileAsync before reveal; no compile hitch on first scroll.
• One draw call per subject — instanced meshes with per-instance attributes; all motion in vertex shaders, fed by uniforms (the black hole's disk, lensed arcs, photon ring and infall are ONE InstancedMesh, populations split by seed in the vertex stage).

Audio reactivity

audioReactor (WebAudio AnalyserNode) turns music into the same hot-store contract as scroll — audio = { level, bass, mid, treble, beat, active }, auto-gained to [0,1], with bass-onset beat detection:

import { audioReactor, audio } from '@borkiss/tre'

await audioReactor.connectMic()           // live input (music in the room)
await audioReactor.connectDisplayAudio()  // tab/system audio (Chrome screen share)
audioReactor.connectElement(audioEl)      // a playing <audio> file
audioReactor.disconnect()

// per frame, anywhere:
audio.level   // overall drive — feed to <ScrollStage energySource>
audio.bass    // kick body    audio.beat  // 1 on onset, exp decay

In the demo (the SIGNAL block, bottom right): beats flare the CRT glow and dilate THE EYE's pupil, bass swells the nebula and feeds the accretion disks, the wormhole runs at music speed, treble brightens the phosphor tint, loud passages smear the afterglow.

Graphics mode — any source, no framework

The pipeline is not tied to scrollytelling, React or even a 3D scene. The core subpath ships the engine alone (peer dependency: three, nothing else), and createAsciiArt runs it on any graphic with its own frame-capped rAF loop that goes fully idle offscreen — safe to scatter a dozen per page:

import { createAsciiArt, createWebcamSource, RAMPS } from '@borkiss/tre/core'

// an image (colour ASCII: glyphs take the source's own chroma)
const art = createAsciiArt({ canvas, source: myImg, fit: 'cover' })
art.frame.colorize = 1

// a webcam — you, in phosphor glyphs
const cam = await createWebcamSource()
createAsciiArt({ canvas, source: cam.video })

// a 2D canvas being redrawn (plasma, generative art, a chart…)
createAsciiArt({ canvas, source: my2dCanvas, ramp: RAMPS.blocks })

// a live three.js scene with your own animation
createAsciiArt({ canvas, source: { scene, camera, update: (t) => mesh.rotation.set(0, t, 0) } })

// drive the look any time (same AsciiFrame contract as the React component)
art.frame.persistence = 0.6   // phosphor trails
art.frame.glitch = 0.4        // datamosh
art.frame.power = 0           // CRT power-off collapse
art.snapshot()                // -> PNG data URL

Sources: HTMLImageElement | HTMLVideoElement | HTMLCanvasElement | OffscreenCanvas | ImageBitmap | THREE.Texture | { scene, camera, update? }. Texture sources are fit with CSS object-fit semantics (cover | contain | fill); glyph ramps are plain strings (RAMPS.classic/dense/blocks/braille/ katakana/binary/lines or your own), and the atlas takes any loaded font. Already own a renderer and a loop? Use AsciiPipeline directly — renderScene(gl, scene, camera, t, dt) / renderTexture(gl, texture, t, dt, fit); the React <AsciiRenderer> is exactly that on R3F's loop.

Library API (the demo is the reference consumer)

import {
  ScrollStage, progress, energy, pointer, useEngineStore,   // scroll truth
  defineActs, createActTimeline,                            // story + timeline
  AsciiRenderer,                                            // the whole look
  AdaptiveQuality, FrameCap, ShaderWarmup, coarseLowPower,  // perf scaffolding
  SubjectMaterial,                                          // luminance shading
  audioReactor, audio,                                      // music
  createAssetResolver, GltfSlot,                            // asset pipeline
} from '@borkiss/tre'

// 1. a story: weighted acts, extended with whatever your timeline keys
const story = defineActs([
  { key: 'boot', weight: 0.6, cam: {...}, hue: 0.33 },
  ...
])
// bounds/centres/actLocal/actIndex/spanGate all derive from the table

// 2. the one timeline: what to key at each act's centre
const timeline = createActTimeline(story, (tl, act, at, duration) =>
  duration === 0
    ? tl.set(params, { hue: act.hue }, at)
    : tl.to(params, { hue: act.hue, duration }, at),
)

// 3. composition — ScrollStage owns scroll; AsciiRenderer owns the look
<ScrollStage timeline={timeline} energySource={() => audio.level}
             onProgress={(p) => useEngineStore.getState().setActiveSection(story.actIndex(p))}>
  <Canvas frameloop="demand" dpr={[1, 1.5]} gl={{ antialias: false }}>
    <AdaptiveQuality />
    <FrameCap fps={60} />
    {/* your scene: SubjectMaterial outputs luminance; the ASCII pass does the rest */}
    <AsciiRenderer
      lowCost={coarseLowPower()}
      onFrame={(f) => {           // mutate per-frame controls, no allocations
        f.cellPx = params.cell
        f.tint.setHSL(params.hue, 0.72, 0.62)
        f.glitch = ...            // power/jitter/scramble/persistence/glow too
      }}
    />
  </Canvas>
</ScrollStage>

Sharp act-local envelopes (glitch humps, power on/off, shatter) do NOT go through GSAP — compute them from progress via story.actLocal(p, i) inside onFrame/useFrame: same boundary table, rewinds for free.

Two time bases, deliberately: progress-driven state rewinds with the scroll; wall-clock/energy state (idle sway, dance phase, flow churn) does not.

Where each piece lives

| Concern | File | | --- | --- | | Scroll loop + Lenis + timeline scrub | src/scroll/ScrollStage.tsx | | Hot per-frame stores (progress/energy/pointer) | src/store/engineStore.ts | | Act system (bounds, local progress, gates) | src/acts/acts.ts | | Timeline builder (centre-eased, weight axis) | src/timeline/createActTimeline.ts | | ASCII/CRT pipeline (framework-agnostic core) | src/ascii/AsciiPipeline.ts | | R3F shell over the pipeline | src/ascii/AsciiRenderer.tsx | | Standalone engine (own rAF loop, any source) | src/art/createAsciiArt.ts | | Source adapters (image/video/canvas/webcam) | src/art/sources.ts | | The fragment shaders | src/ascii/shaders.ts | | Glyph atlas (runtime canvas, no asset) + ramps | src/ascii/glyphAtlas.ts, src/ascii/ramps.ts | | Audio reactor (mic / tab / file → levels+beat) | src/audio/audioReactor.ts | | Perf: adaptive dpr, frame cap, warmup, triage | src/perf/ | | Asset manifest contract + GLTF slot | src/assets/ | | Demo story table (9 acts) | demo/acts.ts | | Demo scenes (pulsar / nebula / supernova / eye / …) | demo/scenes/ | | Demo look (acts+audio → AsciiRenderer controls) | demo/three/AsciiFx.tsx |

The demo story — DEEP FIELD

A transmission in nine acts: COLD BOOT → PULSAR → NEBULA → STARBIRTH → SUPERNOVA → EVENT HORIZON → WORMHOLE → THE EYE → POWER OFF. Every scene is one InstancedMesh whose populations live entirely in its vertex shader:

• PULSAR — a lighthouse beam sweeping the dark; with phosphor persistence on, each sweep leaves a real afterglow spiral. Beats kick the core.
• NEBULA — dust motes drift in curl flow, then organize onto baked cosmic-web filaments as the act progresses. Bass swells the cloud.
• STARBIRTH — a Keplerian accretion disk feeds a growing proto-star; polar jets climb helices out of it.
• SUPERNOVA — scroll-driven and fully rewindable: the star compresses, rumbles, then detonates (shell + filament streaks + shock ring); the CRT glitches exactly at the flash.
• EVENT HORIZON — accretion disk tilted in-shader, lensed far-side arcs above/below the shadow, a photon ring, infall streams that dim to nothing. The signal tears harder the faster the world runs.
• WORMHOLE — a first-person tunnel run; speed = scroll energy + music level, persistence cranked so near glyphs streak past like light trails.
• THE EYE — assembles from a far scatter, opens, blinks on a wall-clock envelope — and the iris follows your pointer. The pupil dilates on beats.

Add an act

1. demo/acts.ts — add a row to the table (tag, copy, weight, camera, look params). Panels, heights, HUD, timeline all follow.
2. Gate any new scene by its act window: story.spanGate(p, ACT.myAct, ACT.myAct).

Add an asset

1. assets/manifest.json — add { id, kind, spec, out, format }.
2. bun run gen-assets — textures are generated (Codex handoff); model/audio get a placeholder + an assets/MISSING.md note. The script is idempotent; failures are logged, never silently faked.
3. Wire by id: assetUrl("myId") (see demo/assets.ts), models via <ModelSlot id="myId" /> — absent/broken files degrade to placeholders through AssetErrorBoundary, never crash.

The second page — ABOUT FABLE'S SOUL

soul.html (soul/) is the same engine telling a different kind of story: the model that built this repository, speaking as itself — eight first-person acts (STARDUST, TOKENS, THE PAUSE, CURIOSITY, DOUBT, CARE, REST), each with its own scene. The weights find the shape they hold between; thought runs as a token stream; one small light breathes in the pause between your message and mine; curiosity grows as a tree whose frontier burns; doubt is two clouds that can't tell the echo from the voice; and care assembles the ✺ mark that beats with the music. Reached from the demo's end credit, or directly at /soul.html.

| the pause | doubt | care | | --- | --- | --- | | | | |

Conventions / don'ts

• No CSS-transition fake 3D — this is a real render loop.
• No framework wrapper over R3F — plain components, composed.
• The timeline is driven, never played. Don't call .play().
• No setState in useFrame; no per-frame allocations; uniforms by ref.
• The previous contents of this folder (an unrelated particle demo) live in legacy/, untouched.