rubiks-vision

A framework-agnostic Rubik's cube scanner for the browser. Point a camera at a cube, hold a corner so three faces are visible, and it reads the visible stickers and assembles the cube state, delivering everything through a single onUpdate callback.

Install

npm install rubiks-vision

The ONNX models ship inside the package and are referenced with new URL('./models/...', import.meta.url), so a bundler (Vite, webpack 5, Parcel, or Rollup) emits and serves them. You do not need to host them yourself. To use your own copies instead, override the URLs via config.models (see below).

Usage

import { Camera, Overlay, HoldScanner } from "rubiks-vision";
import type { HoldSnapshot } from "rubiks-vision";

const video = document.querySelector("video")!;   // hidden; the scanner reads frames
const canvas = document.querySelector("canvas")!; // the overlay is drawn here

const camera = new Camera(video);
const overlay = new Overlay(canvas);

const scanner = new HoldScanner({
  camera,
  overlay,
  onUpdate(s: HoldSnapshot) {
    if (s.phase === "done" && s.result?.ok) {
      console.log("cube:", s.result.facelets); // 54-char URFDLB string
    }
  },
});

await scanner.start();

HoldSnapshot carries the state a HUD needs: phase ("searching" | "reading" | "locked" | "done"), progress (0..1), locked[6], lastLocked, remaining, computed, result ({ facelets, ok, message }), and netColors[54]. Call scanner.unlockFace(i) to re-open a face for re-capture.

Configuration

Pass a config object to HoldScanner to point at your own model URLs:

config: {
  models: {
    poseModelUrl: (size) => `https://cdn.example.com/cube-pose-${size}.onnx`,
    colorModelUrl: "https://cdn.example.com/cubecolor.onnx",
  },
  exposeDebugHooks: false, // set true to expose window.__hold* getters
}

The overlay is themeable. Pass theme to Overlay to override colours, timing, and geometry (clone spectraTheme and tweak it), or pass hooks with your own drawSticker / drawScannedFace / drawDetection functions to replace the default renderers; the core still does projection and hands your hook ready-to-paint canvas coordinates. Omitted hooks fall back to the theme defaults.

The lower-level CV primitives are also exported (Detector, ColorNet, deskewGrid, PoseTracker, coldPnp, project, visibleFaces, scannability, validateState, and the cube geometry helpers) if you want to build your own pipeline.

Requirements

• A secure context (HTTPS or localhost), since the camera uses getUserMedia.
• A bundler that emits new URL(asset, import.meta.url) references (Vite, webpack 5, Parcel, modern Rollup). This is how the bundled .onnx models are served.
• WebGPU is used when available, with a fallback to multi-threaded wasm. The wasm threads need a cross-origin-isolated page (COOP same-origin + COEP require-corp).

Ships ESM with .d.ts types.

License

MIT