ARSY WebAR — Open SDK of the WebAR CV Lib

Official WebAR.Studio SDK for browser-based AR and VR: image, QR, face and body tracking, 360, world tracking with SLAM, plane placement, anchors, visual positioning, and VR that opens on every device, from a headset to an iPhone. 🚀

🔗 Samples repo (ready-to-run demos): https://github.com/WebAR-Studio/open-webar-sdk

🆕 New in 1.0.26

• 🥽 VR on every device (mode: 'VR'). Headsets get an immersive WebXR session with stereo, controllers and hands. Phones and desktops without one get the magic window: the view turns with the phone, and a Cardboard viewer gets stereo.
• 📍 VR triggers (TRIGGER_MODE_VR). Place content in front of the user with offset and scale. It arrives as EVENT_DETECTED / EVENT_POSE / EVENT_LOST, like every other trigger.
• 🕺 Body pose tracking (mode: 'POSE'). 33 body landmarks, plus head, chest, hips, hands and feet as anchors, with metric positions and the full skeleton on every event.
• 📷 Fix: rear-camera modes no longer open the front camera on iPhones set to a language other than English.

✨ Why try it

• ✅ Real CV tracking, not a mock
• ⚡ Fast setup without boilerplate
• 🎯 Proof-of-concepts in minutes
• 🌐 Browser-first AR and VR (no app installs)
• 🔁 One SLAM API across several tracking engines, chosen at runtime
• 🥽 One VR project for headsets, phones and Cardboard viewers

🧭 Who this is for

• 🧑‍💻 Web developers building immersive sites
• 🎨 Marketing and creative teams running AR campaigns
• 🛍️ E-commerce teams prototyping try-on or product previews
• 🏋️ Fitness and body-driven experiences
• 🎓 Educators and students learning AR and VR
• 🚀 Startups validating ideas quickly

📦 Install

npm install @web-ar-studio/webar-engine-sdk

Serve your page over HTTPS: browsers expose the camera, motion sensors and WebXR only in a secure context (localhost counts as secure).

init() needs an apiKey. Each template in the samples repo ships a public test key in .env.example. To get your own key, write to us: the contacts are at the end of this page.

🧩 Tracking modes at a glance

| mode | Marker needed | Best for | Requirements | | ------------ | ------------- | ------------------------ | ---------------------------- | | IMAGE | ✅ | posters, packaging | WebGL / WebRTC | | QR | ✅ | QR-triggered AR | any camera | | MIX | ✅ | image and QR together | WebGL / WebRTC | | FACE | ❌ | filters, try-ons | front camera | | POSE | ❌ | body effects, fitness | any camera, rear recommended | | SLAM_3DOF | ❌ | 360 tours, panoramas | device orientation | | SLAM | ❌ | placement, anchors, VPS | see providers below | | SLAM_WEBXR | ❌ | plane placement (legacy) | WebXR + HTTPS | | VR | ❌ | virtual tours, showrooms | any device, see VR below |

SLAM_WEBXR still works exactly as before; new projects should use SLAM and pick the engine with provider, which is what that mode does internally.

🚀 Quick start

import * as THREE from "three";
import WAS, {
  PROJECT_MODE_SLAM,
  TRIGGER_MODE_SLAM,
  CAMERA_MODE_ENVIRONMENT,
  EVENT_DETECTED,
  EVENT_POSE,
  EVENT_LOST,
  EVENT_FRAME,
  EVENT_XR_SELECT,
} from "@web-ar-studio/webar-engine-sdk";

const was = new WAS();

const { canvas, context } = await was.init({
  apiKey: "your-key",
  mode: PROJECT_MODE_SLAM,
  cameraMode: CAMERA_MODE_ENVIRONMENT,
  container: document.querySelector("#ar"),
  triggers: [{ id: 1, mode: TRIGGER_MODE_SLAM, source: null }],
});

// The SDK draws into its own canvas; render there with any engine.
const renderer = new THREE.WebGLRenderer({ canvas, context });

// A surface came under the crosshair — show your reticle and follow it.
was.on(EVENT_DETECTED, ([hit]) => {
  reticle.visible = true;
  place(hit);
});
was.on(EVENT_POSE, ([hit]) => place(hit));
was.on(EVENT_LOST, () => {
  reticle.visible = false;
});

// Tap: pin content where the reticle is aiming.
was.on(EVENT_XR_SELECT, async () => {
  const result = await was.createAnchor({ center: [x, y, z] });
  if (!result.ok) console.log("not anchored:", result.reason);
});

// Start from a tap: WebXR and iOS motion sensors need a user gesture.
// In SLAM a second on(EVENT_FRAME) call stops the session.
startButton.onclick = () =>
  was.on(EVENT_FRAME, (deltaTime) => renderer.render(scene, camera));

The SDK draws nothing on its own. It reports coordinates, and you render them with your own engine (three.js, Babylon, PlayCanvas, raw WebGL). on(EVENT_FRAME, …) is what starts tracking: register your render loop there.

🔁 Providers: one SLAM API, several engines

mode: 'SLAM' does not hard-code a tracking engine. It picks one at runtime:

| provider | Engine | Notes | | ------------------ | ----------------------- | ------------------------------------------- | | 'auto' (default) | first supported | tries webxr, then 8thwall | | 'webxr' | browser WebXR | Android Chrome; native anchors and planes | | '8thwall' | 8th Wall XR Engine | works where WebXR does not, e.g. iOS Safari | | 'arclip-webar' | ARClip WebAssembly SLAM | opt-in only, never auto-selected |

await was.init({
  /* ... */
  provider: "auto",
  providerFallback: { ios: ["8thwall"], android: ["webxr", "8thwall"] },
  providerOptions: { eighthWall: { worldPoints: true } },
});

Each candidate is checked against the device before it is used, so an engine that cannot run is skipped rather than started and left broken. If none can run, the error names every candidate tried and why each was passed over.

providerFallback accepts a flat array or a per-platform map. It is optional: the default chain covers the common case, since WebXR wins where it exists and 8th Wall picks up where it does not.

📍 VPS — visual positioning

VPS is a capability of a SLAM session, not a separate mode, so it works the same on every provider that supports it:

await was.init({
  /* ... */
  vps: { locationIds: ["your-location"] },
});

was.on(EVENT_VPS_LOCALIZED, (data) => {
  // data.offset is ready to apply — position and orientation
});
was.on(EVENT_VPS_NOT_LOCALIZED, ({ status, statusDescription }) => {
  /* ... */
});
was.on(EVENT_VPS_ERROR, (message) => {
  /* ... */
});

Every field of vps is optional — vps: {} localizes globally against the default server, authenticated with the same apiKey you passed to init. Localization is accepted only when the server's match clears a quality bar and implies a plausible movement, because visual relocalization returns confident-looking wrong answers where architecture repeats.

Runtime control: setupVPS, startVPS, stopVPS, isVPSActive, setVPSLocationIds, updateVPSSession.

📌 Anchors

const result = await was.createAnchor({ center: [x, y, z] });
const pose = was.getAnchorPose(); // call inside your render loop
await was.clearAnchor();

A refusal is a normal outcome, not an error: { ok: false, reason } tells you why — usually too few mapped points under the ray. On webxr anchors are native; on 8thwall they are solved from the engine's point cloud, which requires providerOptions.eighthWall.worldPoints.

Read getAnchorPose() inside the render callback. On WebXR the pose comes from the live frame, and asking later answers null rather than a stale position.

🕺 Body pose tracking (POSE)

Tracks one person. Each trigger follows one point of the body.

import * as THREE from "three";
import WAS, {
  PROJECT_MODE_POSE,
  TRIGGER_MODE_POSE,
  CAMERA_MODE_ENVIRONMENT,
  ANCHOR_TYPE_POSE_CHEST,
  ANCHOR_TYPE_POSE_RIGHT_HAND,
  EVENT_DETECTED,
  EVENT_POSE,
  EVENT_LOST,
  EVENT_FRAME,
} from "@web-ar-studio/webar-engine-sdk";

const was = new WAS();

const { canvas, context } = await was.init({
  apiKey: "your-key",
  mode: PROJECT_MODE_POSE,
  cameraMode: CAMERA_MODE_ENVIRONMENT,
  container: document.querySelector("#ar"),
  fov: 60, // the camera's real vertical field of view, in degrees
  triggers: [
    {
      id: 1,
      mode: TRIGGER_MODE_POSE,
      source: null,
      anchor: ANCHOR_TYPE_POSE_CHEST,
    },
    {
      id: 2,
      mode: TRIGGER_MODE_POSE,
      source: null,
      anchor: ANCHOR_TYPE_POSE_RIGHT_HAND,
    },
  ],
});

const renderer = new THREE.WebGLRenderer({ canvas, context });

was.on(EVENT_DETECTED, (anchors) => anchors.forEach(show));
was.on(EVENT_POSE, (anchors) => anchors.forEach(place)); // every frame
was.on(EVENT_LOST, (anchors) => anchors.forEach(hide));

// Detection runs inside the frame loop, just before your callback.
was.on(EVENT_FRAME, (deltaTime) => renderer.render(scene, camera));

Anchors. You can use the 33 raw landmarks (ANCHOR_TYPE_POSE_NOSE … ANCHOR_TYPE_POSE_RIGHT_FOOT_INDEX) or seven composites: ANCHOR_TYPE_POSE_HEAD, _CHEST, _HIPS, _LEFT_HAND, _RIGHT_HAND, _LEFT_FOOT and _RIGHT_FOOT. anchor on the config is the default for triggers that set none; the SDK's default is the chest. Left and right are the person's own: on someone facing the camera, RIGHT_HAND appears on the left of the image.

Every event carries, per trigger:

• positionVector: camera-space metres, with the camera looking down −Z.
• rotationQuaternion.
• scaleVector: the body's size against an average adult.
• landmarksData: the whole skeleton aligned with the video. Draw bones with was.getPoseConnections().
• worldLandmarksData: the hip-centred metric skeleton, for driving a rig.
• visibilityData: per-landmark trust from 0 to 1.

fov matters. The camera distance is solved from the body's proportions and scales with fov. A wrong value keeps content in the right place on screen but at the wrong depth. Set it to the real vertical field of view of the camera you open.

| Option | Default | What it does | | ---------------------------------------------------------------------------------- | -------- | -------------------------------------------------------------- | | poseModel | 'lite' | 'lite' (5.8 MB), 'full' (9.4 MB), 'heavy' (30.7 MB) | | minLandmarkVisibility | 0.5 | an anchor resolves only when its landmarks clear this | | smoothing | true | one-euro smoothing of position, depth, scale, rotation | | detectionIntervalMs | 0 | minimum time between detections; 0 = every new frame | | minPoseDetectionConfidence, minPosePresenceConfidence, minTrackingConfidence | 0.5 | detection, presence and tracking confidence thresholds, 0 to 1 |

init() validates triggers, anchors and fov before it downloads the model, and rejects with a readable error. A detector failure after start is reported once, on EVENT_XR_ERROR. CAMERA_MODE_USER works too, and the SDK handles the mirroring, but a whole body rarely fits in a selfie camera.

🥽 VR — one project, every device

A VR project opens everywhere. The SDK picks the engine:

| Engine (getActiveProviderId()) | Where | What you get | | -------------------------------- | ------------------------------------------------------------------ | --------------------------------------------------------------------------------------------------- | | 'webxr-vr' | headsets with a WebXR browser, such as Meta Quest | immersive session: stereo, head position and rotation, controllers, hands, haptics | | 'magic-window' | every iOS browser, most Android phones, desktops without a headset | the page canvas turns with the phone (rotation only), drag to look, tap to select, Cardboard stereo |

import WAS, {
  PROJECT_MODE_VR,
  TRIGGER_MODE_VR,
  CAMERA_MODE_ENVIRONMENT,
  PROVIDER_MAGIC_WINDOW,
  EVENT_FRAME,
  EVENT_XR_SESSION,
  EVENT_DETECTED,
} from "@web-ar-studio/webar-engine-sdk";

const was = new WAS();

const { canvas, context } = await was.init({
  apiKey: "your-key",
  mode: PROJECT_MODE_VR,
  cameraMode: CAMERA_MODE_ENVIRONMENT, // required by the type; VR opens no camera
  container: document.querySelector("#vr"),
  triggers: [
    // 1.2 m in front of the user, on the floor, life size
    {
      id: 1,
      mode: TRIGGER_MODE_VR,
      source: null,
      offset: { x: 0, y: 0, z: -1.2 },
      scale: 1,
    },
  ],
});

const isMagicWindow = was.getActiveProviderId() === PROVIDER_MAGIC_WINDOW;

was.on(EVENT_DETECTED, ([trigger]) => placeContent(trigger));
was.on(EVENT_XR_SESSION, (isActive) => toggleVrUi(isActive));

// In VR this only REGISTERS the render callback…
await was.on(EVENT_FRAME, renderVr); // see "Rendering with three.js" below

// …and this opens the session. It must run inside a tap or click.
enterButton.onclick = () => was.requestVRSession();
exitButton.onclick = () => was.endVRSession();

• requestVRSession() has to be called from a user gesture, after the frame callback is registered. On iPhone the same tap asks for motion-sensor permission; if it is refused, drag-to-look still works. Drive your in-VR UI from EVENT_XR_SESSION(true / false). The promise also resolves when the start is cancelled.
• VR triggers are placements: nothing is detected in VR. When the session starts, the anchor point is on the floor under the viewer, turned to face where they were looking. Each trigger sits at its offset (metres, in that frame) and scale. Triggers are optional in VR.
• Cardboard: start split-screen with providerOptions.magicWindow.stereo, or switch at any time with was.setMagicWindowStereo(true).
• getIsSupportVRSession() answers for a real WebXR headset session only, and says false on an iPhone, where the project still opens on the magic window. Use it to gate headset-only features, not the Enter button.

Rendering with three.js

Render once per view into the framebuffer the SDK hands you. On the magic window getXRFramebuffer() is null, and the same code draws into the page canvas.

const renderer = new THREE.WebGLRenderer({ canvas, context });
camera.matrixAutoUpdate = false; // the SDK hands out complete matrices
let xrTarget = null;

function renderVr(deltaTime) {
  simulate(deltaTime / 1000); // once per frame, not once per eye

  const views = was.getViews(); // valid only inside this callback
  if (!views) return;

  const { framebufferWidth: w, framebufferHeight: h } = views;
  if (!xrTarget || xrTarget.width !== w || xrTarget.height !== h) {
    xrTarget = new THREE.WebGLRenderTarget(w, h, {
      colorSpace: renderer.outputColorSpace,
    });
    xrTarget.isXRRenderTarget = true; // as three's own WebXRManager does
  }
  renderer.setRenderTargetFramebuffer(xrTarget, was.getXRFramebuffer());

  for (const view of views.views) {
    const { x, y, width, height } = view.viewport; // framebuffer pixels, no DPR
    xrTarget.viewport.set(x, y, width, height);
    xrTarget.scissor.set(x, y, width, height);
    xrTarget.scissorTest = true; // otherwise the second eye's clear wipes the first
    renderer.setRenderTarget(xrTarget);

    camera.matrix.fromArray(view.transformMatrix);
    camera.updateMatrixWorld(true);
    camera.projectionMatrix.fromArray(view.projectionMatrix); // never updateProjectionMatrix()
    camera.projectionMatrixInverse.copy(camera.projectionMatrix).invert();
    renderer.render(scene, camera);
  }
}

Before the session starts and after it ends, the page is an ordinary flat app, so render your own preview with requestAnimationFrame. When you return to it, call renderer.setRenderTarget(null) and renderer.resetState().

Input

• EVENT_XR_INPUT_SOURCES carries the full set of controllers and hands on every change, as { added, removed, sources }. It is replayed to late subscribers.
• EVENT_XR_INPUT carries select / squeeze and their …start / …end, in kind.
• Per frame, was.getInputSources() gives each source's grip and ray pose, buttons, thumbstick and, with handTracking, hand joints.
• Haptics: was.pulseHapticActuator('right', 0.6, 50).
• On the magic window a tap is a select along the ray through the tapped point (in stereo, through the centre of the view).

Options

providerOptions.webxrVr (headsets):

| Option | Default | What it does | | ------------------------ | ------------- | ----------------------------------------------------------------- | | framebufferScaleFactor | 1 | render resolution; 'native' for the panel's full resolution | | antialias | true | MSAA; false is a large win on Quest | | fixedFoveation | UA default | 0..1, lower resolution at the edges of each eye | | depthNear, depthFar | 0.1, 1000 | clipping planes; your camera's near / far are not used | | targetFrameRate | UA default | request a refresh rate the device supports | | handTracking | false | request hand tracking | | boundsGeometry | false | request the play area; read it with was.getBoundsGeometry() | | fallbackFloorOffset | 1.6 | eye height in metres when the browser grants no floor-level space |

providerOptions.magicWindow (phones and desktops):

| Option | Default | What it does | | ------------ | ----------- | ------------------------------------------------- | | stereo | false | start split for a Cardboard viewer | | fov | 65 / 90 | vertical field of view, mono / stereo, in degrees | | ipd | 0.064 | distance between the eyes in stereo, metres | | eyeHeight | 1.6 | camera height above the floor, metres | | dragToLook | true | turn the view by dragging |

Also on was: getReferenceSpaceType(), getIsActiveVRSession(), and getXRVisibilityState(), which returns 'visible-blurred' while the headset's system menu is open, so you can pause simulation and audio.

📡 Events

Subscribe with was.on(EVENT_…, callback) and unsubscribe with was.un(EVENT_…). The main events:

| Event | Payload | | -------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------ | | EVENT_DETECTED / EVENT_POSE / EVENT_LOST | one entry per trigger: triggerId, positionVector, rotationQuaternion, scaleVector, plus mode-specific data | | EVENT_PROCESS | true / false: tracking has started or stopped | | EVENT_FRAME | deltaTime in ms; your render loop | | EVENT_RESIZE, EVENT_VISIBILITY | the container resized; the page was hidden or shown | | EVENT_XR_SESSION | true / false: an immersive session opened or closed | | EVENT_XR_SELECT, EVENT_XR_ROTATE…, EVENT_XR_PINCH…, EVENT_XR_PAN… | AR taps and gestures | | EVENT_XR_VIEWS, EVENT_XR_INPUT_SOURCES, EVENT_XR_INPUT | VR views, controllers and hands, controller actions | | EVENT_XR_ERROR | a provider or detector failed after start | | EVENT_VPS_READY, EVENT_VPS_LOCALIZED, EVENT_VPS_NOT_LOCALIZED, EVENT_VPS_ERROR | VPS progress |

⚠️ Errors

init() and the session methods reject with an Error whose name says what went wrong:

| name | Meaning | | ----------------- | ---------------------------------------------------------- | | DataError | API key verification failed | | DeviceError | unsupported device or project mode, or a call out of order | | HTMLError | container is missing or not an element | | VideoError | the camera stream is unavailable or permission was denied | | XrError | WebXR or immersive VR cannot start on this device | | GlError | a WebGL context could not be created | | WorkerError | the tracking worker could not be loaded | | VpsError | VPS is not available for this session | | EighthWallError | the 8th Wall engine could not be started | | EventError | unsupported event type, or a duplicate listener |

Each name is also exported as a constant (DATA_ERROR, DEVICE_ERROR, XR_ERROR, …).

🆚 SDK vs samples

| You need | Use | | ----------------------- | ------------------ | | Core CV tracking engine | webar-engine-sdk | | Ready-to-run examples | open-webar-sdk |

✅ Typed API

The library is fully typed, so you can explore available modes, events and options from your editor after installation. Config is a discriminated union on mode: fields that do not belong to the mode you chose are rejected at compile time rather than ignored at runtime.

📄 Third-party software

Sessions running on the 8thwall provider load the 8th Wall XR Engine. See THIRD-PARTY-NOTICES.md. Sessions on webxr or arclip-webar neither load nor execute it.

📮 Support and requests

For any questions about the library, or to request an API key, contact us here:

• https://t.me/was_team
• [email protected]
• https://discord.gg/4q5dbAb4GZ