react-native-nitro-ar

A high-performance Nitro module that provides ARKit functionality for React Native on iOS.

Features

• AR Session Management - Start, pause, and reset AR sessions with full configuration control
• Plane Detection - Detect horizontal and vertical planes with classification (floor, wall, ceiling, table, seat, window, door)
• Raycasting - Hit-test against detected planes and estimated surfaces
• Anchors - Create and track anchors in 3D space
• Measurements - Measure distances between anchor points
• Bounding Boxes - Calculate oriented bounding boxes (OBB) using PCA
• Light Estimation - Access ambient and directional light data with spherical harmonics
• LiDAR Depth - Access scene depth and smoothed depth data (iOS 14+, LiDAR devices)
• World Maps - Save and restore AR world maps for persistent AR experiences
• Camera Data - Access camera pose, intrinsics, projection/view matrices

Requirements

• React Native 0.78.0+
• Node 18.0.0+
• iOS 13.0+ (iOS 14.0+ for LiDAR depth features)

Installation

npm install react-native-nitro-ar react-native-nitro-modules

Expo

If you're using Expo, add the plugin to your app.json:

{
  "expo": {
    "plugins": ["react-native-nitro-ar"]
  }
}

Then run prebuild:

npx expo prebuild

Bare React Native

Add camera usage description to your Info.plist:

<key>NSCameraUsageDescription</key>
<string>This app uses the camera for AR experiences</string>

Then install pods:

cd ios && pod install

Usage

Basic Session

import { createARSession } from "react-native-nitro-ar";

const session = createARSession();

// Start with default configuration
session.start();

// Or with custom configuration
session.start({
  planeDetection: ["horizontal", "vertical"],
  lightEstimation: true,
  environmentTexturing: "automatic",
  worldAlignment: "gravity",
});

// Pause/resume
session.pause();
session.start();

// Reset tracking
session.reset();

Plane Detection

// Get all detected planes
const planes = session.planeAnchors;

for (const plane of planes) {
  console.log("Plane:", plane.identifier);
  console.log("Classification:", plane.classification); // floor, wall, table, etc.
  console.log("Alignment:", plane.alignment); // horizontal or vertical
  console.log("Extent:", plane.extent); // [width, height]
  console.log("Center:", plane.center); // [x, y, z]

  // Access plane geometry for rendering
  const geo = plane.geometry;
  console.log("Vertices:", geo.vertices);
  console.log("Indices:", geo.triangleIndices);
}

// Subscribe to plane updates
const unsubscribe = session.onPlanesUpdated((added, updated, removedIds) => {
  console.log("Added planes:", added.length);
  console.log("Updated planes:", updated.length);
  console.log("Removed plane IDs:", removedIds);
});

// Later: unsubscribe()

Raycasting

// Simple raycast (normalized screen coordinates 0-1)
const hit = session.raycast(0.5, 0.5);

if (hit) {
  console.log("Hit position:", hit.position);
  console.log("Hit rotation:", hit.rotation);
  console.log("Distance:", hit.distance);

  // Create an anchor at the hit location
  const anchor = session.createAnchor(hit);
}

// Advanced raycast with options
const results = session.raycastWithQuery({
  x: 0.5,
  y: 0.5,
  target: "existingPlaneGeometry", // or 'existingPlaneInfinite', 'estimatedPlane', 'any'
  alignment: "horizontal", // or 'vertical', 'any'
});

Anchors

// Create anchor from raycast
const anchor = session.createAnchor(raycastResult);

// Create anchor at specific position
const anchor = session.createAnchorAtPosition(
  [0, 0, -1], // position [x, y, z]
  [0, 0, 0, 1], // rotation quaternion [x, y, z, w]
);

// Access anchor properties
console.log("ID:", anchor.identifier);
console.log("Position:", anchor.position);
console.log("Rotation:", anchor.rotation);
console.log("Is tracked:", anchor.isTracked);

// Remove anchor
session.removeAnchor(anchor);

// Subscribe to anchor updates
const unsubscribe = session.onAnchorsUpdated((added, updated, removedIds) => {
  // Handle anchor changes
});

Measurements

// Create measurement between two anchors
const measurement = session.createMeasurement(startAnchor, endAnchor);

console.log("Length (meters):", measurement.length);
console.log("Is valid:", measurement.isValid);

Frame Updates

// Subscribe to frame updates
const unsubscribe = session.onFrameUpdate((frame) => {
  console.log("Timestamp:", frame.timestamp);
  console.log("Camera position:", frame.cameraPosition);
  console.log("Camera rotation:", frame.cameraRotation);

  // Light estimation
  if (frame.lightEstimate) {
    console.log("Ambient intensity:", frame.lightEstimate.ambientIntensity);
    console.log("Color temperature:", frame.lightEstimate.ambientColorTemperature);
  }

  // Directional light (for realistic lighting)
  if (frame.directionalLightEstimate) {
    console.log("Light direction:", frame.directionalLightEstimate.primaryLightDirection);
    console.log(
      "Spherical harmonics:",
      frame.directionalLightEstimate.sphericalHarmonicsCoefficients,
    );
  }

  // LiDAR depth (iOS 14+, devices with LiDAR)
  if (frame.sceneDepth) {
    const depth = frame.sceneDepth;
    console.log("Depth map size:", depth.width, "x", depth.height);

    // Get depth at specific point
    const depthValue = depth.getDepthAt(0.5, 0.5);
    const confidence = depth.getConfidenceAt(0.5, 0.5);
  }

  // Capture camera image
  const base64Image = frame.getCapturedImage(0.8); // quality 0-1
});

Tracking State

// Check current state
console.log("Is running:", session.isRunning);
console.log("Tracking state:", session.trackingState); // 'normal', 'limited', 'notAvailable'
console.log("Tracking reason:", session.trackingStateReason); // 'none', 'initializing', 'excessiveMotion', etc.
console.log("World mapping:", session.worldMappingStatus); // 'notAvailable', 'limited', 'extending', 'mapped'

// Subscribe to tracking changes
const unsubscribe = session.onTrackingStateChanged((state, reason) => {
  if (state === "limited") {
    console.log("Tracking limited:", reason);
  }
});

World Map Persistence

// Save world map (for relocalization later)
const worldMap = await session.getCurrentWorldMap();
const mapData = worldMap.getData(); // base64 encoded

// Store mapData somewhere (file, cloud, etc.)
await saveToStorage(mapData);

// Later: restore session with saved map
const savedMapData = await loadFromStorage();
session.start({
  initialWorldMap: savedMapData,
});

Bounding Box Builder

import { createARBoundingBoxBuilder } from "react-native-nitro-ar";

const builder = createARBoundingBoxBuilder();

// Add points (from raycasts, anchors, etc.)
builder.addPoint([x, y, z]);
builder.addPoints([
  [x1, y1, z1],
  [x2, y2, z2],
  [x3, y3, z3],
]);

// Get oriented bounding box (uses PCA for optimal orientation)
const obb = builder.getOrientedBoundingBox();
console.log("Center:", obb.center);
console.log("Half extents:", obb.halfExtents);
console.log("Axes:", obb.axes); // 3 orthogonal direction vectors

// Reset for new calculation
builder.clear();

API Reference

ARSession

| Property/Method | Type | Description | | ----------------------------------- | --------------------- | --------------------------------- | | start(config?) | void | Start AR session | | pause() | void | Pause AR session | | reset() | void | Reset tracking and remove anchors | | isRunning | boolean | Session running state | | trackingState | TrackingState | Current tracking quality | | trackingStateReason | TrackingStateReason | Reason for limited tracking | | worldMappingStatus | WorldMappingStatus | World map quality | | getCameraPose() | CameraPose | Current camera position/rotation | | currentFrame | ARFrame? | Latest frame data | | raycast(x, y) | ARRaycastResult? | Hit-test at screen point | | raycastWithQuery(query) | ARRaycastResult[] | Advanced hit-test | | createAnchor(hit) | ARAnchor | Create anchor from raycast | | createAnchorAtPosition(pos, rot?) | ARAnchor | Create anchor at position | | removeAnchor(anchor) | void | Remove anchor | | anchors | ARAnchor[] | All anchors | | planeAnchors | ARPlaneAnchor[] | Detected planes | | createMeasurement(start, end) | ARMeasurement | Measure between anchors | | getCurrentWorldMap() | Promise<ARWorldMap> | Get world map for persistence | | onFrameUpdate(callback) | () => void | Subscribe to frame updates | | onTrackingStateChanged(callback) | () => void | Subscribe to tracking changes | | onAnchorsUpdated(callback) | () => void | Subscribe to anchor changes | | onPlanesUpdated(callback) | () => void | Subscribe to plane changes |

ARSessionConfiguration

| Property | Type | Default | Description | | ---------------------- | ----------------------- | ---------------------------- | --------------------------- | | planeDetection | PlaneDetectionMode[]? | ['horizontal', 'vertical'] | Planes to detect | | lightEstimation | boolean? | true | Enable light estimation | | environmentTexturing | EnvironmentTexturing? | - | Environment texturing mode | | worldAlignment | WorldAlignment? | 'gravity' | World coordinate alignment | | sceneDepth | boolean? | false | Enable LiDAR depth | | smoothedSceneDepth | boolean? | false | Enable smoothed depth | | initialWorldMap | string? | - | Base64 world map to restore |

Development

# Install dependencies
bun install

# Run codegen (generates Nitro bridge code)
bun run codegen

# Type check and build
bun run build

To run the example app:

# from root
bun link
cd example
bun install
bun run ios --device

Physical devices are required.

License

MIT