capacitor-ar-scanner

AR/LiDAR camera preview, 3D scanning and real-world volume/dimension measurement for Capacitor, powered by ARKit (iOS) and ARCore (Android). Built for AI/ML capture pipelines: it renders a native camera preview behind a transparent WebView and returns precise measurements plus base64 frames you can feed straight into a model.

Features

• Native AR camera preview behind a transparent WebView (your UI composites on top).
• Real-world width / height / depth / volume from LiDAR depth (oriented bounding box).
• Live wireframe mesh overlay and scan-state events while you aim.
• Dual image capture: high-res (for AI/ML analysis) + thumbnail (for display/storage).
• Torch control.
• Self-healing preview: recovers from the long-idle cold-start white/blank screen and reports camera-health diagnostics.

Compatibility

| Plugin | Capacitor | | ------ | --------- | | 8.x | 8.x |

The plugin's major version tracks Capacitor's major version.

Installation

npm install capacitor-ar-scanner
npx cap sync

iOS

• Minimum deployment target: iOS 15.0.
• Best results require a LiDAR-capable device; non-LiDAR devices fall back to image-only capture.
• Add a camera usage description to ios/App/App/Info.plist:

<key>NSCameraUsageDescription</key>
<string>Used to scan and measure objects with the camera.</string>

The plugin uses ARKit and SceneKit. The preview is an ARSCNView inserted below the (transparent) WebView, so make sure your web UI uses a transparent background while the preview is active.

Android

• Requires an ARCore-supported device.
• Add the camera permission to android/app/src/main/AndroidManifest.xml:

<uses-permission android:name="android.permission.CAMERA" />

Note: the Android implementation currently provides image-only capture; full measurement parity with iOS LiDAR is in progress.

Usage

import { ARScanner } from 'capacitor-ar-scanner';

// 1. Check support
const support = await ARScanner.checkSupport();
if (!support.isSupported) return;

// 2. Listen for live scan events (tracking, mesh, warnings, health)
const handle = await ARScanner.addListener('scanEvent', (event) => {
  if (event.type === 'health' && event.status) {
    // e.g. report 'stalled' | 'recovered' | 'opaqueReasserted' to analytics
    console.log('camera health:', event.status, event.attempt, event.staleSeconds);
  }
  if (event.type === 'meshReady') {
    console.log('ready to capture');
  }
});

// 3. Start the preview (transparent WebView required)
await ARScanner.startPreview({ mode: 'lidar' });

// 4. Capture + measure
const result = await ARScanner.capture();
console.log(`${result.width} × ${result.height} × ${result.depth} cm, ${result.volume} cm³`);
// result.capturedImageBase64 -> send to your AI/ML model

// 5. Stop + clean up
await ARScanner.stopPreview();
await handle.remove();

API

• checkSupport()
• startPreview(...)
• stopPreview()
• capture(...)
• setTorch(...)
• addListener('scanEvent', ...)
• removeAllListeners()
• Interfaces
• Type Aliases

checkSupport()

checkSupport() => Promise<SupportResult>

Check whether AR scanning is available on the current device and what depth quality to expect (LiDAR vs world-tracking only).

Returns: Promise<SupportResult>

Since: 8.0.0

startPreview(...)

startPreview(options?: PreviewOptions | undefined) => Promise<{ started: boolean; }>

Start the native AR camera preview rendered behind a transparent WebView.

On iOS the preview is an ARSCNView inserted below the WebView; the WebView is kept transparent so your UI composites on top of the live camera.

On Android the returned promise resolves only once the CameraX use cases are actually bound (since 8.0.1), so a resolved promise means capture() is safe to call. It rejects if the camera fails to bind or permission is denied.

On iOS the camera permission is requested (first call) and checked before the AR session starts (since 8.0.2); like Android, the promise rejects with "Camera permission denied" when access is denied or restricted, instead of resolving into a silently black preview.

| Param | Type | | ------------- | --------------------------------------------------------- | | options | PreviewOptions |

Returns: Promise<{ started: boolean; }>

Since: 8.0.0

stopPreview()

stopPreview() => Promise<{ stopped: boolean; }>

Stop the AR camera preview and tear down the AR session, restoring the WebView to its opaque state. Safe to call when no preview is running.

Returns: Promise<{ stopped: boolean; }>

Since: 8.0.0

capture(...)

capture(options?: CaptureOptions | undefined) => Promise<ScanResult>

Capture the current frame and measure the object at the center of the viewfinder, returning real-world dimensions plus base64 images for any downstream AI/ML analysis.

| Param | Type | | ------------- | --------------------------------------------------------- | | options | CaptureOptions |

Returns: Promise<ScanResult>

Since: 8.0.0

setTorch(...)

setTorch(options: TorchOptions) => Promise<{ enabled: boolean; }>

Toggle the device torch (flashlight) while the preview is running.

| Param | Type | | ------------- | ----------------------------------------------------- | | options | TorchOptions |

Returns: Promise<{ enabled: boolean; }>

Since: 8.0.0

addListener('scanEvent', ...)

addListener(eventName: 'scanEvent', listener: (event: ScanEvent) => void) => Promise<PluginListenerHandle>

Listen for live scan events emitted during the preview: tracking state, mesh progress, capture warnings/errors and camera-health diagnostics.

| Param | Type | | --------------- | ------------------------------------------------------------------- | | eventName | 'scanEvent' | | listener | (event: ScanEvent) => void |

Returns: Promise<PluginListenerHandle>

Since: 8.0.0

removeAllListeners()

removeAllListeners() => Promise<void>

Remove all listeners registered by this plugin.

Since: 8.0.0

Interfaces

SupportResult

| Prop | Type | Description | Since | | ------------------ | -------------------------------------------------- | ------------------------------------------------------------------------- | ----- | | isSupported | boolean | Whether AR (world tracking) is supported at all on this device. | 8.0.0 | | hasLidar | boolean | Whether the device has a LiDAR sensor (high-accuracy depth). | 8.0.0 | | hasDepthApi | boolean | Whether a depth API is available without LiDAR (reserved for future use). | 8.0.0 | | depthQuality | 'high' | 'medium' | 'low' | 'none' | Expected depth/measurement quality on this device. | 8.0.0 |

PreviewOptions

| Prop | Type | Description | Default | Since | | ------------------- | -------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------ | ----- | | mode | 'lidar' | Preview mode. Currently only 'lidar' (depth-aware world tracking) is supported; the plugin gracefully falls back to image-only on devices without a LiDAR sensor. | | 8.0.0 | | forceLidarOff | boolean | Force LiDAR/scene-reconstruction off even on capable devices (image-only capture). Useful for debugging or to match Android behavior. | false | 8.0.0 |

ScanResult

| Prop | Type | Description | Since | | ------------------------- | ------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------ | ----- | | hasLidar | boolean | Whether the measurement used LiDAR depth (true) or was image-only (false). | 8.0.0 | | width | number | Object width in centimeters. | 8.0.0 | | height | number | Object height in centimeters. | 8.0.0 | | depth | number | Object depth in centimeters. | 8.0.0 | | volume | number | Object volume in cubic centimeters (oriented bounding box, W × H × D). | 8.0.0 | | depthQuality | 'high' | 'medium' | 'low' | 'estimated' | Quality of the depth data used for this measurement. | 8.0.0 | | pointCount | number | Number of depth points used in the measurement. | 8.0.0 | | scanMode | 'single' | 'multi-angle' | Whether this was a single capture or a multi-angle scan. | 8.0.0 | | cameraAngle | number | Angle of the camera relative to the measured surface, in degrees. | 8.0.0 | | measureMethod | 'lidar' | The measurement method used. | 8.0.0 | | capturedImageBase64 | string | High-resolution (1280px) JPEG, base64-encoded, intended for AI/ML analysis. Not persisted by the plugin. | 8.0.0 | | thumbnailBase64 | string | Thumbnail (1024px) JPEG, base64-encoded, intended for display/storage. | 8.0.0 | | lidarFallbackCode | CaptureIssueCode | iOS: set when the LiDAR measurement failed and the capture fell back to the photo alone (hasLidar: false, zero dimensions). Carries the reason code. | 8.0.4 | | barcodes | DetectedBarcode[] | The retail barcodes read on the device, present only when capture() was called with detectBarcodes: true. Empty when none was found. | 8.1.0 |

DetectedBarcode

| Prop | Type | Description | Since | | ------------ | -------------------------------------------------------------------------- | ---------------------------------------------------------------------------- | ----- | | value | string | The digits as encoded. UPC-A reads as a 13-digit EAN-13 with a leading zero. | 8.1.0 | | format | 'EAN13' | 'EAN8' | 'UPCE' | The symbology. | 8.1.0 | | box | { left: number; top: number; right: number; bottom: number; } | Where the barcode is, as fractions of the upright photo, origin top-left. | 8.1.0 |

CaptureOptions

| Prop | Type | Description | Default | Since | | -------------------- | -------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------ | ----- | | detectBarcodes | boolean | Also read EAN-13, EAN-8 and UPC-E barcodes on the device: on iOS with Apple Vision, from the photo's own frame and then a high-resolution still on iOS 16+; on Android with ML Kit, from the full-resolution photo. The photo and every other field are unchanged. | false | 8.1.0 |

TorchOptions

| Prop | Type | Description | Since | | ------------- | -------------------- | --------------------------------------------------------- | ----- | | enabled | boolean | Whether the torch should be on (true) or off (false). | 8.0.0 |

PluginListenerHandle

| Prop | Type | | ------------ | ----------------------------------------- | | remove | () => Promise<void> |

ScanEvent

| Prop | Type | Description | Since | | ------------------- | ------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----- | | type | 'error' | 'tracking' | 'meshProgress' | 'meshReady' | 'warning' | 'processing' | 'health' | The kind of event being emitted. | 8.0.0 | | trackingState | 'normal' | 'limited' | 'notAvailable' | For type: 'tracking': the current AR tracking state. | 8.0.0 | | limitedReason | 'initializing' | 'excessiveMotion' | 'insufficientFeatures' | 'relocalizing' | For type: 'tracking' with a limited state: why tracking is limited. | 8.0.0 | | meshCount | number | For type: 'meshProgress' \| 'meshReady': number of mesh anchors so far. | 8.0.0 | | vertexCount | number | For type: 'meshProgress' \| 'meshReady': total reconstructed vertices. | 8.0.0 | | isStable | boolean | For type: 'meshProgress': whether the mesh has stabilized. | 8.0.0 | | isReady | boolean | For type: 'meshProgress': whether enough mesh exists to capture. | 8.0.0 | | message | string | For type: 'warning' \| 'error': a human-readable message. | 8.0.0 | | code | CaptureIssueCode | For type: 'warning' \| 'error': a machine-readable issue code. | 8.0.0 | | angle | number | For type: 'warning' (HOLD_LEVEL): the offending surface angle in degrees. | 8.0.0 | | status | string | For type: 'processing': the capture phase. For type: 'health': one of 'stalled' \| 'recovered' \| 'sessionFailed' \| 'interrupted' \| 'interruptionEnded' \| 'noSuperview' \| 'opaqueReasserted'. | 8.0.0 | | staleSeconds | number | Health diagnostics: seconds since the last camera frame when a stall was detected. | 8.0.0 | | attempt | number | Health diagnostics: which self-heal attempt this is. | 8.0.0 |

Type Aliases

CaptureIssueCode

Codes describing why a capture could not produce a measurement.

'NO_SURFACE' | 'NOT_ENOUGH_DEPTH' | 'CANNOT_ISOLATE' | 'HOLD_LEVEL'

Changelog

See CHANGELOG.md.

License

MIT