react-native-document-scanner-ai

🚀 High-Performance Document Scanner with AI

Advanced document scanning library for React Native with ONNX Runtime + OpenCV integration for real-time document detection, scanning, and processing.

Features

🚀 High Performance

• Native C++ implementation with ONNX Runtime
• Real-time document detection and segmentation
• Optimized for mobile devices with hardware acceleration
• TurboModule + JSI architecture for native performance

🤖 AI-Powered

• Uses ONNX Runtime for ML inference
• Pre-trained YOLOv8 document segmentation models included
• Advanced edge detection and perspective correction
• Intelligent auto-capture functionality

📱 Cross-Platform

• iOS and Android support with consistent APIs
• Automatic native dependency management
• Camera integration with react-native-vision-camera

🎯 Smart Features

• Auto-capture when document is stable
• Real-time preview with quadrilateral overlay
• Multiple enhancement modes (B&W, contrast boost)
• Batch processing support
• Configurable confidence thresholds

Installation

1. Install the Package

npm install react-native-document-scanner-ai
# or
yarn add react-native-document-scanner-ai

⚠️ Important: Use npx react-native-document-scanner-ai setup commands from your app project, NOT npm run setup:windows. The npm run scripts are only for library development.

2. Automatic Setup (Recommended)

The library includes automated setup scripts that configure all required models and dependencies:

# For Windows users
npx react-native-document-scanner-ai setup

# For macOS/Linux users  
npx react-native-document-scanner-ai setup

# Or use the cross-platform setup (auto-detects your OS)
npx react-native-document-scanner-ai setup

This will:

• ✅ Download and configure the ONNX model
• ✅ Set up platform-specific dependencies
• ✅ Configure native module linking
• ✅ Verify the installation

3. Verification

After installation, verify everything is working correctly:

npx react-native-document-scanner-ai verify-setup

4. Manual Model Setup (If Needed)

If automatic setup fails, you can manually generate the required ONNX model:

Using NPX commands:

# Generate model (cross-platform)
npx react-native-document-scanner-ai generate-model

Manual Python approach:

pip install ultralytics
python -c "from ultralytics import YOLO; model = YOLO('yolov8n.pt'); model.export(format='onnx', imgsz=640)"

The model will be automatically placed in the correct platform directories (models/ for general use, ios/ and android/assets/ for platform-specific deployment).

Platform-Specific Setup

iOS Setup

1. Automatic Dependencies: The library automatically includes OpenCV and ONNX Runtime dependencies via CocoaPods

2. Install iOS Dependencies:

cd ios && pod install && cd ..

3. Model Configuration: The ONNX model is automatically bundled with the library

4. Permissions: Add camera permissions to your Info.plist:

<key>NSCameraUsageDescription</key>
<string>This app needs access to camera to scan documents</string>
<key>NSPhotoLibraryUsageDescription</key>
<string>This app needs access to photo library to process images</string>

5. Additional Configuration (if needed):

<!-- For camera access -->
<key>NSCameraUsageDescription</key>
<string>Camera access is required to scan documents</string>

<!-- For saving processed images -->
<key>NSPhotoLibraryAddUsageDescription</key>
<string>Photo library access is needed to save scanned documents</string>

Android Setup

1. Automatic Dependencies: The library automatically includes OpenCV and ONNX Runtime dependencies via Gradle

2. Model Configuration: The ONNX model is included in the assets folder and bundled automatically

3. Permissions: Add the following permissions to your android/app/src/main/AndroidManifest.xml:

<uses-permission android:name="android.permission.CAMERA" />
<uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE" />
<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />
<!-- For Android 13+ photo permissions -->
<uses-permission android:name="android.permission.READ_MEDIA_IMAGES" />

4. Gradle Configuration: Add to your android/app/build.gradle (if not already present):

android {
  packagingOptions {
    pickFirst '**/libc++_shared.so'
    pickFirst '**/libjsc.so'
    pickFirst '**/libfbjni.so'
  }
  
  // Increase heap size for ONNX processing
  dexOptions {
    javaMaxHeapSize "4g"
  }
}

5. ProGuard Configuration (for release builds):

# Keep ONNX Runtime classes
-keep class ai.onnxruntime.** { *; }
-keep class com.documentscannerai.** { *; }

# Keep OpenCV classes  
-keep class org.opencv.** { *; }

Post-Installation Steps

1. Metro Configuration

Ensure your metro.config.js includes the following to properly handle ONNX and model files:

const { getDefaultConfig } = require('metro-config');

module.exports = (async () => {
  const config = await getDefaultConfig();
  
  // Add support for .onnx files
  config.resolver.assetExts.push('onnx', 'pt');
  
  return config;
})();

2. Camera Integration

For camera functionality, install react-native-vision-camera:

npm install react-native-vision-camera
# or
yarn add react-native-vision-camera

3. Testing Installation

Run the verification script to ensure everything is properly configured:

npx react-native-document-scanner-ai verify-setup

This will check:

• ✅ Package installation
• ✅ Model file presence
• ✅ Platform dependencies
• ✅ Permissions configuration
• ✅ Build configuration

Usage

⚠️ Important: Safe Import Pattern

To avoid the "Tried to access a JS module before the React instance was fully set up" error, use one of these safe import patterns:

Option 1: Lazy Loading (Recommended)

import React, { useEffect, useState } from 'react';

function MyComponent() {
  const [scanImage, setScanImage] = useState(null);
  const [scannerReady, setScannerReady] = useState(false);

  useEffect(() => {
    // Load the scanner module after React is ready
    const loadScanner = async () => {
      try {
        const scannerModule = await import('react-native-document-scanner-ai');
        setScanImage(() => scannerModule.scanImage);
        setScannerReady(true);
      } catch (error) {
        console.warn('Failed to load document scanner:', error);
      }
    };

    loadScanner();
  }, []);

  const handleScan = async (imageUri: string) => {
    if (!scanImage || !scannerReady) {
      throw new Error('Document scanner not ready');
    }
    return await scanImage(imageUri);
  };

  // Your component JSX...
}

Option 2: Try-Catch Import

let scanImage = null;

try {
  const scannerModule = require('react-native-document-scanner-ai');
  scanImage = scannerModule.scanImage;
} catch (error) {
  console.warn('Failed to import react-native-document-scanner-ai:', error);
}

// Use scanImage safely
if (scanImage) {
  const result = await scanImage(imageUri);
}

Option 3: Built-in Safe Import Helper

import { createSafeImport } from 'react-native-document-scanner-ai';

const scanner = createSafeImport();

if (scanner.isAvailable && scanner.scanImage) {
  const result = await scanner.scanImage(imageUri);
}

📖 More details: See SAFE_IMPORT.md for comprehensive examples and troubleshooting.

Quick Start

import { scanImage, scanFrame } from 'react-native-document-scanner-ai';

// Scan a single image
const result = await scanImage('file://path/to/image.jpg');
console.log('Detected corners:', result.quadrilateral);
console.log('Confidence:', result.confidence);

Basic Image Scanning

Scan a static image file for document detection:

import { scanImage } from 'react-native-document-scanner-ai';

const scanDocument = async (imageUri: string) => {
  try {
    const result = await scanImage(imageUri, {
      enhance: 'contrast',
      saveOutput: true,
      outputFormat: 'jpg',
      outputQuality: 90,
      maxSize: 1024, // Resize for faster processing
    });

    if (result.confidence > 0.8) {
      console.log('Document detected with high confidence!');
      console.log('Corners:', result.quadrilateral);
      console.log('Enhanced image saved at:', result.outputUri);
    } else {
      console.log('Low confidence detection, manual review needed');
    }
  } catch (error) {
    console.error('Scanning failed:', error);
  }
};

Real-time Camera Integration

Process camera frames in real-time for live document detection:

import React, { useRef, useState } from 'react';
import { View, Text, TouchableOpacity } from 'react-native';
import { Camera, useFrameProcessor, useCameraDevices } from 'react-native-vision-camera';
import { scanFrame } from 'react-native-document-scanner-ai';
import { runOnJS } from 'react-native-reanimated';

function DocumentCameraScreen() {
  const camera = useRef<Camera>(null);
  const devices = useCameraDevices();
  const device = devices.back;
  const [detectedDocument, setDetectedDocument] = useState(null);

  const frameProcessor = useFrameProcessor((frame) => {
    'worklet';
    
    // Convert frame to RGBA array
    const frameData = frame.toArrayBuffer();
    const rgba = new Uint8Array(frameData);
    
    // Process on JS thread
    runOnJS(async (rgbaData: Uint8Array, width: number, height: number) => {
      try {
        const result = await scanFrame(rgbaData, width, height, {
          autoCapture: true,
          captureConfidence: 0.85,
          captureConsecutiveFrames: 5, // Require 5 stable frames
          maxProcessingFps: 10, // Limit processing to 10 FPS
          saveOutput: true,
        });
        
        if (result.outputUri) {
          // Document auto-captured!
          setDetectedDocument(result);
          console.log('Auto-captured document:', result.outputUri);
        }
      } catch (error) {
        console.error('Frame processing error:', error);
      }
    })(rgba, frame.width, frame.height);
  }, []);

  const captureManually = async () => {
    if (camera.current) {
      const photo = await camera.current.takePhoto({
        quality: 90,
        enableAutoRedEyeReduction: true,
      });
      
      const result = await scanImage(photo.path, {
        enhance: 'contrast',
        saveOutput: true,
      });
      
      setDetectedDocument(result);
    }
  };

  if (!device) {
    return <Text>Camera not available</Text>;
  }

  return (
    <View style={{ flex: 1 }}>
      <Camera
        ref={camera}
        device={device}
        isActive={true}
        frameProcessor={frameProcessor}
        photo={true}
        style={{ flex: 1 }}
      />
      
      <TouchableOpacity
        onPress={captureManually}
        style={{
          position: 'absolute',
          bottom: 50,
          alignSelf: 'center',
          backgroundColor: 'white',
          padding: 15,
          borderRadius: 50,
        }}
      >
        <Text>📷 Capture</Text>
      </TouchableOpacity>
    </View>
  );
}

Advanced Configuration

Customize scanning behavior with detailed options:

import { scanImage, ScanOptions } from 'react-native-document-scanner-ai';

const advancedScanOptions: ScanOptions = {
  // Model configuration
  onnxModel: 'custom_model.onnx', // Use custom model
  threshold: 0.5, // Segmentation sensitivity
  
  // Processing options
  maxSize: 1024, // Max image dimension
  enhance: 'bw', // Black & white enhancement
  returnMask: true, // Get segmentation mask
  
  // Output configuration
  saveOutput: true,
  outputFormat: 'png',
  outputQuality: 95,
  
  // Auto-capture settings
  autoCapture: true,
  captureConfidence: 0.9, // High confidence required
  captureConsecutiveFrames: 8, // More stable frames
  maxProcessingFps: 5, // Conservative processing rate
};

const result = await scanImage(imageUri, advancedScanOptions);

// Access additional outputs
if (result.maskUri) {
  console.log('Segmentation mask saved at:', result.maskUri);
}

Batch Processing

Process multiple images efficiently:

const processBatch = async (imageUris: string[]) => {
  const results = await Promise.all(
    imageUris.map(uri => 
      scanImage(uri, {
        enhance: 'contrast',
        maxSize: 512, // Smaller size for batch processing
        saveOutput: false, // Skip saving for speed
      })
    )
  );
  
  const validDocuments = results.filter(r => r.confidence > 0.7);
  console.log(`Found ${validDocuments.length} valid documents`);
  
  return validDocuments;
};

Available Scripts

The library provides different commands depending on whether you're a consumer or developer:

For App Developers (Consumer Usage)

Use these NPX commands from your React Native project:

# Setup the library and download models
npx react-native-document-scanner-ai setup

# Verify installation is working
npx react-native-document-scanner-ai verify-setup

# Generate ONNX model (if needed)
npx react-native-document-scanner-ai generate-model

Note: Do NOT use npm run setup:windows in your app - that only works during library development.

For Library Developers (Development)

These scripts are available when developing the library itself:

# Setup Scripts (library development only)
npm run setup                    # Cross-platform setup
npm run setup:windows           # Windows-specific setup
npm run setup:unix              # macOS/Linux setup

# Model Management (library development only)
npm run generate:model          # Generate ONNX model (Unix)
npm run generate:model:windows  # Generate ONNX model (Windows)

# Verification (library development only)
npm run verify-setup            # Verify installation

# Development Tools
npm run clean                   # Clean build directories
npm run build:android          # Build Android example
npm run build:ios              # Build iOS example
npm run auto-publish           # Automated publishing

API Reference

Types

export type ScanOptions = {
  onnxModel?: string;                  // Custom model path
  threshold?: number;                  // Segmentation threshold (0-1)
  maxSize?: number;                    // Max image size for processing
  enhance?: 'none' | 'bw' | 'contrast'; // Image enhancement
  returnMask?: boolean;                // Return segmentation mask
  saveOutput?: boolean;                // Save processed image
  outputFormat?: 'jpg' | 'png';       // Output format
  outputQuality?: number;              // Quality (0-100)
  autoCapture?: boolean;               // Enable auto-capture
  captureConfidence?: number;          // Min confidence for auto-capture
  captureConsecutiveFrames?: number;   // Stable frames required
  maxProcessingFps?: number;           // Processing throttle
};

export type ScanResult = {
  quadrilateral: [number, number][];   // Detected document corners
  confidence: number;                  // Detection confidence (0-1)
  outputUri?: string;                  // Processed image path
  maskUri?: string;                    // Segmentation mask path
};

Functions

scanImage(uri: string, options?: ScanOptions): Promise<ScanResult>

Scans a single image file for document detection and processing.

Parameters:

• uri: File URI of the image to process
• options: Scan configuration options

Returns: Promise resolving to scan result

scanFrame(rgba: Uint8Array, width: number, height: number, options?: ScanOptions): Promise<ScanResult>

Processes a camera frame for real-time document detection.

Parameters:

• rgba: RGBA pixel data as Uint8Array
• width: Frame width in pixels
• height: Frame height in pixels
• options: Scan configuration options

Returns: Promise resolving to scan result

Troubleshooting

Common Issues

0. "Tried to access a JS module before the React instance was fully set up"

Tried to access a JS module before the React instance was fully set up. 
Calls to ReactContext#getJSModule should only happen once initialize() has been called on your native module.

Problem: The native module is being accessed before React Native is fully initialized.

Solution: Use the safe import patterns described in the Usage section above:

1. Lazy Loading (Recommended):

   const [scanImage, setScanImage] = useState(null);
      
   useEffect(() => {
     import('react-native-document-scanner-ai').then(module => {
       setScanImage(() => module.scanImage);
     });
   }, []);

2. Try-Catch Import:

   let scanImage = null;
   try {
     const module = require('react-native-document-scanner-ai');
     scanImage = module.scanImage;
   } catch (error) {
     console.warn('Scanner not available:', error);
   }

3. Always restart Metro after installing: npx react-native start --reset-cache

1. "Missing script" Error

npm error Missing script: "setup:windows"

Problem: You're trying to run library development scripts from your app project.

Solution: Use NPX commands instead:

# ❌ Wrong (only works in library development)
npm run setup:windows

# ✅ Correct (use in your app)
npx react-native-document-scanner-ai setup

1. Model Not Found Error

Error: ONNX model not found at path

Solution: Run the setup script to download the model:

npx react-native-document-scanner-ai setup

2. Build Errors on Android

Error: Failed to resolve: ai.onnxruntime

Solution: Ensure your android/app/build.gradle includes the packaging options:

android {
  packagingOptions {
    pickFirst '**/libc++_shared.so'
    pickFirst '**/libjsc.so'
  }
}

3. iOS Pod Install Issues

[!] CocoaPods could not find compatible versions for pod "ONNX"

Solution: Update CocoaPods and clear cache:

cd ios
rm -rf Pods Podfile.lock
pod install --repo-update

4. Camera Permission Denied

Solution: Ensure permissions are properly configured in your platform files and request permissions at runtime.

5. Performance Issues

• Reduce maxSize option for faster processing
• Lower maxProcessingFps for real-time processing
• Use enhance: 'none' to skip post-processing

Getting Help

1. Check Setup: Run npx react-native-document-scanner-ai verify-setup
2. Review Logs: Enable verbose logging in development
3. Platform Issues: Check platform-specific setup instructions
4. Create Issue: GitHub Issues

Performance Tips

1. Image Size: Use maxSize option to limit processing resolution (recommended: 1024px)
2. Frame Rate: Set maxProcessingFps to throttle real-time processing (recommended: 5-10 FPS)
3. Model Selection: The included YOLOv8n model is optimized for mobile devices
4. Threading: Processing runs on background threads automatically
5. Memory: Enable returnMask: false unless segmentation masks are needed
6. Auto-capture: Use higher captureConsecutiveFrames for more stable captures

Model Information

Included Models

• document_segmentation.onnx: Pre-trained YOLOv8n model for document detection
• Input Format: RGB images, normalized to [0,1], 640x640 resolution
• Output Format: Segmentation masks with confidence scores
• Model Size: ~6MB (optimized for mobile)

Custom Models

You can use custom ONNX models by specifying the onnxModel path:

const result = await scanImage(imageUri, {
  onnxModel: 'path/to/custom_model.onnx',
  threshold: 0.5,
});

Model Requirements:

• Input: [1, 3, H, W] tensor (RGB image, normalized 0-1)
• Output: [1, 1, H, W] tensor (segmentation mask, 0-1 values)
• Format: ONNX with standard operators
• Optimization: Use ONNX Runtime optimization tools for best performance

Training Custom Models

For training custom document detection models:

1. Datasets: Use DocLayNet, PubLayNet, or custom document datasets
2. Framework: Train with YOLOv8, Detectron2, or similar frameworks
3. Export: Convert to ONNX format with appropriate input/output shapes
4. Optimization: Use ONNX Runtime tools for mobile optimization

Example App

The example app demonstrates all library features:

• ✨ Real-time camera document detection with live preview
• 🎯 Auto-capture functionality with confidence thresholds
• 🎨 Image enhancement modes (B&W, contrast boost, original)
• 📐 Quadrilateral overlay visualization
• 📱 Cross-platform implementation (iOS & Android)
• 🔧 Configuration options and performance tuning

Run the example:

git clone https://github.com/AaronAnablon/React-Native-Document-Cleaner.git
cd React-Native-Document-Cleaner/example
npm install

# iOS
cd ios && pod install && cd ..
npx react-native run-ios

# Android  
npx react-native run-android

File Structure

When properly installed, your project should include:

your-project/
├── node_modules/
│   └── react-native-document-scanner-ai/
│       ├── lib/                    # Compiled TypeScript
│       ├── src/                    # Source TypeScript files
│       ├── android/                # Android native code
│       ├── ios/                    # iOS native code
│       ├── cpp/                    # C++ core implementation
│       ├── models/                 # ONNX models
│       │   └── document_segmentation.onnx
│       ├── assets/                 # Additional assets
│       │   └── yolov8n.pt         # Training checkpoint
│       ├── scripts/                # Setup and utility scripts
│       └── bin/                    # CLI tools
├── android/
│   └── app/
│       └── src/main/assets/        # Android model deployment
│           └── document_segmentation.onnx
└── ios/
    └── document_segmentation.onnx  # iOS model deployment

Package Information

Included Files

The npm package includes:

• ✅ Compiled JavaScript/TypeScript libraries
• ✅ Native Android and iOS code
• ✅ C++ core implementation
• ✅ Pre-trained ONNX models
• ✅ Setup and verification scripts
• ✅ CLI tools for easy management
• ✅ Documentation and examples

Version Information

Check your installed version:

npm list react-native-document-scanner-ai

Update to latest:

npm update react-native-document-scanner-ai

Quick Reference

Installation Commands

# Install package
npm install react-native-document-scanner-ai

# Setup (automated)
npx react-native-document-scanner-ai setup

# Verify installation
npx react-native-document-scanner-ai verify-setup

# Generate model (if needed)
npx react-native-document-scanner-ai generate-model

Platform Setup

# iOS
cd ios && pod install

# Android (permissions in AndroidManifest.xml)
# Build configuration in build.gradle

Basic Usage

import { scanImage } from 'react-native-document-scanner-ai';

const result = await scanImage('file://path/to/image.jpg', {
  enhance: 'contrast',
  saveOutput: true,
});

Camera Integration

import { scanFrame } from 'react-native-document-scanner-ai';
// Use with react-native-vision-camera frameProcessor

Contributing

• Development workflow
• Sending a pull request
• Automated publishing guide
• Code of conduct

Publishing

This package uses automated publishing for releases. See PUBLISHING.md for detailed information.

Quick commands:

# Check for changes and publish automatically
npm run auto-publish

# Preview what would be published (dry-run)
npm run auto-publish:dry-run

# Windows users
npm run auto-publish:windows

License

MIT

Made with create-react-native-library