react-native-pixel-data

A high-performance React Native SDK for extracting raw pixel data from images and preparing it for machine learning inference. Native implementations in Swift (iOS) and Kotlin (Android) ensure optimal performance.

Features

• 🖼️ Multiple Image Sources: URL, local file, base64, app assets, photo library
• 🎨 Color Formats: RGB, RGBA, BGR, BGRA, Grayscale
• 📐 Smart Resizing: Cover, contain, and stretch strategies
• 🔢 Normalization Presets: ImageNet, TensorFlow, scale (0-1), raw, custom
• 📊 Data Layouts: HWC, CHW, NHWC, NCHW (for ML framework compatibility)
• ✂️ ROI Extraction: Crop specific regions before processing
• ⚡ Batch Processing: Process multiple images with concurrency control
• 📱 Native Performance: Swift (iOS) and Kotlin (Android) implementations

Installation

npm install react-native-pixel-data
# or
yarn add react-native-pixel-data

iOS

cd ios && pod install

Android

No additional setup required - auto-linking handles everything.

Quick Start

import { getPixelData } from 'react-native-pixel-data';

// Basic usage - get RGB pixel data from a URL
const result = await getPixelData({
  source: { type: 'url', value: 'https://example.com/image.jpg' },
  resize: { width: 224, height: 224 },
});

console.log('Pixel data:', result.data); // Float array of normalized RGB values
console.log('Dimensions:', result.width, result.height, result.channels);

API Reference

getPixelData(options)

Extracts pixel data from a single image.

const result = await getPixelData({
  source: ImageSource;           // Required: Image source
  colorFormat?: ColorFormat;     // Default: 'RGB'
  resize?: ResizeOptions;        // Optional: Resize dimensions
  roi?: Roi;                     // Optional: Region of interest
  normalization?: Normalization; // Default: { preset: 'scale' }
  dataLayout?: DataLayout;       // Default: 'HWC'
  outputFormat?: OutputFormat;   // Default: 'array'
});

batchGetPixelData(optionsArray, batchOptions?)

Processes multiple images with concurrency control.

const results = await batchGetPixelData(
  [
    { source: { type: 'url', value: 'https://example.com/image1.jpg' } },
    { source: { type: 'url', value: 'https://example.com/image2.jpg' } },
  ],
  { concurrency: 4 }
);

Image Sources

URL

{ type: 'url', value: 'https://example.com/image.jpg' }

Local File

{ type: 'file', value: '/path/to/image.png' }
// or with file:// prefix
{ type: 'file', value: 'file:///path/to/image.png' }

Base64

{ type: 'base64', value: 'data:image/png;base64,iVBORw0KGgo...' }
// or without data URI prefix
{ type: 'base64', value: 'iVBORw0KGgo...' }

App Asset

{ type: 'asset', value: 'images/sample.png' }

Photo Library (iOS Photos / Android Content URI)

// iOS
{ type: 'photoLibrary', value: 'ph://ASSET-LOCAL-IDENTIFIER' }
// Android
{ type: 'photoLibrary', value: 'content://media/external/images/media/123' }

Color Formats

| Format | Channels | Description | |--------|----------|-------------| | RGB | 3 | Red, Green, Blue | | RGBA | 4 | Red, Green, Blue, Alpha | | BGR | 3 | Blue, Green, Red (OpenCV style) | | BGRA | 4 | Blue, Green, Red, Alpha | | GRAYSCALE | 1 | Luminance (ITU-R BT.601) |

Resize Strategies

resize: {
  width: 224,
  height: 224,
  strategy: 'cover' | 'contain' | 'stretch'
}

| Strategy | Description | |----------|-------------| | cover | Fill target dimensions, crop excess (default) | | contain | Fit within dimensions, letterbox with black | | stretch | Stretch to exact dimensions |

Normalization Presets

Scale (Default)

Scales pixel values to [0, 1] range.

normalization: { preset: 'scale' }
// Result: pixel / 255

ImageNet

Standard ImageNet normalization for PyTorch models.

normalization: { preset: 'imagenet' }
// Result: (pixel/255 - mean) / std
// mean: [0.485, 0.456, 0.406]
// std: [0.229, 0.224, 0.225]

TensorFlow

Scales to [-1, 1] range for TensorFlow models.

normalization: { preset: 'tensorflow' }
// Result: (pixel - 127.5) / 127.5

Raw

No normalization - returns raw 0-255 values.

normalization: { preset: 'raw' }

Custom

Define your own mean and standard deviation.

normalization: {
  preset: 'custom',
  mean: [0.5, 0.5, 0.5],
  std: [0.5, 0.5, 0.5]
}

Data Layouts

| Layout | Description | Use Case | |--------|-------------|----------| | HWC | Height × Width × Channels | General purpose (default) | | CHW | Channels × Height × Width | PyTorch | | NHWC | Batch × H × W × C | TensorFlow | | NCHW | Batch × C × H × W | PyTorch batched |

PyTorch Example

const result = await getPixelData({
  source: { type: 'url', value: imageUrl },
  resize: { width: 224, height: 224, strategy: 'cover' },
  normalization: { preset: 'imagenet' },
  dataLayout: 'CHW', // PyTorch format
});

// result.data shape: [3, 224, 224] (flattened)

TensorFlow Example

const result = await getPixelData({
  source: { type: 'url', value: imageUrl },
  resize: { width: 224, height: 224, strategy: 'cover' },
  normalization: { preset: 'tensorflow' },
  dataLayout: 'NHWC', // TensorFlow format
});

// result.data shape: [1, 224, 224, 3] (flattened)

Output Formats

Control the type of array returned for pixel data:

| Format | Description | |--------|-------------| | array | Standard JavaScript number array (default) | | float32Array | Float32Array for better memory efficiency | | uint8Array | Uint8Array for raw 0-255 values |

// Default: JavaScript array
const result = await getPixelData({
  source: { type: 'url', value: imageUrl },
  outputFormat: 'array', // number[]
});

// Float32Array for ML frameworks
const result = await getPixelData({
  source: { type: 'url', value: imageUrl },
  outputFormat: 'float32Array', // Float32Array
});

// Uint8Array for raw pixel values
const result = await getPixelData({
  source: { type: 'url', value: imageUrl },
  normalization: { preset: 'raw' },
  outputFormat: 'uint8Array', // Uint8Array
});

Region of Interest (ROI)

Extract a specific region before resizing:

const result = await getPixelData({
  source: { type: 'url', value: imageUrl },
  roi: {
    x: 100,      // Left offset
    y: 50,       // Top offset
    width: 200,  // Region width
    height: 200  // Region height
  },
  resize: { width: 224, height: 224 }
});

Batch Processing

Process multiple images efficiently:

import { batchGetPixelData, isPixelDataError } from 'react-native-pixel-data';

const images = [
  'https://example.com/image1.jpg',
  'https://example.com/image2.jpg',
  'https://example.com/image3.jpg',
];

const batchResult = await batchGetPixelData(
  images.map(url => ({
    source: { type: 'url', value: url },
    resize: { width: 224, height: 224 },
    normalization: { preset: 'imagenet' },
    dataLayout: 'CHW',
  })),
  { concurrency: 4 } // Process 4 images in parallel
);

console.log(`Processed ${batchResult.results.length} images`);
console.log(`Total time: ${batchResult.totalTimeMs}ms`);

// Check for errors in individual results using type guard
batchResult.results.forEach((result, index) => {
  if (isPixelDataError(result)) {
    console.error(`Image ${index} failed:`, result.code, result.message);
  } else {
    console.log(`Image ${index}: ${result.width}x${result.height}`);
  }
});

Error Handling

import { getPixelData, PixelDataException } from 'react-native-pixel-data';

try {
  const result = await getPixelData({
    source: { type: 'url', value: 'https://invalid-url.com/image.jpg' }
  });
} catch (error) {
  if (error instanceof PixelDataException) {
    console.error('Code:', error.code);    // e.g., 'LOAD_ERROR'
    console.error('Message:', error.message);
  }
}

Error Codes

| Code | Description | |------|-------------| | INVALID_SOURCE | Invalid image source configuration | | LOAD_ERROR | Failed to load image from source | | FILE_NOT_FOUND | Local file does not exist | | PERMISSION_DENIED | Missing permissions for photo library | | PROCESSING_ERROR | Error during pixel processing | | INVALID_RESIZE | Invalid resize dimensions or strategy | | INVALID_ROI | Invalid region of interest parameters | | INVALID_NORMALIZATION | Invalid normalization configuration | | UNKNOWN | Unknown or unexpected error |

TypeScript Types

All types are fully exported:

import type {
  ImageSource,
  ColorFormat,
  ResizeOptions,
  ResizeStrategy,
  Roi,
  Normalization,
  NormalizationPreset,
  DataLayout,
  OutputFormat,
  GetPixelDataOptions,
  PixelDataResult,
  BatchOptions,
  BatchResult,
  PixelDataError,
  PixelDataErrorResult,
} from 'react-native-pixel-data';

// Type guard for error checking
import { isPixelDataError, getChannelCount } from 'react-native-pixel-data';

// Utility: Get channel count for a color format
getChannelCount('RGB');       // Returns 3
getChannelCount('RGBA');      // Returns 4
getChannelCount('GRAYSCALE'); // Returns 1

Result Object

interface PixelDataResult {
  data: number[];        // Pixel values (normalized)
  width: number;         // Output width
  height: number;        // Output height
  channels: number;      // Number of channels (1, 3, or 4)
  dataLayout: DataLayout; // Layout used
  shape?: number[];      // Array dimensions based on layout (e.g., [224, 224, 3] for HWC)
  processingTimeMs: number; // Processing time in milliseconds
}

Batch Error Object

When batch processing fails for individual images:

interface PixelDataError {
  error: true;           // Always true for errors
  code: string;          // Error code (e.g., 'LOAD_ERROR')
  message: string;       // Human-readable error message
  index: number;         // Index of failed image in batch (0-based)
}

Performance Tips

1. Use appropriate resize dimensions: Smaller images process faster
2. Batch similar operations: Use batchGetPixelData for multiple images
3. Choose the right concurrency: Too high can cause memory pressure
4. Use ROI for large images: Extract only what you need
5. Cache results: Store processed data if you'll reuse it

Requirements

• React Native >= 0.72
• iOS 13.0+
• Android API 21+

License

MIT © Manish Kumar

Contributing

Contributions are welcome! Please read our Contributing Guide and Code of Conduct before submitting pull requests.

Support

• 🐛 Report a bug
• 💡 Request a feature
• 📖 Read the changelog