@skinanalytics/react-native-ai-image-processor — Installation & Usage Guide

On-device AI image analysis for React Native. Runs ML models to validate skin images before upload — checking skin type, image quality, and blur. Uses ExecuTorch on both iOS and Android.

Platform-specific native setup and use cases: docs/IOS_INTEGRATION.md / docs/ANDROID_INTEGRATION.md. This file covers the cross-platform JS API only. For how the AI models themselves work — inputs/outputs, thresholds, and the full processing pipeline with diagrams — see docs/AI_MODELS.md.

Version history

| SDK version | What changed | AI models & versions | |---|---|---| | 1.0.0 | Initial extraction of the SDK from the main app. iOS used CoreML. | iOS (CoreML): mole_not_mole_model_v5 (v5) · macro_image_quality_model_v2 (v2, dual input — pixels + FFT) · dermo_image_quality_v2 (v2) | | 2.0.4 | React Native integration completed, plus a skin-confidence bug fix. Android migrated its model runtime from PyTorch Mobile (.ptl) to ExecuTorch (.pte), then re-exported the models with a hardware-acceleration backend. iOS still on CoreML. | iOS (CoreML): unchanged from 1.0.0Android (ExecuTorch 1.3.1): mole_not_mole_v5.pte (v5) · macro_image_quality_v2.pte (v2) · dermo_image_quality_v2.pte (v2) | | 2.1.0 | IQC pipeline unification epic: iOS migrated from CoreML to ExecuTorch; TTA, the FFT log-magnitude transform and channel normalisation moved out of per-platform native code into shared C++ (common/cpp), reached via JSI on both platforms; threshold logic and macro/dermo orchestration moved into shared TypeScript. Macro gained its own dedicated skin classifier (it previously reused the dermo classifier). The macro quality model was rebuilt as single-input (the FFT input was dropped). Both platforms now load one shared set of .pte files from SDK/…/models/. | Shared, ExecuTorch 1.3.1 (iOS + Android): mole_not_mole_v5.pte (v5, now dermo-only) · macro_mole_not_mole_v1.pte (new, v1 — macro's dedicated classifier, replaces macro's use of v5) · macro_image_quality_v2-1.pte (new — distinct weights from the retired dual-input macro_image_quality_v2.pte, single input only) · dermo_image_quality_v2.pte (v2, unchanged) | | 2.1.2 | Performance fix for the processing-time regression the 2.1.0 pipeline unification introduced (large tensor/pixel buffers were being marshaled across the RN bridge as boxed-number arrays). forward() and resizeDecodedImage() now transfer those buffers as base64, encoded/decoded in shared C++ (common/cpp/Base64.cpp) via the same JSI mechanism as applyTTA/normalizeChannels, instead of a JS loop. TTA variants are now generated lazily (one at a time) instead of all 8 upfront. Also fixes the npm package files allowlist so common/ and models/ are published (2.1.1 omitted them). No model, threshold, or public API changes. | Shared, ExecuTorch 1.3.1 (iOS + Android): unchanged from 2.1.0 |

Requirements

| Requirement | Version | |---|---| | iOS | 17.0+ (ExecuTorch's vendored frameworks are built for iOS 17 — hard floor) | | Android | 7.0+ (API level 24) | | React Native | 0.71+ | | Xcode | Swift 5.9+ | | Node | 18+ |

Installation

1. Authenticate with the registry

1. Add a .npmrc file to your project root:
2. The global .npmrc file is located in /Users/{user}/.npmrc, if you do not have it please create it.
3. Paste the following in the .npmrc file and update with your token. If you don't have an access token please request it from the MD tech team

//registry.npmjs.org/:_authToken=${NPM_TOKEN}

2. Install the package

npm install @skinanalytics/react-native-ai-image-processor
# or
yarn add @skinanalytics/react-native-ai-image-processor

3. Install the native pod

iOS

cd ios && pod install

Full native setup detail (deployment target, vendored frameworks, first-build notes) is in docs/IOS_INTEGRATION.md.

Android

No additional step required. The ExecuTorch dependency (executorch-android:1.3.1) is resolved automatically by Gradle.

Full native setup detail (autolinking, NDK/CMake notes, troubleshooting) is in docs/ANDROID_INTEGRATION.md.

4. Rebuild the app

iOS

yarn ios

Android

yarn android

Metro hot reload does not pick up native module changes — a full rebuild is required after installation. Android Emulator supports the camera for testing; iOS Simulator does not expose a camera.

Image requirements

The image must meet the following requirements before it can be assessed by the AI:

• A good quality image of a skin lesion (the AI will validate this and an appropriate message will be returned)
• A .jpeg/.jpg format image file (PNG is not accepted on either platform)
• Minimum 1000 x 1000 pixels (Android also rejects above 10000 x 10000; iOS currently has no upper bound)
• Aspect ratio must be 16:9
• Minimum 100kb file size

For exactly which model checks which of these, the per-model thresholds, and a diagram of the full pipeline, see docs/AI_MODELS.md.

Setup

Call initModels once at app startup before any image processing. It loads the ML models into memory (ExecuTorch on both platforms).

import { initModels } from '@skinanalytics/react-native-ai-image-processor';

// In your root component or app initialisation
useEffect(() => {
    initModels();
}, []);

initModels is safe to call multiple times — it returns "ALREADY_INITIALIZED" on repeat calls and does not reload models.

Processing an image

import { processImage } from '@skinanalytics/react-native-ai-image-processor';

const result = await processImage(
    photo.uri,                       // URI from camera or image picker
    'macro',                         // photo type: "macro" | "dermo"
    true                             // validate resolution + aspect ratio
);

if (result.result === 'passed') {
    // image is valid, proceed with upload
} else {
    // image failed validation — inspect result.checks for the reason
}

Photo types

| Value | Use for | |---|---| | "macro" | Standard smartphone camera | | "dermo" | Dermoscope camera |

Example results

Passed — macro image

{
    "result": "passed",
    "photoType": "macro",
    "checks": {
        "isDermoSkinImage": "n/a",
        "isMacroSkinImage": "passed",
        "imageQuality": "passed",
        "isBlurryImage": "passed",
        "isDarkImage": "passed"
    },
    "modelConfidence": {
        "isDermoSkinConfidence": null,
        "isMacroSkinConfidence": 0.97,
        "isAbnormalLightConfidence": 0.04,
        "isBlurImageConfidence": 0.02
    }
}

Failed — blurry dermo image

{
    "result": "failed",
    "photoType": "dermo",
    "checks": {
        "isDermoSkinImage": "passed",
        "isMacroSkinImage": "n/a",
        "imageQuality": "failed",
        "isBlurryImage": "failed",
        "isDarkImage": "passed"
    },
    "modelConfidence": {
        "isDermoSkinConfidence": 0.91,
        "isMacroSkinConfidence": null,
        "isAbnormalLightConfidence": 0.11,
        "isBlurImageConfidence": 0.89
    }
}

Failed — not a skin image

{
    "result": "failed",
    "photoType": "macro",
    "checks": {
        "isDermoSkinImage": "n/a",
        "isMacroSkinImage": "failed",
        "imageQuality": "n/a",
        "isBlurryImage": "n/a",
        "isDarkImage": "n/a"
    },
    "modelConfidence": {
        "isDermoSkinConfidence": null,
        "isMacroSkinConfidence": 0.08,
        "isAbnormalLightConfidence": 0.0,
        "isBlurImageConfidence": 0.0
    }
}

"n/a" checks are skipped when the image fails an earlier gate (e.g. not a skin image) or when the check is not applicable to the photoType.

Full example

import { useEffect } from 'react';
import { initModels, processImage } from '@skinanalytics/react-native-ai-image-processor';
import type { ImageAnalysisResult } from '@skinanalytics/react-native-ai-image-processor';

export function useImageProcessor() {

    useEffect(() => {
        initModels().catch(console.error);
    }, []);

    async function analyseImage(uri: string): Promise<ImageAnalysisResult> {
        const result = await processImage(uri, 'macro', true);

        if (result.result === 'failed') {
            const failed = Object.entries(result.checks)
                .filter(([, status]) => status === 'failed')
                .map(([check]) => check);

            console.warn('Image failed checks:', failed);
        }

        return result;
    }

    return { analyseImage };
}

API reference

initModels(): Promise<InitStatus>

Loads ML models into memory (ExecuTorch on both platforms). Call once before processImage.

type InitStatus = "READY" | "ALREADY_INITIALIZED";

processImage(path, photoType, isImageValidationRequired): Promise<ImageAnalysisResult>

Runs the full AI pipeline on a local JPEG file.

| Parameter | Type | Description | |---|---|---| | path | string | URI from camera or image picker (e.g. photo.uri). On Android, content:// URIs are also accepted. | | photoType | PhotoType | "macro" or "dermo" | | isImageValidationRequired | boolean | Pre-validate resolution and aspect ratio |

Types

type PhotoType = "macro" | "dermo";
type CheckStatus = "passed" | "failed" | "n/a";
type InitStatus = "READY" | "ALREADY_INITIALIZED";

interface ImageAnalysisResult {
    result: CheckStatus;
    photoType: PhotoType;
    modelConfidence: ModelConfidence;
    checks: AnalysisChecks;
}

interface AnalysisChecks {
    isDermoSkinImage: CheckStatus;
    isMacroSkinImage: CheckStatus;
    imageQuality: CheckStatus;
    isBlurryImage: CheckStatus;
    isDarkImage: CheckStatus;
}

interface ModelConfidence {
    isDermoSkinConfidence: number | null;
    isMacroSkinConfidence: number | null;
    isAbnormalLightConfidence: number;
    isBlurImageConfidence: number;
}

Platform support

| Platform | Supported | ML Framework | Model Format | |---|---|---|---| | iOS | ✅ | ExecuTorch 1.3.1 | .pte | | Android | ✅ | ExecuTorch 1.3.1 | .pte |

Error codes and troubleshooting

See docs/ERROR_CODES.md and docs/TROUBLESHOOTING.md for the full, per-platform reference.

Build error: module not found Ensure your .npmrc is configured with a valid NPM_TOKEN and the registry URL matches where the package was published.

License

Proprietary — © Skin Analytics. All rights reserved.