capacitor-faceantispoofing-cpp

Capacitor plugin for real-time passive liveness detection (face anti-spoofing) using the ncnn C++ engine ported from the GreatDay kiosk app (greatdayhr-kioskv2).

Unlike capacitor-plugin-faceantispoofing (single-shot TFLite + MTCNN), this plugin wraps the kiosk EngineWrapper pipeline:

upright frame (JPEG/base64 from the WebView camera preview)
  → decode → BGR (cv::Mat)
  → FaceDetector  (ncnn, RetinaFace-style, 192px input, mean {104,117,123})
  → Live          (2 anti-spoofing models, 80×80 ROI, confidence = mean softmax[1])
  → { hasFace, confidence (liveness score), face box, timeMs }

• confidence >= threshold → real face (default threshold 0.92, the kiosk-calibrated LivenessHelper.EFFECTIVE_THRESHOLD)
• confidence < threshold → spoof (photo / printed image)

Works on Android and iOS with the same models and the same JS API.

API

import { FaceAntiSpoofingCpp } from 'capacitor-faceantispoofing-cpp';

// 1) Load ncnn models (call once, e.g. when entering the camera page)
await FaceAntiSpoofingCpp.initialize({ threshold: 0.92 });
// => { ready, libraryFound, message }

// 2) Feed throttled preview frames (suggested every 250-400 ms) to drive
//    real-time mask feedback like the kiosk (green / red silhouette)
const frame = await FaceAntiSpoofingCpp.detectFrame({ image: base64Jpeg });
// => { error, hasFace, liveness, confidence, score, threshold, box: {left, top, right, bottom}, timeMs, message }

// 3) Final check on the captured photo before submitting attendance
const shot = await FaceAntiSpoofingCpp.detect({ image: base64Jpeg });
if (shot.liveness) { /* proceed */ }

// 4) Free native handles when leaving the page
await FaceAntiSpoofingCpp.destroy();

image accepts a base64 string (with or without the data:image/...;base64, prefix) or a file:///absolute path. Frames must be upright (the engine applies no rotation; the kiosk passed orientation=7 for its front camera, WebView canvas frames are already rotated correctly).

The detect/detectFrame promise resolves (not rejects) with error: true + message for per-frame soft failures (not initialized, decode failure, no models), so a frame loop never needs try/catch. reject is reserved for missing parameters.

Platform notes

Android

• Rebuilds the kiosk libengine.so from source via CMake (android/src/main/cpp), linking vendored libncnn.a + OpenCV .so (arm64-v8a, armeabi-v7a, NDK 27.0.12077973).
• Models ship in android/src/main/assets/{detection,live}.
• JNI symbols are bound to io.github.asephermann.plugins.faceantispoofingcpp.engine.{FaceDetector,Live} (definition.h).

iOS

• ncnn and openmp are not published on CocoaPods, so static frameworks from the Tencent ncnn releases (20260526, non-vulkan, device + simulator) are vendored as hand-assembled xcframeworks in ios/Frameworks/.
• OpenCV comes from the official OpenCV2 pod.
• Models ship in the CapacitorFaceAntiSpoofingCpp resource bundle (ios/Sources/FaceAntiSpoofingCppPlugin/models).
• The C++ core (engine/face_detector.cpp, engine/live.cpp) is the kiosk code with AAssetManager loading replaced by bundle-file loading; the bridge is Obj-C++ (EngineWrapper.mm) behind a serial mutex, mirroring the kiosk engineLock/destroyed SIGSEGV guard.

Web

Not supported; web.ts resolves with ready: false / error: true results (falls back to the gesture-based web liveness in the app).

Size impact

libncnn.a/OpenCV are only linked symbols for iOS (static, stripped by app thinning) and ~80 MB uncompressed per-ABI .so set on Android (AAB splits by ABI; the APK impact is roughly the kiosk's own footprint).

Install

npm install capacitor-faceantispoofing-cpp
npx cap sync

No extra permission entries are needed (the plugin does not touch the camera; the webview getUserMedia preview already does).

Origin / provenance

• Engine source: greatdayhr-kioskv2 app/src/main/cpp (a port of the open-source ncnn LiveBody/FaceMaskDemo by yuanhao, ncnn + OpenCV).
• Kotlin wrapper: com.greatday.kiosk.app.library.mv.engine.EngineWrapper → ported to Java under engine/ (Android) and Obj-C++ FASCppEngine (iOS).
• Models: kiosk assets/detection/* and assets/live/* (unmodified, byte-identical), so detection behavior and the 0.92 threshold calibration carry over 1:1.