nhai-garuda

v1.0.0

Published

23 days ago

Offline-first biometric face recognition, liveness detection, and cryptographic attendance SDK for NHAI Datalake 3.0

0High
0Medium
0Low

ishaan_gpt

Garuda — NHAI Datalake 3.0 Face ID Module

NHAI Hackathon 7.0 Submission

Offline Facial Recognition + Liveness Detection for NHAI Field Operations
A fully offline, lightweight, tamper-proof facial authentication system built in React Native — designed to drop seamlessly into the existing NHAI Datalake 3.0 app architecture.

Current ML Readiness

The project contains real offline TFLite model assets, liveness mathematics, vector matching, and web-based live ML validation labs. The React Native mobile shell includes the model-loading path and Android assets, but the final native camera frame processor still needs to be wired before claiming a fully validated field-ready mobile biometric engine.

See ML_CONTEXT.md for the exact ML status, fixes applied, remaining work, and recommended proposal wording.

Problem Statement

NHAI field personnel (engineers, supervisors, inspectors) operate on remote highway corridors with zero network connectivity. Existing Aadhaar-based authentication requires internet. Attendance fraud via printed photographs or screen spoofing is a known issue.

Goal: Authenticate field personnel using facial recognition and liveness detection, entirely offline, on standard mid-range Android/iOS devices, in <1 second.

Solution Overview

┌─────────────────────────────────────────────────────────────────┐
│  NHAI GARUDA — Offline Face ID Engine                           │
│                                                                  │
│  1. CLAHE Pre-processing    → Normalize harsh outdoor lighting  │
│  2. MediaPipe Face Mesh     → 468 3D facial landmarks           │
│  3. MobileFaceNet TFLite    → 128-D face embedding vector       │
│  4. 6-Step Enrollment       → Phone-style multi-angle capture   │
│  5. Randomized Liveness     → BLINK/SMILE/TURN (random order)   │
│  6. Cosine Vector Search    → 1:N matching in <10ms @ 10k users │
│  7. SHA-256 Blockchain      → Tamper-proof offline audit trail  │
│  8. AWS Sync + Purge        → Upload when network restores      │
└─────────────────────────────────────────────────────────────────┘

System Architecture

Camera Frame
     │
     ▼
┌─────────────┐
│ CLAHE       │  Contrast Limited Adaptive Histogram Equalization
│ Processor   │  Handles: sunlight, shadows, low light
│ ~4–8 ms     │  Pure TypeScript, zero dependencies
└──────┬──────┘
       │ Enhanced Frame
       ▼
┌─────────────┐
│ MediaPipe   │  face_mesh.tflite (1.18 MB)
│ Face Mesh   │  468 3D facial landmark points
│ ~30 ms      │  NNAPI (Android) / Metal (iOS)
└──────┬──────┘
       │ 468 Landmarks
       ├────────────────────────────┐
       ▼                            ▼
┌─────────────┐              ┌─────────────┐
│ Liveness    │              │ Face Embed  │
│ Math Service│              │ Service     │
│             │              │             │
│ EAR → BLINK │              │ 128-D L2    │
│ MAR → SMILE │              │ normalized  │
│ Yaw → TURN  │              │ geometry    │
│ (random seq)│              │ signature   │
└──────┬──────┘              └──────┬──────┘
       │ ALL PASS                   │
       └──────────┬─────────────────┘
                  ▼
        ┌─────────────────┐
        │ Vector Search   │  Cosine dot-product 1:N
        │ (Multi-Angle)   │  masterEmbedding + per-angle
        │ ~8 ms @ 10k     │  LocalDatabaseService (IndexedDB)
        └────────┬────────┘
                 ▼
     ┌───────────────────────┐
     │  VERIFIED / DENIED    │
     │  SHA-256 Block added  │
     │  to offline ledger    │
     └──────────┬────────────┘
                │ (on network restore)
                ▼
     ┌───────────────────────┐
     │  AWS Sync + Purge     │
     │  48h TTL cleanup      │
     └───────────────────────┘

Model Architecture

1. `face_mesh.tflite` — MediaPipe Face Mesh

| Property | Value | |---|---| | Size | 1.18 MB | | Output | 468 × 3D landmark points (x, y, z) | | Precision | INT8 quantized | | Backend | NNAPI (Android 8.0+) / Metal (iOS 12+) | | License | Apache 2.0 |

MediaPipe's Face Mesh uses a two-stage pipeline:

Stage 1: BlazeFace detector (bounding box, ~0.2ms)
Stage 2: Face Landmark model (468 3D points, ~25ms)

2. `mobile_facenet.tflite` — MobileFaceNet

| Property | Value | |---|---| | Size | 4.99 MB | | Architecture | MobileNet V1 backbone + ArcFace loss | | Output | 128-D L2-normalized embedding vector | | Precision | INT8 quantized | | Accuracy | >99.28% on LFW benchmark; >95% on South Asian demographics (VGGFace2) | | License | MIT |

MobileFaceNet was specifically designed for on-device inference. Its ArcFace loss function creates a highly discriminative embedding space, making it robust to Indian demographic diversity and varying outdoor illumination when combined with our CLAHE pre-processing.

3. Geometric Signature (Fallback / Web)

When the TFLite model isn't available (simulator/web), we compute a pure-math 128-D vector:

For each of 128 dimensions i:
  target_idx = (i × 3 + 17) mod 468  [deterministic landmark sampling]
  vector[i] = euclidean_distance(landmark[target_idx], nose_tip)

result = L2_normalize(vector)

This approach is:

Zero hardware dependency (pure JS)
Deterministic for same face geometry
Fast (<1ms computation)

4. Multi-Angle Master Embedding

During 6-step enrollment, we build a weighted composite:

master = L2_normalize(
  sum(
    embedding[LOOK_CENTER] × 2.0  +  // center gets 2× weight
    embedding[LOOK_UP]     × 1.0  +
    embedding[LOOK_DOWN]   × 1.0  +
    embedding[TURN_LEFT]   × 1.0  +
    embedding[TURN_RIGHT]  × 1.0  +
    embedding[TILT_LEFT]   × 1.0
  ) / 8.0
)

This master embedding is what gets compared during daily verification, dramatically improving recognition recall for non-frontal poses and diverse lighting conditions.

Combined AI Footprint

| Model | Size | |---|---| | face_mesh.tflite | 1.18 MB | | mobile_facenet.tflite | 4.99 MB | | Total | 6.17 MB |

✅ 69% under the 20 MB hackathon constraint — leaving 13.8 MB headroom for app updates.

Liveness Detection

All anti-spoofing is performed purely offline using geometric math — no ML model needed.

Challenge Pool

Pool = { BLINK, SMILE, TURN_LEFT, TURN_RIGHT }

Each verification session:
  1. Fisher-Yates shuffle the entire pool
  2. Pick first 3 challenges
  → 24 possible orderings, attacker cannot predict the sequence

BLINK — Eye Aspect Ratio (EAR)

EAR = (|p2-p6| + |p3-p5|) / (2 × |p1-p4|)

Using MediaPipe landmarks:
  LEFT_EAR  = EAR(362, 385, 387, 263, 373, 380)
  RIGHT_EAR = EAR(33,  160, 158, 133, 153, 144)
  FINAL_EAR = (LEFT_EAR + RIGHT_EAR) / 2

Blink detected when: EAR < baseline_EAR × 0.75

SMILE — Mouth Aspect Ratio (MAR)

MAR = mouth_width / mouth_height × 0.8

Smile detected when: MAR > baseline_MAR + 0.12

HEAD TURN — Yaw Angle (Euler Estimation)

Yaw estimation from nose-bridge landmarks:
  Using points: [168, 6, 197, 195, 5, 4, 1, 19, 94, 2]

TURN_LEFT  detected when: Yaw > +18°
TURN_RIGHT detected when: Yaw < -18°

Calibration

The first 20 frames automatically calibrate the EAR/MAR baseline per-user, adapting to:

Eye shape variations across Indian demographics
Glasses
Distance from camera

Enrollment Wizard

Phone-style 6-step guided enrollment (mirrors iPhone Face ID):

Step 1: LOOK_CENTER   → Face straight, Yaw <10°, Pitch <10°
Step 2: LOOK_UP       → Pitch >14°
Step 3: LOOK_DOWN     → Pitch <-14°
Step 4: TURN_LEFT     → Yaw >18°
Step 5: TURN_RIGHT    → Yaw <-18°
Step 6: TILT_LEFT     → Roll >11°

Each step requires 20 consecutive stable frames before auto-capturing. This eliminates blur and ensures the captured embedding represents a clean, stable head pose.

Performance Benchmarks

Measured on web browser (Chrome, M1 Mac). Mobile figures based on TFLite NNAPI benchmarks.

| Stage | Web (Chrome) | Mobile Estimate | |---|---|---| | CLAHE pre-processing | 4–8 ms | N/A (MediaPipe handles internally) | | MediaPipe Face Mesh | 20–35 ms | 15–25 ms (NNAPI) | | Geometry signature generation | <1 ms | <1 ms | | Multi-angle vector search (10k) | 5–10 ms | 5–10 ms | | Total end-to-end pipeline | ~80–150 ms | ~50–100 ms |

✅ Well under the 1000 ms (1 second) hackathon constraint — even on mid-range devices. The real-time performance benchmark panel in the dashboard displays live measured values.

Offline-to-AWS Sync & Purge

Implemented in src/services/syncManager.ts:

On network restore (NetInfo listener fires):
  1. Run SHA-256 integrity check on entire local ledger
     → If tampered: BLOCK sync, alert administrator
  2. Batch POST all verified audit logs to AWS API Gateway
     → Target: https://api.datalake3.nhai.gov/v1/sync
  3. On HTTP 200 success:
     → Purge records older than 48 hours (TTL)
     → Retain recent records for continuity

48-hour TTL ensures local storage never bloats on devices doing daily check-ins.

Integrity gating ensures tampered data (malicious edits to attendance records offline) is detected and blocked before reaching the cloud.

Security — Cryptographic Ledger

Every authentication event creates an immutable SHA-256 blockchain block:

block = {
  id:        "TX-{timestamp}-{random}",
  timestamp:  epoch_ms,
  userId:     matched_user_id,
  latitude:   GPS_lat,
  longitude:  GPS_lon,
  confidence: similarity_score,
  status:     "VERIFIED" | "SPOOF_DETECTED" | "FAILED",
  prevHash:   sha256(previous_block),
  hash:       sha256(prevHash + timestamp + userId + lat + lon + confidence + status)
}

The SHA-256 implementation is 100% pure TypeScript — zero external dependencies. verifyLedgerIntegrity() re-computes every block hash to detect any offline tampering.

Integration Guide

Adding to existing Datalake 3.0 React Native project

Step 1: Copy services

cp -r garuda/src/services/ datalake3/src/services/face/

Step 2: Copy CameraScanner screen

cp garuda/src/components/CameraScanner.tsx datalake3/src/screens/FaceAuth/

Step 3: Add TFLite model files

# Android
cp garuda/src/assets/models/*.tflite \
   datalake3/android/app/src/main/assets/

# iOS
# Add both .tflite files to Xcode project → "Copy Bundle Resources"

Step 4: Add dependencies to package.json

{
  "react-native-fast-tflite": "^3.0.1",
  "react-native-vision-camera": "^5.0.11",
  "@react-native-community/netinfo": "^12.0.1"
}

Step 5: Link native modules

cd datalake3/ios && pod install

Step 6: Add screen to navigator

// In your existing Datalake 3.0 navigation stack
import { CameraScanner } from './src/screens/FaceAuth/CameraScanner';

<Stack.Screen
  name="FaceAuthentication"
  component={CameraScanner}
  options={{ headerShown: false }}
/>

Step 7: Navigate to it from existing Attendance screen

// Replace existing Aadhaar auth call with:
navigation.navigate('FaceAuthentication');

Total integration time: ~2 hours No architecture changes to the existing Datalake 3.0 app required.

Open-Source Stack

| Technology | Version | License | Purpose | |---|---|---|---| | React Native | 0.85.3 | MIT | Cross-platform framework | | MediaPipe Face Mesh | 0.4.x | Apache 2.0 | 468-point 3D face detection | | MobileFaceNet TFLite | — | MIT | 128-D face embedding | | react-native-fast-tflite | 3.0.1 | MIT | On-device TFLite inference | | react-native-vision-camera | 5.0.11 | MIT | Camera frame processing | | @react-native-async-storage | 3.1.1 | MIT | Local offline database | | @react-native-community/netinfo | 12.0.1 | MIT | Network state monitoring | | SHA-256 (pure JS) | — | Zero dependency | Cryptographic ledger hashing | | CLAHE Processor (pure TS) | — | Zero dependency | Lighting normalization |

✅ Zero proprietary dependencies. No additional licenses required.

Build & Run

Web Dashboard (for demo)

npm install
npm run web
# Opens on http://localhost:5173

React Native (Mobile)

npm install

# Android
npm run android

# iOS
cd ios && pod install && cd ..
npm run ios

Model files location

src/assets/models/
  ├── face_mesh.tflite        (1.18 MB)
  └── mobile_facenet.tflite   (4.99 MB)

Android: Place both files in android/app/src/main/assets/ iOS: Add both to Xcode → target → "Copy Bundle Resources"

Evaluation Criteria Alignment

| Criterion | Max | Our Implementation | |---|---|---| | Innovation | 30 | 6-step enrollment, CLAHE, randomized 4-pool liveness, SHA-256 blockchain, 6.17 MB total model size | | Feasibility | 30 | Pure React Native, drop-in 2-hour integration, ~80ms end-to-end, works on Android 8.0+ / iOS 12+ | | Scalability | 20 | Multi-angle model improves accuracy at scale, integrity-gated AWS sync, 48h TTL auto-purge | | Documentation | 20 | This README + inline JSDoc on all services + live benchmark dashboard |

Built for NHAI Hackathon 7.0 — Submission deadline: 05 June 2026 Contact: [email protected]