proctor-vision

Calibration-free, client-side interview/exam proctoring for the browser — eye-gaze look-away, head-pose, multiple-person and device detection. Built on MediaPipe (Apache-2.0). Framework-agnostic, TypeScript-first, zero server-side video.

• 👁️ Eye look-away — calibration-free (auto-baseline), with hysteresis + smoothing to avoid false flags
• 🙂 Head-turn — absolute 3D head pose (no calibration)
• 👥 Multiple people — including a partially-visible second person
• 📱 Devices — phone / book (foreign objects only; your own laptop won't false-flag)
• ⏱️ Per-episode durations + one-shot prolonged alerts (e.g. 5s+)
• 📄 JSON evidence report for human review
• 🧩 Works with any framework (React, Vue, Angular, vanilla) in TS or JS

Privacy: all video is processed in the candidate's browser. Nothing is uploaded.

Install

npm i proctor-vision @mediapipe/tasks-vision

@mediapipe/tasks-vision is a peer dependency (you control the version).

Quick start

import { createProctor } from "proctor-vision";

const proctor = createProctor({
  // Uniform per-detector shape: { enabled, sensitivity } (+ optional message/prolongedMs/watch).
  // sensitivity is 0..1 — HIGHER = MORE SENSITIVE (flags more readily). One dial per detector.
  features: {
    faceDetection:  { enabled: true, sensitivity: 0.5 },
    eyeGaze:        { enabled: true, sensitivity: 0.5, prolongedMs: 5000, message: "Please look at the screen" },
    headMovement:   { enabled: true, sensitivity: 0.5, prolongedMs: 5000, message: "Please face the screen" },
    multiplePerson: { enabled: true, sensitivity: 0.6, message: "Another person detected" },
    device:         { enabled: true, sensitivity: 0.7, watch: ["cell phone", "book"], prolongedMs: 5000 },
  },
});

proctor.on("violation",    e => console.log("start", e.type, e.direction, e.message));
proctor.on("prolonged",    e => showBanner(e.message));           // fired at prolongedMs
proctor.on("violationEnd", e => console.log("end", e.type, e.durationMs, e.prolonged));
proctor.on("state",        s => renderHud(s));                    // per-frame live signals

await proctor.start(videoElement);   // an HTMLVideoElement or a MediaStream

// later…
const report = proctor.getReport();  // { summary, episodes[] } — proctoring evidence
proctor.stop();

Optional drop-in UI

import { mountDebugUI } from "proctor-vision/ui";
const ui = mountDebugUI(document.body, proctor);   // status pill + live panel + event log + toast
// ui.destroy() to remove

Or ignore it and build your own from the state / violation / prolonged events.

Configuration

Every detector shares the same shape — enabled + sensitivity (0..1, higher = more sensitive) — plus optional extras. One dial per detector; the SDK maps it to the right internal threshold.

| Detector | sensitivity controls | Extra knobs | Default sensitivity | |---|---|---|---| | faceDetection | how eagerly a face is detected (→ min detection confidence) | message | 0.5 | | eyeGaze | how small a glance off-screen counts | prolongedMs, message | 0.5 | | headMovement | how small a head turn counts (→ degrees) | prolongedMs, message | 0.5 | | multiplePerson | how partial a 2nd person counts (→ confidence) | message | 0.6 | | device | how partial a device counts (→ confidence) | watch[], prolongedMs, message | 0.7 |

All detectors have enabled (default true) — set false to turn one off.

Global: autoBaseline (default true), smoothing (0.25), headSmoothing (0.4), objectDetectIntervalMs (400), maxFaces (3), debug (false), modelBaseUrl (self-host models).

Live-tune anytime: proctor.configure({ features: { device: { sensitivity: 0.9, enabled: true } } }).

Best-use guidance (environment)

For the most reliable gaze/head detection, tell candidates:

| Factor | Recommendation | Why | |---|---|---| | Distance | ~50–60 cm (20–24 in); acceptable 40–75 cm | Iris needs enough pixels for gaze; too close clips the mesh on head turns | | Framing | Face ≈ ⅓ of the frame, whole head + shoulders, centered | Device-independent way to state distance (laptop cam or external) | | Camera height | At eye level (±15°) | A low webcam reads "looking down" as neutral and skews gaze | | Lighting | Even, front-facing; avoid backlight (window behind) | Backlight silhouettes the face and breaks landmark detection | | Glasses | ✅ Fully supported — just avoid strong lens glare | See below |

Glasses / spectacles

Works with glasses out of the box: MediaPipe's face model is trained on faces with glasses, head-pose is computed from facial geometry (unaffected by lenses), and the auto-baseline learns the wearer's neutral gaze as-is. The only degrader is strong glare/reflection on the lenses (a light or window bouncing off the glass) washing out the iris for gaze — mitigate with anti-glare/matte lenses or by tilting the screen/light so reflections miss the camera. Head-pose, multiple-person and device detection are unaffected by glasses regardless.

Events

| Event | Payload | When | |---|---|---| | violation | { type, direction?, message, startedAt, detail? } | an episode begins (after debounce) | | prolonged | { type, direction?, message, thresholdMs, startedAt } | episode reaches prolongedMs | | violationEnd | { type, durationMs, prolonged, startedAt, endedAt } | episode ends | | state | ProctorState | every processed frame | | started / stopped / error | — / — / Error | lifecycle |

How it works (and its limits)

• Calibration-free via auto-baseline: it learns the candidate's neutral gaze from their own attention, only while they're facing the screen and not blinking, and freezes during a look-away so a sustained glance can't poison it.
• Robustness: EMA smoothing + enter/exit hysteresis + episode debounce → stable near the noise floor.
• Honest limits: a fully-hidden phone can't be seen by any webcam detector — the sustained eyes/head-down signal is the behavioral backstop. Treat output as evidence for human review, not an automatic verdict.

Framework notes

• React/Next: create the proctor in a useEffect, pass your <video> ref, clean up with stop(). See examples/react.md.
• Vanilla: see examples/vanilla.html.
• Requires HTTPS or localhost (browser rule for camera access).

Browser support

Works on Chromium browsers (Chrome, Edge, Brave), Safari and Firefox. MediaPipe's GPU (WebGL) delegate is used when available; on browsers where it fails (notably iOS Safari, and Firefox) the library transparently falls back to the CPU delegate — same detections, lower frame rate. The active delegate is reported on the detectors as delegate: "GPU" | "CPU".

License

MIT © Arpanoob