react-native-pose-detection

Real-time human pose detection for React Native and Expo.

33 body landmarks per frame, detected and drawn entirely in the native layer, powered by MediaPipe. Works in Expo and bare React Native projects alike. Nothing crosses the bridge until you ask.

Installation · Quick start · Do more · Full surface · Example · Docs

Snaps from the example app

Why this one

• One component. <PoseCamera /> opens the camera, finds the body, draws the skeleton. Every default overridable, none required.
• Both install paths, first class. The Expo config plugin and a CLI for bare React Native do the same job; both are built and tested in CI on every commit.
• Zero bridge traffic by default. Detection, smoothing, drawing and trigger logic run natively. Landmarks cross to JavaScript only when you opt in, as one zero-copy buffer.
• Tunes itself to the phone. Measures inference cost, converges on the fastest sustainable frame rate, backs off with heat, remembers the answer for the next launch.
• Native triggers. Declare "knee bent past 90 degrees for 300 ms", get one event when it happens. Thirty reps is thirty bridge crossings, not nine hundred.
• Files too. Landmarks from any photo or video on disk, or a full-quality painted copy, without slowing the live camera.
• Zero runtime dependencies. Peers are expo, react, react-native. No VisionCamera, no Reanimated, no worklets.
• Models handled for you. Downloaded, checksum-verified and installed at build time. A mismatch fails the build, never warns.

Installation

One package, two setups. Both end in the same place: the model inside your native projects and the camera permission declared.

Expo

npx expo install react-native-pose-detection

In app.json, add the config plugin:

{
  "expo": {
    "plugins": [
      [
        "react-native-pose-detection",
        {
          "model": "full", // 'lite' | 'full' | 'heavy'
          "cameraPermissionText": "We use the camera to analyze your movement."
        }
      ]
    ]
  }
}

npx expo prebuild

The plugin installs the model into both native projects and writes the camera permission into Info.plist and AndroidManifest.xml for you. Nothing downloads at runtime.

Expo Go is not supported: this package contains native code, so use a development build.

Bare React Native

npm i react-native-pose-detection
npx react-native-pose-detection fetch-model full

The CLI installs the model into both native projects. Declare the camera permission yourself, in ios/<YourApp>/Info.plist:

<key>NSCameraUsageDescription</key>
<string>We use the camera to analyze your movement.</string>

and in android/app/src/main/AndroidManifest.xml:

<uses-permission android:name="android.permission.CAMERA" />

Either setup can be verified with npx react-native-pose-detection doctor, which checks the install and names anything missing. Full detail, including EAS and release builds: installation guide.

Choosing a model

Exactly one ships, whichever you pick. Changing it is one word in the config plus a rebuild.

| Model | Best for | | --- | --- | | lite | budget devices, the highest frame rates | | full (default) | most apps: the accuracy and cost balance | | heavy | accuracy-critical work on flagship hardware |

Sizes and the full trade-off table: app size.

Quick start

App.tsx

import { PoseCamera, useCameraPermission } from 'react-native-pose-detection';

export default function App() {
  const permission = useCameraPermission();
  if (!permission.granted) return null;

  return <PoseCamera style={{ flex: 1 }} />;
}

That is a live camera with a tracked skeleton, tuned to the device, zero bridge traffic.

Do more

Count reps without streaming a single coordinate. The condition runs on the camera thread; you hear about it once per rep:

<PoseCamera
  triggers={[
    {
      id: 'squat',
      enter: { angle: 'leftKnee', below: 90 },
      exit: { angle: 'leftKnee', above: 160 },
      emit: 'cycle',
      debounceMs: 300,
    },
  ]}
  onTrigger={(e) => setReps(e.count)}
/>

Read landmarks when you actually want them, as typed arrays from one shared buffer:

<PoseCamera
  data={{ mode: 'throttled', throttleMs: 100, angles: ['leftKnee'] }}
  onPose={(frame) => {
    // frame.landmarks is a Float32Array of [x, y, z, visibility] per joint
    setKneeAngle(frame.angles?.leftKnee);
  }}
/>

Paint a photo or video into a full-quality copy, without slowing the live camera:

import { exportPose } from 'react-native-pose-detection';

const { uri } = await exportPose(videoUri, { directory: 'documents' }).result;

It tunes itself

No frame-rate table to maintain. The package measures what inference costs on each phone, converges on the fastest rate that phone sustains, steps down with heat, and caches the answer:

await cam.current.getProfile();
// { phase: 'settled', tier: 'high',
//   resolved: { delegate: 'GPU', targetFps: 34, preview: '1080p', analysis: '480p' },
//   p50InferenceMs: 16.2, measuredFps: 33 }

Every axis is still yours: profile, targetFps, resolution, analysisResolution, delegate, thermalPolicy.

The whole surface at a glance

Every prop on one component. All of them optional; an explicit value pins that axis and the rest stay automatic.

<PoseCamera
  ref={cam}
  style={{ flex: 1 }}
  // camera
  facing="front"                    // 'auto' | 'front' | 'back'
  active={isFocused}                // the whole session on/off
  detection={true}                  // inference on/off; false frees the model
  resolution="auto"                 // preview: '480p' | '720p' | '1080p'
  // detection
  maxPoses={1}                      // 1 to 5
  minConfidence={0.6}               // what counts as a body
  smoothing={{ minCutoff: 1, beta: 4 }}
  // performance
  profile="auto"                    // 'efficient' | 'balanced' | 'quality' | 'unrestricted'
  targetFps="auto"                  // a number pins the rate
  analysisResolution="auto"         // what the model sees: '360p' | '480p' | '720p'
  delegate="auto"                   // 'gpu' | 'cpu'
  thermalPolicy="adaptive"          // 'critical-only' | 'off'
  // drawing, all native
  overlay={{
    color: '#00E5FF',
    lineWidth: 3,
    pointRadius: 4,
    angles: [{ joint: 'leftKnee' }, { joint: 'rightKnee' }],
  }}
  // data out, off unless asked
  data={{ mode: 'throttled', throttleMs: 100, select: ['leftKnee', 'rightKnee'] }}
  triggers={[squatTrigger]}
  logLevel="off"
  // events
  onReady={(e) => console.log(e.delegate, e.targetFps)}
  onError={(e) => console.warn(e.code, e.message)}
  onCameraChange={(e) => setFacing(e.facing)}
  onPerformanceChange={(e) => console.log(e.reason, e.targetFps)}
  onTrigger={(e) => setReps(e.count)}
  onPose={(frame) => setFrame(frame)}
  onLog={(entries) => entries.forEach((e) => console.log(e.message))}
/>

| Prop | Default | What it does | | --- | --- | --- | | style | none | View style; { flex: 1 } is the usual answer | | facing | 'auto' | Which lens, 'front' or 'back'; auto prefers front | | active | true | Camera session on/off | | detection | true | Inference on/off; false frees GPU memory | | overlay | true | The skeleton; boolean or a config object | | smoothing | true | One-Euro filter; boolean or { minCutoff, beta } | | maxPoses | 1 | Detection ceiling, 1 to 5 | | minConfidence | unset = auto | What counts as a body, 0.1 to 1; unset follows maxPoses: 0.6 for one person, 0.3 above | | profile | 'auto' | Performance envelope: 'efficient' 'balanced' 'quality' 'unrestricted' | | targetFps | 'auto' | Inference rate; a number pins it | | resolution | 'auto' | Preview: '480p' '720p' '1080p' | | analysisResolution | 'auto' | What the model sees: '360p' '480p' '720p' | | delegate | 'auto' | Inference engine, 'gpu' or 'cpu'; auto probes and falls back | | thermalPolicy | 'adaptive' | Heat response: 'critical-only' or 'off'; off never stops reporting | | data | { mode: 'off' } | What crosses to JavaScript: 'throttled' 'batched' 'live' | | triggers | [] | Native conditions, validated at render | | logLevel | 'off' | Diagnostics, 'error' through 'trace', global or per category | | onReady … onLog | none | Callbacks: lifecycle, errors, performance, triggers, frames, logs |

Exact types, clamping rules and edge behavior: <PoseCamera> reference.

Requirements

| | Minimum | | --- | --- | | React Native | 0.74 | | Expo SDK | 51 | | iOS | 15.1 | | Android | API 24 |

Expo Go cannot run native code, so use a development build. The JavaScript itself is 62.5 KB with zero runtime dependencies.

Documentation

| Guide | Covers | | --- | --- | | Getting started | Install, first camera, first data | | Installation | Expo, bare RN, EAS, release builds | | Camera control | Lenses, switching, pausing, lifecycle | | Data delivery | Modes, the wire format, retention | | Triggers | Conditions, phases, snapshots | | Photos and video files | Landmarks from files, painted copies | | Performance | Profiles, the governor, thermal, app size | | What you can build | Trigger syntax, feasibility, limits | | API reference | Every prop, method, event, type, error code | | Troubleshooting | Real problems, and the log channel |

The example app shows all of it running: a live camera with every prop on a panel, a studio that paints picked files, and a diagnostics screen with stress scenarios. It exists twice, once per install path, so both stay proven end to end.

Contributing

Issues and PRs are welcome, especially device reports from hardware we have not measured. Start with contributing.

License

MIT © khalid999devs