expo-mediapipe-pose

On-device pose detection for Expo and React Native, powered by Google's MediaPipe Pose Landmarker. Integrates camera, photo and video inference with customizable skeletons, named landmarks and developer-defined feedback.

Community-maintained; not an official Google or Expo package. Frames stay on the device. The module does not download models or upload camera data.

Who this helps

For Expo and React Native developers building camera interactions, movement visualizations, local photo/video analysis, or feedback driven by named body landmarks. The library handles native inference, coordinates, configurable overlays and event delivery; your app supplies the conditions and user experience.

Features

| Need | API guide | | --- | --- | | Control preview, inference and callback rates; inspect measured metrics | Frame rates and performance | | Measure named angles/distances and react to pose presence | Geometry, tracking feedback | | Evaluate independent rules with hysteresis and joint-specific feedback | Multiple rules | | Discover selectable cameras and frame rates | Camera discovery | | Record/replay bounded landmark sessions | Recording and replay | | Analyze local photos and sample local videos with cancellation | Photo analysis, video analysis | | Select among multiple detected poses | Multiple poses | | Composite opt-in masks with explicit cleanup and backpressure | Segmentation |

Version 0.3.0 includes these features. Styling and rules are configurable without replacing Google's model. See the changelog for release history.

Status and requirements

Version 0.3.0 targets Expo SDK 57, React Native 0.86, and React 19.2. Requires an iOS/Android development build; Expo Go cannot load this native module. Native builds, TypeScript, packaging and automated behavior tests are checked. Physical-device alignment, long-session performance and accuracy comparisons remain release evaluation work; no performance advantage over other wrappers is claimed.

Documentation

• Pose detection in Expo / React Native: installation, permissions, lifecycle and a camera screen.
• Custom skeleton styling: body regions, landmark colors, joint sizes and connection styles.
• Angle-triggered feedback: stable thresholds, red/green/unknown states and transition callbacks.
• API reference: components, hooks, file analysis, coordinates, errors and resource ownership.
• FAQ: platform support, native use, privacy, models, performance and limitations.
• Integration prompt: copyable instructions for your app's coding assistant.
• llms.txt: a concise public documentation index for tools that accept it.
• Runnable example: camera, photo/video analysis and feedback flows.

Install

Install expo-mediapipe-pose from npm with pnpm:

pnpm add expo-mediapipe-pose@^0.3.0 effect@^3.21.4
pnpm exec expo install react-native-svg expo-camera

The package includes compiled JavaScript, TypeScript declarations, native Swift/Kotlin sources and the full pose model. Use the v0.3.0 API for this release; main may advance independently. Native dependency or configuration changes require a new development build. The older v0.2.0 GitHub tag contains the earlier camera API.

Effect validates events at the native boundary. react-native-svg draws the optional overlay. The example uses expo-camera for permissions; another permission provider is also fine.

Configure the permission message in your app config, then rebuild:

{
  "expo": {
    "plugins": [
      ["expo-camera", {
        "cameraPermission": "Allow camera access for on-device pose tracking.",
        "recordAudioAndroid": false
      }]
    ]
  }
}

pnpm exec expo run:ios
# or: pnpm exec expo run:android

Show a pose camera

Mount after permission is granted. The camera fills its layout bounds and draws the skeleton by default:

import { PoseCameraView } from "expo-mediapipe-pose";

<PoseCameraView
  style={{ flex: 1 }}
  cameraFacing="front"
  frameLimit={15}
  onLandmark={frame => handlePose(frame.landmarks)}
  onInferenceError={error => showCameraError(error.code)}
/>

Change cameraFacing to switch cameras. Set isActive={false} when the screen loses focus to release capture while keeping the component mounted. The runnable example includes permissions, camera switching, pause/resume, error recovery and feedback.

Style joints and body parts

<PoseCameraView
  style={{ flex: 1 }}
  skeleton={{
    bodyParts: ["leftArm", "rightArm", "torso"],
    color: "#38bdf8",
    jointRadius: 5,
    lineWidth: 3,
    joints: { leftWrist: { color: "#f59e0b", radius: 8 } },
    connections: { "leftElbow:leftWrist": { color: "#f59e0b" } },
  }}
/>

Body-part selection changes the overlay, not the detector or raw results. Individual overrides take precedence over global styles. Use skeleton={false} to render your own overlay and avoid internal per-frame overlay state updates. Styling does not restart capture.

React to a condition

The application defines the condition. usePoseRule handles confidence checks, hold duration, stale input and transition callbacks:

const feedbackColors = {
  pass: "#22c55e",
  fail: "#ef4444",
  unknown: "#94a3b8",
};

const raisedArm = usePoseRule({
  landmarks: ["leftWrist", "leftShoulder"],
  minVisibility: 0.6,
  holdMs: 250,
  isActive: screenIsFocused,
  evaluate: pose => pose.leftWrist.y < pose.leftShoulder.y,
  onChange: status => handleArmStateChange(status),
});

<PoseCameraView
  style={{ flex: 1 }}
  isActive={screenIsFocused}
  onLandmark={raisedArm.update}
  onCameraConfigured={raisedArm.reset}
  onInferenceError={raisedArm.reset}
  skeleton={{ bodyParts: ["leftArm"], color: feedbackColors[raisedArm.status] }}
/>

Import usePoseRule from the package. Uncertain, missing or stale landmarks produce unknown; that is distinct from a failed condition. Callbacks fire only on transitions. Show text or icons alongside color, as the example does. The hook does not judge exercise form or provide medical interpretation.

For tests or application logic without a native view, import helpers from expo-mediapipe-pose/core:

import { getLandmark } from "expo-mediapipe-pose/core";

const leftWrist = getLandmark(frame, "leftWrist");

Develop

Use Node 24 LTS (nvm use) and the pinned pnpm version for contributor checks, matching CI. This development requirement does not change the library’s mobile runtime contract.

pnpm install --frozen-lockfile
pnpm check
pnpm verify:package
pnpm --filter pose-camera-example ios
# or: pnpm --filter pose-camera-example android

pnpm check runs Biome, TypeScript checks for the package/example, and tests. CI builds both native example applications. The native fixture runner exercises real SDK inference on public images/videos; it runs separately from CI build checks. Use physical phones to assess front/back alignment, all interface orientations, zoom, interruptions, permissions, model changes, repeated mounts and sustained capture. Include device, OS, SDK and model details when reporting bugs through GitHub Issues.

API and scope

See API details for props, coordinate semantics, ownership, errors and migration notes. Public API and native integration explains the supported entry points.

Recordings contain landmarks/metadata, not camera video. Result indices are not persistent person identities. Repetition counting, exercise scoring and medical interpretation remain application responsibilities. Only the full pose model is bundled; lite/heavy require local model files.

Native inference uses Swift with AVFoundation on iOS and Kotlin with CameraX on Android. Detector ownership stays on a serial worker, with camera backpressure and stale-generation rejection. This is an Expo integration, not a replacement pose model.

FAQ

It is a community-maintained Expo integration built on Google's official MediaPipe Tasks SDKs and model. It adds a React Native camera component, typed events, skeleton styling, named geometry, rule/tracking hooks, local-media APIs and explicit mask ownership. It does not introduce a new pose model or claim better detection accuracy than the underlying SDK.

No. The Expo integration uses a different public API: replace RNMediapipe with PoseCameraView, use controlled camera props and configure SkeletonOptions. Follow the migration guide, rebuild the native app and verify your coordinate/overlay handling. Attribution is preserved; no unmeasured speed or accuracy advantage over ThinkSys is claimed.

Camera and file inference require an iOS/Android development build. Expo Go cannot load this custom native module. An existing React Native app needs Expo modules installed and compatible dependencies; the current development target is Expo SDK 57 / React Native 0.86, not a verified compatibility matrix for older versions. There is no browser inference backend. Pure helpers are available through expo-mediapipe-pose/core without initializing React Native.

The supported consumer API is TypeScript for Expo/React Native. The Swift and Kotlin sources implement the Expo bridge; this repository does not publish a standalone Swift Package or independent Android library API. For a fully native app, start with Google's official iOS or Android guide. Native source ownership and extension points are mapped in the API reference.

Install expo-mediapipe-pose@^0.3.0 from npm for the features listed here. The older v0.2.0 GitHub tag provides the earlier camera API. Read the documentation for your installed version; main can advance independently.

Yes. SkeletonOptions supports global, joint and connection styling and body-part selection. Define a condition with usePoseRule, or multiple conditions with usePoseRules, then map their states to colors, text, haptics or app actions. unknown means missing, uncertain or stale input. These are developer-defined conditions, not built-in exercise-form judgments. See styling and rules.

previewFps, frameLimit and callbackFps control capture targets, inference limits and JavaScript delivery independently. Metrics report observed work; no requested FPS is guaranteed. Lower callback FPS does not reduce detector work. Use isActive with screen focus and app foreground state. There is no automatic thermal/model-switching policy or measured battery-saving guarantee. See performance.

The library runs inference locally and does not upload frames or download models. Applications control their own analytics, networking and storage. Landmark recording/replay stores pose data and metadata, not camera video or segmentation files; local-video analysis reads an existing video. See recording.

No. maxPoses supports up to six detections, and poseIndex selects within one result. Indices are not persistent identities. Missing selections remain empty; reset temporal rules/tracking when your selection changes. See multiple poses.

Segmentation is opt-in. At most two result leases can be outstanding per module; each owns bounded PNG mask files. Call releasePoseSegmentation when your consumer finishes, including on errors or when an overlay is replaced/unmounted. The overlay does not release files for you. Already-delivered video masks remain your responsibility after cancellation. See mask ownership.

Only the full pose model is bundled. Lite/heavy require an explicit local model file. Native builds, behavioral tests and public photo/video fixtures verify integration behavior. They do not prove physical-camera alignment, sustained performance, medical accuracy or superiority over another wrapper. Repetition counting, exercise scoring and medical interpretation are not current library features.

Integrate with a coding assistant

Copy the prompt below and replace the bracketed goal. Supply this repository URL and your chosen tag/commit. llms.txt provides a short reading list; the versioned API remains authoritative.

Integrate expo-mediapipe-pose into this app for: [describe the interaction].
Repository: https://github.com/YosefHayim/expo-mediapipe-pose

Inspect this app's instructions, Expo/React Native versions, navigation,
permission handling and existing camera code before changing anything.
Read the README and API documentation for the exact installed tag/commit.
The npm 0.3.0 API includes the current guides; the older v0.2.0 GitHub tag
predates the newer APIs. Check the installed version and compatibility.

Use supported exports from expo-mediapipe-pose; import pure helpers from
expo-mediapipe-pose/core. Do not invent API names or treat the internal
Swift/Kotlin Expo bridge as a separately distributed native SDK.

Implement the smallest complete integration for the stated goal:
- Use a native development build, obtain permission before mounting, and
  connect isActive to screen focus and foreground state.
- Keep preview, inference and callback rates explicit where needed.
  Report measured performance without promising a requested frame rate.
- Preserve named anatomical landmarks, the documented coordinate system,
  and unknown/missing states. Do not mirror coordinates twice or treat
  pose indices as persistent person IDs.
- Enable segmentation only if needed. Release each delivered mask lease
  after use/replacement/unmount, including error and cancellation paths.
- Keep exercise rules application-defined. Do not imply medical accuracy,
  built-in repetition counting, video recording, or unsupported platforms.
- Follow the app's structure. Use descriptive names and early returns;
  avoid nested ternaries, nested conditions and duplicated infrastructure.

Run relevant type/behavior checks and native builds. Exercise the actual
screen and report which devices were used. Separate simulator/fixture
coverage from physical-camera and sustained-performance validation.
Explain the changed files, installation/build steps and remaining limits.

License and attribution

MIT. This project evolved from ThinkSys/mediapipe-reactnative; its attribution is preserved in third-party notices, alongside MediaPipe notices. The native inference SDKs and pose model come from Google. Contributor agent guidance lives in AGENTS.md.