MotionOS Core

Add AI movement correction to your app in 5 lines.

MotionOS is a browser-native movement intelligence SDK. It turns any device camera into a real-time exercise coach — detecting squat depth, forward lean, knee valgus, and more — with no server, no cloud processing, and no specialised hardware.

const motionOS = new MotionOS({ exercise: 'squat_barbell' })
motionOS.mount(videoElement)
motionOS.on('rep_completed', (rep) => console.log(rep))
motionOS.on('correction',   (cue) => console.log(cue))
motionOS.start()

How it works

Camera → MoveNet (17 keypoints) → Phase Classifier → Deviation Scorer → Correction Engine
  L0          L1                       L2                   L3                  L5

| Layer | Responsibility | Technology | |-------|---------------|------------| | L0 Capture | Frame quality & resolution gating | Browser MediaDevices API | | L1 Pose | 17-keypoint skeleton at 30 fps | MoveNet Lightning (TF.js) | | L2 Classify | Movement phase + rep counting | Angle-based state machine | | L3 Deviation | Joint angle vs. form envelope | Pure geometry, no ML | | L4 Feedback | Timing, priority, cooldowns | Deterministic rules engine | | L5 Correction | Visual skeleton + voice cues | Canvas 2D + Web Speech API |

Everything runs on-device. No video is sent to any server.

Installation

npm install @motionos/sdk

Requires a modern browser with WebGL support (Chrome 90+, Safari 15+, Firefox 90+).

Supported exercises

| Key | Exercise | Phase detection | Rep counting | |-----|----------|:-:|:-:| | squat_barbell | Barbell Back Squat | ✓ | ✓ | | deadlift_conventional | Conventional Deadlift | ✓ | ✓ | | pushup_standard | Standard Push-up | ✓ | ✓ | | lunge_forward | Forward Lunge | ✓ | ✓ | | plank_standard | Standard Plank | ✓ | — | | pull_up_standard | Pull-up | ✓ | ✓ | | bench_press_standard | Bench Press | ✓ | ✓ | | overhead_press_standard | Overhead Press | ✓ | ✓ | | romanian_deadlift | Romanian Deadlift | ✓ | ✓ | | hip_thrust_standard | Hip Thrust | ✓ | ✓ |

Integration examples

Plain JavaScript

<div style="position:relative; width:100vw; height:100vh;">
  <video id="cam" autoplay playsinline muted
         style="width:100%;height:100%;object-fit:cover;transform:scaleX(-1)">
  </video>
</div>

<script type="module">
import { MotionOS } from '@motionos/sdk'

const motionOS = new MotionOS({ exercise: 'squat_barbell' })

motionOS.mount(document.getElementById('cam'))

motionOS.on('ready', () => {
  console.log('Model loaded — inference is live')
})

motionOS.on('rep_completed', ({ count, formScore }) => {
  document.getElementById('reps').textContent  = count
  document.getElementById('score').textContent = formScore
})

motionOS.on('correction', ({ message }) => {
  document.getElementById('banner').textContent = message
})

motionOS.on('error', (err) => {
  console.error('MotionOS error:', err.message)
})

await motionOS.start()
</script>

React

import { useEffect, useRef, useState } from 'react'
import { MotionOS } from '@motionos/sdk'

export function SquatCoach() {
  const videoRef = useRef<HTMLVideoElement>(null)
  const [reps,      setReps]      = useState(0)
  const [formScore, setFormScore] = useState(100)
  const [cue,       setCue]       = useState<string | null>(null)

  useEffect(() => {
    const motionOS = new MotionOS({ exercise: 'squat_barbell' })

    if (videoRef.current) {
      motionOS.mount(videoRef.current)
    }

    motionOS
      .on('rep_completed', ({ count, formScore }) => {
        setReps(count)
        setFormScore(formScore)
      })
      .on('correction', ({ message }) => {
        setCue(message)
        setTimeout(() => setCue(null), 3000)
      })
      .on('form_score', (score) => setFormScore(score))

    motionOS.start().catch(console.error)

    return () => motionOS.dispose()
  }, [])

  return (
    <div style={{ position: 'relative', width: '100vw', height: '100vh' }}>
      <video
        ref={videoRef}
        autoPlay playsInline muted
        style={{ width: '100%', height: '100%', objectFit: 'cover', transform: 'scaleX(-1)' }}
      />

      <div style={{ position: 'absolute', top: 24, left: 16 }}>
        <strong>FORM</strong> {formScore}
      </div>
      <div style={{ position: 'absolute', top: 24, right: 16 }}>
        <strong>REPS</strong> {reps}
      </div>

      {cue && (
        <div style={{ position: 'absolute', bottom: 32, left: '50%', transform: 'translateX(-50%)',
                      background: 'rgba(255,51,102,0.9)', color: '#fff', borderRadius: 24,
                      padding: '12px 24px', fontWeight: 600 }}>
          {cue}
        </div>
      )}
    </div>
  )
}

iOS (Swift + WKWebView)

Embed the MotionOS web SDK inside a WKWebView with camera permission bridged from the native layer.

import WebKit

class MotionOSViewController: UIViewController, WKUIDelegate {

    var webView: WKWebView!

    override func loadView() {
        let config = WKWebViewConfiguration()
        // Allow inline media playback (required for getUserMedia)
        config.allowsInlineMediaPlayback = true
        config.mediaTypesRequiringUserActionForPlayback = []

        webView = WKWebView(frame: .zero, configuration: config)
        webView.uiDelegate = self
        view = webView
    }

    override func viewDidLoad() {
        super.viewDidLoad()
        // Load your MotionOS web app (local or remote)
        let url = URL(string: "https://your-app.com/squat-coach")!
        webView.load(URLRequest(url: url))
    }

    // Bridge camera permission prompt to native iOS dialog
    func webView(_ webView: WKWebView,
                 requestMediaCapturePermissionFor origin: WKSecurityOrigin,
                 initiatedByFrame frame: WKFrameInfo,
                 type: WKMediaCaptureType,
                 decisionHandler: @escaping (WKPermissionDecision) -> Void) {
        decisionHandler(.grant)
    }
}

Add NSCameraUsageDescription to Info.plist:

<key>NSCameraUsageDescription</key>
<string>MotionOS uses your camera to analyse exercise form in real time.</string>

SDK Reference

new MotionOS(config)

| Option | Type | Default | Description | |--------|------|---------|-------------| | exercise | MotionOSExercise | — | Required. Exercise key (see table above). | | facingMode | 'user' \| 'environment' | 'user' | Camera direction. |

motionOS.mount(videoElement)

Attaches the SDK to an existing <video> element. Creates and injects a <canvas> overlay inside the video's parent container. Returns this.

motionOS.start()

Opens the camera, lazy-loads TF.js + MoveNet Lightning (~1.9 MB weights), and begins the inference loop. Returns a Promise<void> that resolves when inference is live. Rejects on permission denial or network failure.

motionOS.stop()

Pauses the inference loop. Camera stream stays open. Call start() to resume (skips model loading).

motionOS.dispose()

Tears down everything: stream, model weights, canvas overlay, event listeners. The instance is unusable after dispose().

motionOS.on(event, handler) / motionOS.off(event, handler)

Subscribe/unsubscribe to events. Returns this for chaining.

Events

| Event | Payload | Description | |-------|---------|-------------| | ready | — | Model loaded; inference is live. | | rep_completed | { count, formScore, phase } | Fires at the top of each completed rep. formScore is the average across the rep. | | form_score | number (0–100) | Current frame form score. Fires every frame when the value changes. | | correction | CorrectionInstruction | A form deviation was detected and a correction cue triggered. | | phase_changed | MovementPhase | The detected movement phase changed. | | error | Error | Unrecoverable startup error (permission denied, network failure, etc.). |

Form envelopes

Each exercise is defined by a FormEnvelope — a JSON file mapping joint IDs to acceptable angle ranges per movement phase.

{
  "exercise_id": "squat",
  "version": "1.0.0",
  "nodes": [
    {
      "joint":             "lumbar_spine",
      "phase":             "transition_bottom",
      "min_angle_degrees": 148,
      "max_angle_degrees": 178,
      "critical":          true
    }
  ]
}

Custom envelopes (e.g., for rehabilitation or sport-specific movement) can be loaded at runtime:

import { MotionOS } from '@motionos/sdk'
import myEnvelope from './my_exercise.json'

const motionOS = new MotionOS({ exercise: 'squat_barbell' })
// Override the envelope after construction (advanced usage)
motionOS.loadEnvelope(myEnvelope)

Architecture

src/
├── sdk/
│   └── index.ts                  ← Public entry point (MotionOS class)
├── form-envelopes/
│   ├── squat_barbell.json
│   ├── deadlift_conventional.json
│   ├── pushup_standard.json
│   ├── lunge_forward.json
│   └── plank_standard.json
├── layer0-capture/               ← Frame quality & resolution gating
├── layer1-pose/                  ← MoveNet wrapper
├── layer2-classification/        ← Phase detection + rep counting
├── layer3-deviation/             ← Joint angle scoring
├── layer4-feedback/              ← Timing + priority engine
├── layer5-correction/            ← Canvas renderer + Web Speech
├── layer6-memory/                ← Session history (coming soon)
└── types/
    └── motionos.types.ts

REST API

For server-side video analysis or persistent session storage, see the REST API documentation.

Base URL: https://api.motionos.ai/v1

Key endpoints:

POST /v1/sessions           Create a real-time session (+ WebSocket stream URL)
GET  /v1/sessions/:id       Retrieve session results
GET  /v1/exercises          List exercises + supported phases
POST /v1/analyze            Upload a video file → async session report

Live demo

motionos-core.vercel.app — barbell squat coach, runs entirely in the browser.

Development

npm install
npm run dev          # Vite dev server at localhost:5173
npm run test         # Vitest (677 tests)
npm run typecheck    # tsc --noEmit
npm run build        # Production bundle → dist/

License

MIT