@fabi-play/motion

Body movements from pose landmarks, as Fabi Play reads them: jumps, squats, claps, shapes (arms up, T, V, star), lanes, freezing, the distance from the screen, and hands as pointers.

npm install @fabi-play/motion

import { MotionTracker, shapeOf, hands } from '@fabi-play/motion'
import type { Body } from '@fabi-play/protocol'

declare const body: Body // 33 landmarks × (x, y, visibility), in player space
const shape = shapeOf(body) // 'star', 'arms-up', … or null
const { left, right } = hands(body)

System gestures and menus

• armsCrossed(body): the arms crossed in an X, Fabi Play's own gesture (held 1.5 s by the app: pause, exit), so games never get it as a move and shapeOf is null while it is made. It counts in front of the chest (the wrists together on the chest, both seen, at the same height, rising gently from elbows out to the sides, the hands along the forearms, standing: what the pose model reports for crossed arms, which is rarely wrists past each other) and as a big X above the head (each wrist past the middle of the body, the forearms crossing, both arms stretched up). Arms up, folded arms, a self-hug, hands joined or clasped and hands on the head are not it. crossOf(body) says which X: 'chest', 'overhead' or null. The thresholds are in CROSS, tuned on a child's proportions. It reads one frame: when holding it, also want it still (arms waving above the head cross now and then). Measured on the pose model's answers for generated still photos of adults (test/fixtures/x-real-poses.json, points only): the chest X in 7 of 10; none of the poses that must not pause. An X turned away from the camera, or held low with the elbows in, isn't read. No children and no live video went into these figures.
• armOut(body): one arm stretched out sideways at shoulder height, 'left' or 'right' as seen on the screen (player space is a mirror: the side of the stretched wrist), for previous / next in menus. Null when both arms are out (a star, arms wide, also with one drooping) and while the arms are crossed. The thresholds are in ARM_OUT, in arm lengths.
• menuPointer(body): the raised hand as a cursor; Hold and Dwell for holding a gesture or staying on a button.

Games get these movements as events from @fabi-play/sdk; this package is for tools and for the Fabi Play apps. ESM only, types included; depends on @fabi-play/protocol. Docs: https://docs.fabiplay.app.

License

Fabi Play Kit License: free to use, for making games and tools for Fabi Play only. See LICENSE.