[!NOTE] This is one of 214 standalone projects, maintained as part of the @thi.ng/umbrella monorepo and anti-framework.

🚀 Please help me to work full-time on these projects by sponsoring me on GitHub. Thank you! ❤️

• About
  • Alogrithm description
• Status
• Installation
• Dependencies
• Usage examples
• API
• Authors
• License

About

Naive, lightweight CPU-based dense optical flow implementation. This is a support package for @thi.ng/pixel.

This package is a TypeScript port of an old 2008 Processing project, implementing a homemade Optical Flow approach created for a gestural user interface system for Nokia retail stores.

The algorithm is only single channel and not very scalable (in terms of image resolutions), but provides good & stable results for its intended purposes, and without requiring a massive 12MB dependency like OpenCV. YMMV... Using the default config, a 320x240 frame takes ~10ms to process on a MBA M1 2021.

A short 40 second video demonstration (from 2008), first showing the low-res flow field and averaged movement vector, then adding a 3D cube being rotated via hand movments:

https://github.com/user-attachments/assets/7929dee1-b78e-4fd2-8756-129b5db14c58

Alogrithm description

The algorithm requires a previous and current frame. The flow field is obtained by sampling the current frame at a given step distance. For each of these sample/grid positions a kernel window (of 2*windowSize+1 pixels) is being swept/applied to compute the differences to the previous frame. To compute these differences, the previous frame is offset multiple times in both X/Y directions within the [-displace, +displace) interval. The kernel computes the summed difference for each of these displaced window regions and selects the window with the minimum or maximum change (depending on mode). The relative (displacement) position of that selected window is then used as the flow vector for that cell, which will then be linearly interpolated to apply temporal smoothing of the field (configurable) and minimize jittering.

Status

BETA - possibly breaking changes forthcoming

Search or submit any issues for this package

Installation

yarn add @thi.ng/pixel-flow

ESM import:

import * as pf from "@thi.ng/pixel-flow";

Browser ESM import:

<script type="module" src="https://esm.run/@thi.ng/pixel-flow"></script>

JSDelivr documentation

For Node.js REPL:

const pf = await import("@thi.ng/pixel-flow");

Package sizes (brotli'd, pre-treeshake): ESM: 851 bytes

Dependencies

• @thi.ng/api
• @thi.ng/pixel

Note: @thi.ng/api is in most cases a type-only import (not used at runtime)

Usage examples

One project in this repo's /examples directory is using this package:

| Screenshot | Description | Live demo | Source | |:---------------------------------------------------------------------------------------------------------------------|:-------------------------------------------------|:---------------------------------------------------|:--------------------------------------------------------------------------------| | | Optical flow analysis of web cam or video inputs | Demo | Source |

API

Generated API docs

TODO

Authors

• Karsten Schmidt

If this project contributes to an academic publication, please cite it as:

@misc{thing-pixel-flow,
  title = "@thi.ng/pixel-flow",
  author = "Karsten Schmidt",
  note = "https://thi.ng/pixel-flow",
  year = 2008
}

License

© 2008 - 2026 Karsten Schmidt // Apache License 2.0