react-opencv-fiber

A React renderer for OpenCV.js. Write image-processing pipelines as JSX — each element maps to an OpenCV operation, and the tree executes bottom-up through a custom React reconciler.

Live demo

<CvCanvas>
  <cvCanny threshold1={50} threshold2={100}>
    <cvCvtColor code={11}>
      <cvGaussianBlur ksize={[5, 5]} sigmaX={0}>
        <cvImage src="photo.jpg" />
      </cvGaussianBlur>
    </cvCvtColor>
  </cvCanny>
</CvCanvas>

How it works

The library uses a custom React Fiber reconciler to build a tree of CvNodes from JSX. When the tree changes (props update, nodes added/removed), the pipeline re-executes:

1. Leaf nodes (<cvImage>) load source images into OpenCV Mat objects
2. Inner nodes (<cvGaussianBlur>, <cvCanny>, ...) receive their child's Mat as input, apply the corresponding cv.* function, and pass the result up
3. <CvCanvas> displays the final Mat on an HTML canvas

Intermediate Mat objects are automatically cleaned up to avoid WebAssembly memory leaks.

API

<OpenCVProvider>

Loads OpenCV.js (WASM, ~8 MB) and provides it via context. Wrap your app with this.

<OpenCVProvider>
  <App />
</OpenCVProvider>

useOpenCV()

const { cv, loading, error } = useOpenCV();

<CvCanvas>

Renders a <canvas> and executes the CV pipeline defined by its children.

<CvCanvas
  style={{ maxWidth: "100%" }}
  className="my-canvas"
  onResult={(mat) => { /* cloned Mat of the final result */ }}
>
  {/* CV operation tree */}
</CvCanvas>

| Prop | Type | Description | |---|---|---| | style | CSSProperties | CSS styles for the canvas element | | className | string | CSS class for the canvas element | | headless | boolean | When true, runs the pipeline without rendering a canvas (returns null). Useful for producing intermediate results via onResult. | | onResult | (mat: Mat) => void | Called with a cloned Mat after each pipeline run. The clone stays valid across re-runs — you own it. | | ref | Ref<HTMLCanvasElement> | Forwarded to the underlying canvas element |

CV operation elements

Any OpenCV function can be used as a JSX element with a cv prefix:

| JSX element | OpenCV function | |---|---| | <cvGaussianBlur ksize={[5,5]} sigmaX={0}> | cv.GaussianBlur(...) | | <cvCanny threshold1={50} threshold2={100}> | cv.Canny(...) | | <cvCvtColor code={11}> | cv.cvtColor(...) | | <cvThreshold thresh={127} maxval={255} type={0}> | cv.threshold(...) | | <cvResize dsize={[320, 240]}> | cv.resize(...) | | <cvMedianBlur ksize={5}> | cv.medianBlur(...) | | <cvBilateralFilter d={9} sigmaColor={75} sigmaSpace={75}> | cv.bilateralFilter(...) |

Operations are nested inside-out — the innermost element runs first:

// Execution order: image load -> blur -> grayscale -> edge detection
<cvCanny threshold1={50} threshold2={100}>
  <cvCvtColor code={11}>
    <cvGaussianBlur ksize={[5, 5]} sigmaX={0}>
      <cvImage src="photo.jpg" />
    </cvGaussianBlur>
  </cvCvtColor>
</cvCanny>

Props map directly to OpenCV function parameters. Array values are coerced to the appropriate OpenCV types (Size, Point, Scalar).

__srcParam / __dstParam

By default, each operation is called as cv.op(src, dst, ...params) — matching the OpenCV.js convention of src at position 0 and dst at position 1. Some functions use a different argument order. Use __srcParam and __dstParam to override the position:

{/* applyColorMap expects cv.applyColorMap(src, colormap, dst) */}
<cvApplyColorMap colormap={2} __dstParam={2}>
  <cvCvtColor code={11}>
    <cvImage src="photo.jpg" />
  </cvCvtColor>
</cvApplyColorMap>

Special leaf elements

These are leaf nodes that provide input to the pipeline (they have no children).

<cvImage>

Loads an image (URL or data URI) into a Mat.

<cvImage src="https://example.com/photo.jpg" />

<cvMat>

Accepts an existing Mat as pipeline input. The mat is cloned internally, so the original stays valid and is never disposed by the pipeline.

<cvMat mat={someMatFromState} />

Use this to share the output of one pipeline as input to another (see Sharing results between pipelines).

Sharing results between pipelines

Use headless mode to run a pipeline without a canvas, capture its result with onResult, and feed it into other pipelines via <cvMat>:

const [blurredMat, setBlurredMat] = useState<Mat | null>(null);

{/* Headless pipeline: blur and share result */}
<CvCanvas headless onResult={setBlurredMat}>
  <cvGaussianBlur ksize={[5, 5]} sigmaX={0}>
    <cvImage src="photo.jpg" />
  </cvGaussianBlur>
</CvCanvas>

{/* Display the blurred intermediate */}
<CvCanvas>
  <cvMat mat={blurredMat} />
</CvCanvas>

{/* Apply further operations on the shared result */}
<CvCanvas>
  <cvCanny threshold1={50} threshold2={150}>
    <cvMat mat={blurredMat} />
  </cvCanny>
</CvCanvas>

Both onResult and <cvMat> clone the Mat, so there are no ownership conflicts — each pipeline manages its own memory.

Examples

The examples/ directory contains interactive demos with different filter combinations. Live demo

To run locally:

npm install
npm run build
cd examples && npm install && npx vite

Acknowledgements

• OpenCV — the computer vision library that powers all the operations
• OpenCV.js — the JavaScript/WebAssembly port of OpenCV
• React Three Fiber — the inspiration for using a custom React reconciler to drive a non-DOM library

License

MIT