[!NOTE] This is one of 211 standalone projects, maintained as part of the @thi.ng/umbrella monorepo and anti-framework.

🚀 Please help me to work full-time on these projects by sponsoring me on GitHub. Thank you! ❤️

• About
  • Strided convolution & pooling
  • Normal map generation
• Status
• Installation
• Dependencies
• Usage examples
• API
• Authors
• License

About

Extensible bitmap image convolution, kernel presets, normal map & image pyramid generation. This is a support package for @thi.ng/pixel.

This package contains functionality which was previously part of and has been extracted from the @thi.ng/pixel package.

• Convolution w/ arbitrary shaped/sized kernels, pooling, striding
• Convolution kernel & pooling kernels presets
  • Higher order kernel generators (Gaussian, Lanczos)
• Image pooling filters (min/max, mean, adaptive threshold, custom)
• Image pyramid generation (w/ customizable kernels)
• Customizable normal map generation (i.e. X/Y gradients plus static Z component)

Strided convolution & pooling

Floating point buffers can be processed using arbitrary convolution kernels. The following convolution kernel presets are provided for convenience:

| Kernel | Size | |------------------|-------------| | BOX_BLUR3 | 3x3 | | BOX_BLUR5 | 5x5 | | EDGE3 | 3x3 | | EDGE5 | 5x5 | | GAUSSIAN_BLUR3 | 3x3 | | GAUSSIAN_BLUR5 | 5x5 | | GAUSSIAN(n) | 2n+1 x 2n+1 | | HIGHPASS3 | 3x3 | | LANCZOS(a,s) | as+1 x as+1 | | SHARPEN3 | 3x3 | | SOBEL_X | 3x3 | | SOBEL_Y | 3x3 | | UNSHARP_MASK5 | 5x5 |

Custom kernels can be defined (and code generated) using an array of coefficients and a given kernel size. See above presets and defKernel() for reference.

Furthermore, convolution supports striding (i.e. only processing & keeping every nth pixel column/row, aka downscaling) and pixel pooling (e.g. for ML applications). Available pooling kernel presets (kernel sizes must be configured independently):

| Kernel | Description | |------------------------|--------------------| | POOL_MEAN | Moving average | | POOL_MAX | Local maximum | | POOL_MIN | Local minimum | | POOL_NEAREST | Nearest neighbor | | POOL_THRESHOLD(bias) | Adaptive threshold |

Convolution can be applied to single, multiple or all channels of a FloatBuffer. See convolveChannel() and convolveImage().

See ConvolveOpts for config options.

import { floatBufferFromImage, FLOAT_RGB, imageFromURL } from "@thi.ng/pixel";
import { convolveImage, SOBEL_X } from "@thi.ng/pixel-convolve";

// convolutions are only available for float buffers (for now)
const src = floatBufferFromImage(await imageFromURL("test.jpg"), FLOAT_RGB);

// apply horizontal Sobel kernel preset to all channels
// downscale image by factor 2 (must be integer)
// scale kernel result values by factor 4
const dest = convolveImage(src, { kernel: SOBEL_X, stride: 2, scale: 4 });

Normal map generation

Normal maps can be created via normalMap(). This function uses an adjustable convolution kernel size to control gradient smoothness & details. Result X/Y gradients can also be scaled (uniform or anisotropic) and the Z component can be customized to (default: 1.0). The resulting image is in FLOAT_NORMAL format, using signed channel values. This channel format is auto-translating these into unsigned values when the image is converted into an integer format.

| Step | Scale = 1 | Scale = 2 | Scale = 4 | Scale = 8 | |------|------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------| | 0 | | | | | | 1 | | | | | | 2 | | | | | | 3 | | | | |

import { ARGB8888, FLOAT_GRAY, floatBufferFromImage, imageFromURL } from "@thi.ng/pixel";
import { normalMap } from "@thi.ng/pixel-convolve";

// read source image into a single channel floating point buffer
const src = floatBufferFromImage(await imageFromURL("noise.png"), FLOAT_GRAY);

// create normal map (w/ default options)
// this results in a new float pixel buffer with FLOAT_RGB format
const nmap = normalMap(src, { step: 0, scale: 1 });

// pixel lookup (vectors are stored _un_normalized)
nmap.getAt(10, 10);
// Float32Array(3) [ -0.019607841968536377, -0.04313725233078003, 1 ]

// convert to 32bit packed int format
const nmapARGB = nmap.as(ARGB8888);

Status

STABLE - used in production

Search or submit any issues for this package

Installation

yarn add @thi.ng/pixel-convolve

ESM import:

import * as pc from "@thi.ng/pixel-convolve";

Browser ESM import:

<script type="module" src="https://esm.run/@thi.ng/pixel-convolve"></script>

JSDelivr documentation

For Node.js REPL:

const pc = await import("@thi.ng/pixel-convolve");

Package sizes (brotli'd, pre-treeshake): ESM: 2.29 KB

Dependencies

• @thi.ng/api
• @thi.ng/checks
• @thi.ng/errors
• @thi.ng/math
• @thi.ng/pixel

Note: @thi.ng/api is in most cases a type-only import (not used at runtime)

Usage examples

Three projects in this repo's /examples directory are using this package:

| Screenshot | Description | Live demo | Source | |:--------------------------------------------------------------------------------------------------------------------------|:----------------------------------------------------------------|:---------------------------------------------------------|:--------------------------------------------------------------------------------------| | | Interactive image processing (adaptive threshold) | Demo | Source | | | 2.5D hidden line visualization of digital elevation files (DEM) | Demo | Source | | | Normal map creation/conversion basics | Demo | Source |

API

Generated API docs

Authors

• Karsten Schmidt

If this project contributes to an academic publication, please cite it as:

@misc{thing-pixel-convolve,
  title = "@thi.ng/pixel-convolve",
  author = "Karsten Schmidt",
  note = "https://thi.ng/pixel-convolve",
  year = 2021
}

License

© 2021 - 2025 Karsten Schmidt // Apache License 2.0