salient-autofocus

Auto-focus capabilities based on saliency maps.

Basic Usage

Extract a 640x480 region from a 1024x768 image using salient auto-focus

const autoFocus = require('salient-autofocus');

const salientMatrix = [[0, 0.25, 0.5], [0, 0, 0.25], [0, 0.25, 0.75]]; // custom 3x3 matrix
const region = autoFocus.getRegionFromSalientMatrix(salientMatrix, { imageWidth: 1024, imageHeight: 768, regionWidth: 640, regionHeight: 480 });
// { left, top, right, bottom, width, height }

OpenCV Usage

Extract a 640x480 region from a 1024x768 image using salient auto-focus

const autoFocus = require('salient-autofocus');

const salientMatrix = mySalientMap.getDataAsArray(); // works with opencv or opencv4nodejs packages
const region = autoFocus.getRegionFromSalientMatrix(salientMatrix, { imageWidth: 1024, imageHeight: 768, regionWidth: 640, regionHeight: 480 });
// { left, top, right, bottom, width, height }

Production Usage

The real power of this package is the metadata that it extracts from saliency data, which can be stored or cached, allowing subsequent auto-focus requests to be lightning fast! No need to re-create salient maps, store salient maps, or re-process salient maps.

const autoFocus = require('salient-autofocus');

const salientMatrix = mySalientMap.getDataAsArray(); // works with opencv or opencv4nodejs packages
const salientMeta = autoFocus.getMetaFromSalientMatrix(salientMatrix);
// TODO: store meta for subsequent auto-focus requests
const region = autoFocus.getRegionFromMeta(salientMeta, { imageWidth: 1024, imageHeight: 768, regionWidth: 640, regionHeight: 480 }); // SUPA FAST!
// { left, top, right, bottom, width, height }

Where do I get my salient map?

That's out of the scope of this project as there are many models for generating salient maps based on TONS of research. However you're in luck, because I just happen to author one of those solutions: https://github.com/asilvas/salient-maps

Example Results

The following examples are visual representations of the original image side by side with the salient map, overlayed by the regions of saliency density to illustrate where salient-autofocus will focus on depending on the available size of the region requested. The more accurate the saliency map, the better salient-autofocus will perform.

Region Data

Data returned by getRegionFromSalientMatrix or getRegionFromMeta to indicate the region to focus on.

{ left, top, right, bottom, width, height }

| Property | Type | Info | | --- | --- | --- | | left | number | Left-most edge of region on a 0 to {imageWidth} scale in columns (typically pixels) | | top | number | Top-most edge of region on a 0 to {imageHeight} scale in rows (typically pixels) | | width | number | Width of region on a 0 to {imageWidth} scale in columns (typically pixels) | | height | number | Height of region on a 0 to {imageHeight} scale in rows (typically pixels) | | right | number | Right-most edge of region added for convenience (left+width) | | bottom | number | Bottom-most edge of region added for convenience (top+height) |

Meta Data

If extracting meta from salient data via getMetaFromSalientMatrix (as you should for production usage), below is the structure of that data for reference.

{
  v,
  c: {
    x,
    y
  },
  r25th,
  r40th,
  r50th,
  r75th,
  r90th
}

| Property | Type | Info | | --- | --- | --- | | v | number | Version of the data structure. If this version ever changes, you should consider the old meta invalid | | c | object | Center point based on average saliency. Typically ignored unless no regions are identified, but still can be useful | | c.x | number | Center x dimension from 0 (left) to 1 (right) | | c.y | number | Center y dimension from 0 (top) to 1 (bottom) | | r25th | MetaRegion | MetaRegion that encompasses ~25% of all saliency | | r40th | MetaRegion | MetaRegion that encompasses ~40% of all saliency | | r50th | MetaRegion | MetaRegion that encompasses ~50% of all saliency | | r75th | MetaRegion | MetaRegion that encompasses ~75% of all saliency | | r90th | MetaRegion | MetaRegion that encompasses ~90% of all saliency |

Meta Region Data

The meta region structure used for caching, not to be confused by the Region returned by getRegionFromSalientMatrix or getRegionFromMeta.

{ l, t, w, h }

| Property | Type | Info | | --- | --- | --- | | l | number | Left-most edge of region on a 0 to 1 scale | | t | number | Top-most edge of region on a 0 to 1 scale | | w | number | Width of region on a 0 to 1 scale | | h | number | Height of region on a 0 to 1 scale |