Leaflet.DataTileLayer

A leaflet tile layer for encoded data tiles.

WARNING: UNDER HEAVY DEVELOPMENT AND SUBJECT TO MAJOR BREAKING CHANGES

Data tiles are a way of encoding multiple raster data layers into a single layer PNG for transport to the browser. Once in the browser, the tiles are decoded to original values.

Tiles are currently packaged into 8-bit grayscale or 24-bit RGB PNG files.

Why?

We needed to display pixel values at a location across multiple layers. We wanted to update these in near real time based on user interaction such as panning or clicking on the map, in order to provide a highly dynamic data exploration interface.

The traditional web GIS approach is to use a full-featured mapserver that provicdes a query / identify API, which takes a location as input and returns information about that location across one or more data layers. Unfortunately, that is non-ideal for a couple reasons:

• due to the round-trip latency of making these requests to a server, that information is hard to update in near real-time due to user interaction. This means that the user interaction would need to be handled differently and follow more traditional approaches such as clicking on the map and waiting for the response.
• a full-featured mapserver is complex to setup and may require licenses. Our ideal setup uses the tiniest tile server stack possible, mbtileserver.

RGB encoding schemes for elevation data (e.g., Mapbox elevation tiles) demonstrate that it is possible to encode data into PNG tiles and decode within the browser.

Basic approach

The core idea of data tiles is to use an encoding to "stack" multiple raster data tiles into a single layer and cut stacked values into PNG tiles. Once received in the browser, the RGB values can be obtained from a canvas element for a single pixel (e.g., Leaflet color picker).

One way to stack data layers into a single layer is to use an exponential encoder. Given a base and values for each layer at a pixel (a, b, c, d), we can encode data by multiplying the original values by base raised to a power corresponding to the 0-based index of that value. Example:

output = a + (b * base) + (c * (base ** 2)) + (d * (base ** 3))

All values must be unsigned integers between 0 and base in order for this approach to work. This means that we can vary base based on the value range of the data to create a fairly compact encoding: we can have several binary data layers stacked into a single PNG, or a few layers with more values.

The final output value can be encoded as either 8-bit grayscale or 24-bit RGB data and cut into tiles.

To decode, we reverse the process. First, we convert the RGBA values from the canvas back into an unsigned integer using bit-shifting (note: alpha is discarded, see below). To do so, you need to know if the original value was 8 or 24 bits. From there, we iteratively decode values:

remainder_d = output % (base ** 3)
d = (output - remainder_d) / (base ** 3)

remainder_c = remainder_d % (base ** 2)
c = (remainder_d - remainder_c) / (base ** 2)

a = remainder_c % base
b = (remainder_c - a) / base

This approach does not allow random access to values from a single layer; all values must be decoded at once.

Other encoders are currently being investigated.

Unique values

In order to make the encoding more compact, you can convert the original values to indexed values. The indexed values are then encoded, and the table of indices to values is provided as part of the encoding. For example, the values 1, 10, 42, 97 would require a base of at least 97, which is very inefficient. Instead, these values can be stored using their index in array (e.g., value 1 is index 0, 42 is index 2, etc). This only requires a base of 4 which is much more compact.

Nodata values

In practice, we handle nodata values in the source data in 2 ways:

• nodata for a single layer is stored as base - 1. These are decoded to null for that layer.
• nodata present in all layers is stored as the max of that data type - 1 (8-bit is 255, 24-bit is 16777215). In this case, null is returned instead of an object.

Encoding metadata

In order to decode RGB values to their original values, we need to know the basic encoding information that was used when the tiles were created. The essential information for decoding is passed in as encoding to this layer.

Example:

{
    "type": "exponential",
    "base": 8,
    "dtype": "uint16",
    "nodata": 65535,
    "layers": [
        {
            "id": "layer1",
            "nodata": 7,
            "type": "indexed",
            "values": [1, 2, 3, 4, 5]
        },
        {
            "id": "layer2",
            "nodata": 7,
            "type": "indexed",
            "values": [10, 20, 30, 40, 50]
        }
    ]
}

Note: exponential is currently the only supported encoder; others are planned.

Reduced size tiles

The default tile size is 256 x 256. However, this is most likely unnecessary precision when responding to user interactions on the frontend, so instead we can create smaller tiles. Leaflet automatically stretches the display of these tiles to 256 x 256, and we do the same when decoding values.

In practice, we found 1/2 resolution tiles (128 x 128) achieve a reasonable balance between tile size and precision. You can vary this down further based on the nature of your data and use case.

Note: tiles greater than 256 x 256 are not supported, as this is too much precision.

Installation and usage

yarn install leaflet-datatilelayer

Add a datatile layer

const layer = L.dataTileLayer(url, {
    encoding: ...see encoding above...,
    opacity: 0, // you generally do not want to actually see data on the map unless debugging
    imageSize: 128 // you can use tiles at 1/2 resolution to reduce size (default is 256)
}).addTo(map);

Decode values at a location

In this example, we get values for the center of the map.

function getValue() {
    var values = layer.decodePoint(map.getCenter());
    console.log("values", values);
}

layer.on("load", getValue); //call after tiles have loaded
map.on("move", getValue);

The decoded values may be

• null if all pixels are nodata
• an object:

{
    layer1: 3,
    layer2: null // layer2 has nodata at this location
}

See examples/index.html for a more complete example (under development).

Limitations

Due to issues with RGBA decoding, RGBA PNGs are not currently supported. This is because different browsers apply gamma correction differently for RGBA PNG files, which means that the RGBA values derived from the image no longer match the values used when encoding the data tiles. Unfortunately, this completely breaks the decoding process.

Supported formats are 8-bit grayscale and 24-bit RGB PNG tiles.

Development

rollup is used with babel to package the files into dist/index.js.

1. yarn install or npm install
2. yarn watch to run the rollup watch process on the files.

Credits:

This project was supported in part by a grant from the South Atlantic Landscape Conservation Cooperative.

The core idea of datatiles was inspired by encoded elevation data, such as Mapbox elevation tiles.

The tile pixel data extraction process was derived and heavily modified from: https://github.com/frogcat/leaflet-tilelayer-colorpicker