lz4-hc-wasm

WebAssembly implementation for LZ4 high compression encoder and decoder with support for both frame-level and block-level data formats.

Credits

Pierre Curto wrote lz4 as a Golang module.

This repo includes:

• his Golang module as a git submodule
• a main.go wrapper to define the JavaScript API, and manage data type conversion
• a build script to generate a WASM library
• dist
  • the WASM library
  • a helper script to load the WASM library into JavaScript for each of the following environments:
    • CommonJS
    • ESM
    • web
• a test for each environment

Installation

npm:

  npm install "@warren-bank/lz4-hc-wasm"

Usage

CommonJS:

  const {init} = require('@warren-bank/lz4-hc-wasm')

ESM:

  import {init} from '@warren-bank/lz4-hc-wasm'

common:

  const lz4 = await init()

JavaScript API

  // frame-level data format
  type lz4.compressFrame           = (src: Uint8Array, depth?: int) => Promise<Uint8Array>
  type lz4.uncompressFrame         = (src: Uint8Array) => Promise<Uint8Array>

  // block-level data format
  type lz4.compressBlockBound      = (n: int) => int
  type lz4.compressBlock           = (src: Uint8Array, dstSize?: int) => Promise<Uint8Array>
  type lz4.compressBlockHC         = (src: Uint8Array, depth: int, dstSize?: int) => Promise<Uint8Array>
  type lz4.uncompressBlock         = (src: Uint8Array, dstSize?: int) => Promise<Uint8Array>
  type lz4.uncompressBlockWithDict = (src: Uint8Array, dict: Uint8Array, dstSize?: int) => Promise<Uint8Array>

where:

• frame-level data format includes:
  • a header
    • magic number
    • descriptor fields
  • a sequence of blocks of compressed data
  • a footer
    • end mark
    • checksum
• block-level data format includes:
  • a single block of compressed data
• Promise<Uint8Array> means that a Promise is returned, which will either:
  • resolve to an Uint8Array
  • reject if any error occurs
• the following block-level details are low-level and can generally be ignored:
  • the function: lz4.compressBlockBound
  • the parameter: dstSize?: int
• the depth parameter is used to control the level of compression
  • its value can range: 0 to 9
  • for frame-level compression, this value defaults to 0
    • lz4.compressBlock is used when this value is equal to 0
    • lz4.compressBlockHC is used when this value is greater than 0
  • for block-level high compression, this is a required value
    • note that in this case, lz4.compressBlock is not used when this value is equal to 0

notes:

• test results confirm this open issue,which is that the depth compression-level parameter to compressBlockHC has no effect

To Do

• reduce the size of the WASM library by using a different Golang compiler
  • ex: TinyGo

Legal

• copyright: Pierre Curto
• license: BSD-3-Clause