@qtsurfer/lastra

TypeScript reader for the Lastra columnar time series format.

Features

• Read .lastra files written by the Java LastraWriter
• Selective column access — only requested columns are decompressed
• Built-in decoders: ALP, Gorilla, Pongo (doubles), delta-varint (timestamps), ZSTD/gzip (binary)
• Apache Arrow interop — convert to Arrow Table for use with DuckDB-WASM, Perspective, Arquero, etc.
• Browser + Node.js compatible
• Zero-copy Float64Array output for decoded doubles

Install

npm install @qtsurfer/lastra

Usage

import { LastraReader } from '@qtsurfer/lastra';

// From fetch response
const buffer = await fetch('/data/btc-1h.lastra').then(r => r.arrayBuffer());
const reader = new LastraReader(buffer);

// Series data
const ts = reader.readSeriesLong('ts');           // Float64Array (timestamps)
const close = reader.readSeriesDouble('close');    // Float64Array (prices)

// Column metadata
const meta = reader.getSeriesColumn('ema1').metadata;
// { indicator: 'ema', periods: '10' }

// Events (signals with independent timestamps)
const signalTs = reader.readEventLong('ts');
const types = reader.readEventBinary('type');      // (Uint8Array | null)[]
const data = reader.readEventBinary('data');       // ZSTD-compressed JSON

// Decode event strings
const decoder = new TextDecoder();
types.forEach((t, i) => {
  if (t) console.log(decoder.decode(t), data[i] ? JSON.parse(decoder.decode(data[i]!)) : null);
});

API

LastraReader

| Property / Method | Returns | Description | |---|---|---| | seriesRowCount | number | Number of series rows | | eventsRowCount | number | Number of event rows | | seriesColumns | ColumnInfo[] | Series column descriptors | | eventColumns | ColumnInfo[] | Event column descriptors | | readSeriesLong(name) | Float64Array | Decode a LONG series column | | readSeriesDouble(name) | Float64Array | Decode a DOUBLE series column | | readSeriesBinary(name) | (Uint8Array\|null)[] | Decode a BINARY series column | | readEventLong(name) | Float64Array | Decode a LONG event column | | readEventDouble(name) | Float64Array | Decode a DOUBLE event column | | readEventBinary(name) | (Uint8Array\|null)[] | Decode a BINARY event column | | readEventBinaryAsync(name) | Promise<...> | Async decode (native gzip in browser) | | getSeriesColumn(name) | ColumnInfo | Get column descriptor with metadata | | getEventColumn(name) | ColumnInfo | Get event column descriptor |

ColumnInfo

interface ColumnInfo {
  name: string;
  dataType: DataType;
  codec: Codec;
  metadata: Record<string, string>;
}

Codec Support

| Codec | Type | Description | Bundle cost | |---|---|---|---| | DELTA_VARINT | LONG | Delta-of-delta + zigzag varint (timestamps) | 0 kB | | ALP | DOUBLE | Adaptive lossless floating-point (~3-4 bits/value) | 0 kB | | GORILLA | DOUBLE | XOR compression (Facebook VLDB 2015) | 0 kB | | PONGO | DOUBLE | Decimal-aware erasure + Gorilla XOR (best for prices) | 0 kB | | VARLEN | BINARY | Variable-length encoding | 0 kB | | VARLEN_ZSTD | BINARY | Variable-length + ZSTD | ~3.8 kB gz | | VARLEN_GZIP | BINARY | Variable-length + gzip (async, native DecompressionStream) | 0 kB | | RAW | LONG/DOUBLE | Uncompressed / Float64Array zero-copy | 0 kB |

Arrow Interop

Convert Lastra data to an Apache Arrow Table for integration with any Arrow-compatible tool.

import { LastraReader, lastraToArrow, lastraToArrowColumns } from '@qtsurfer/lastra';

const buffer = await fetch('/data/btc-1h.lastra').then(r => r.arrayBuffer());
const reader = new LastraReader(buffer);

// All series columns
const table = lastraToArrow(reader);

// Selected columns only (others are not decompressed)
const partial = lastraToArrowColumns(reader, ['t', 'cls', 'vol']);

// One-liner: buffer → Arrow Table
import { readLastraAsArrow } from '@qtsurfer/lastra';
const table2 = readLastraAsArrow(buffer);

The resulting Arrow Table can be registered in DuckDB-WASM, used with Perspective, Arquero, Observable Plot, or any tool that accepts Arrow IPC.

Cross-Language Testing

Test fixtures are generated by the Java LastraWriter and read by the TS reader, ensuring wire format compatibility. Includes real BTC/USDT ticker data with mixed codecs (ALP + Gorilla + Pongo).

License

Copyright 2026 Wualabs LTD. Apache License 2.0 — see LICENSE.