@gmod/gbz-base

A pure TypeScript reader for gbz-base pangenome databases (.gbz.db).

Install

npm install @gmod/gbz-base

Usage

import { RemoteFile } from 'generic-filehandle2'
import { GBZBase } from '@gmod/gbz-base'

const db = await GBZBase.open(
  new RemoteFile('https://example.org/graph.gbz.db'),
)

// one record per haplotype crossing the window
const alignments = await db.getAlignmentsForRange(
  'GRCh38#0#chr6',
  31500000,
  31501000,
)

// the same window as a subgraph, for a pangenome view
const subgraph = await db.getSubgraphForRange(
  'GRCh38#0#chr6',
  31500000,
  31501000,
)
const gfa = await subgraph?.toGFA({ names: 'resolved' })

Coordinates are 0-based half-open, and are offsets along the path you named, so ('GRCh38#0#chr6', 31500000, 31501000) is the same window gbz-base query --interval 31500000..31501000 gives. The path is a PanSN sample#haplotype#contig string, or a bare contig for a graph whose reference paths have no sample.

GBZBase reads only the SQLite pages a query touches, so it queries a multi-gigabyte database on an HTTP server through range requests without downloading it. Any generic-filehandle2 source works — LocalFile, RemoteFile, BlobFile — and there is a gbz-base-query command line that mirrors the upstream tool. Every option, method and query function: docs/api.md; what a query does between the call and the records, drawn: docs/dataflow.md.

Alignment records

for (const alignment of alignments) {
  const { refStart, refEnd, strand, cigar } = alignment
  if (alignment.resolved) {
    console.log(alignment.label, alignment.hapStart, alignment.hapEnd)
  }
}

A record is one haplotype's passage through the window, aligned to the reference path you queried. Where a haplotype leaves the subgraph and comes back — a private insertion, a bubble the window does not hold — the pieces are joined back into one record and the stretch between them becomes its insertion and deletion, so context changes what is read rather than how many records you get. The fields, the joining rules and what resolved means: docs/alignments.md.

subgraph.pairAlignments aligns one haplotype to another with the bases compared, so two haplotypes sharing sequence the reference lacks align through it, and gbz-base-query --stack prints each row of a stacked synteny view against the next as PAF: docs/alignments.md.

getAlignmentsForRange hands back data and spans path fragments; getSubgraphForRange hands back the Subgraph itself, because two disjoint fragments do not merge into one graph.

Naming haplotypes

Upstream gbz-base cannot determine which haplotype a subgraph path belongs to, so it emits unknown#N. This package adds that with two side tables that a small Rust tool (tools/haplotype-index/) writes into an existing database, or into a standalone companion beside a database someone else hosts:

const db = await GBZBase.open(new RemoteFile(graphUrl), {
  haplotypeIndex: new RemoteFile(indexUrl),
})

With names in hand, keep cuts a window to a chosen set of haplotypes — the reference walk, the walks the predicate accepts and the nodes those walks visit, so the drawing shows that set's private sequence and nothing else's. On a companion carrying anchors those haplotypes are walked from an anchor before the window, and nothing else is extracted or named, so the query's cost scales with the set rather than the graph: the eight haplotypes of the MHC class II tutorial window come back in 0.97 s against 9.16 s for all 464, both warm. docs/haplotype-index.md covers building the index and the two routes; docs/performance.md has the measurements.

Snarls

The snarls option uses the top-level chains a .gbz.db already stores, the way upstream's --snarls and --extend-snarls do — contained brings a window's variation back without widening it by a bp radius. docs/snarls.md.

Why not compile the Rust to wasm

gbwt-rs and simple-sds serialize usize at native width, so a wasm32 build misreads every file written on a 64-bit host, and the PR to fix that was closed unmerged — reasonably, since the same hazard sits on every other usize in the crate. wasm64 fixes the width but cannot carry gbz-base, whose bundled SQLite has no wasm64 libc. A TypeScript reader has neither problem, and it supports HTTP range access and runs in a JBrowse RPC worker without extra code for either: docs/why-not-wasm.md.

In JBrowse

jbrowse-plugin-graphgenomeviewer is the consumer this was written for. Its GbzBaseSyntenyAdapter opens a .gbz.db and its companion through JBrowse's own file access layer, and calls getSubgraphForRange for the graph view and getAlignmentsForRange for the haplotype lanes — in an RPC worker, like any other adapter.

Because the worker is where the subgraph is built and the main thread is where it is drawn, the representation it crosses in matters more than the representation it is built in. toSubgraphJson returns upstream's format, which uses an object and a stringified id for each of a human window's ~350,000 steps and costs about 295 ms to structured-clone; toCompactSubgraph returns the same subgraph as typed arrays of GBWT handles, which clones in 1.9 ms and can be transferred instead of copied. docs/api.md has both formats and the measurements.

Docs

• docs/contents-and-limitations.md — how a GBZ differs from an rGFA, and the limits of a query
• docs/api.md — every option, query function, output format and command line flag
• docs/dataflow.md — how a query flows, and where the two identification routes branch
• docs/alignments.md — what an alignment record is, and how walk fragments are joined into one
• docs/haplotype-index.md — building the haplotype index, and the two routes a named query takes
• docs/snarls.md — the snarl options and --between
• docs/performance.md — measured windows, and picking context
• docs/optimizations.md — why the alignment and the anchored walk look the way they do
• docs/internals.md — reading SQLite without SQLite, and where this reader's CIGARs differ from upstream's
• docs/why-not-wasm.md — why the Rust was not compiled to WebAssembly instead
• CONTRIBUTING.md — development, test data and release steps

Footnote

Started from ideas at MemPanG 26!

License

MIT © Colin Diesh