@truelies/osm-dybuf

Gridex osm.dybuf parser utilities shared by the exporter, inspection scripts, and Next.js frontend.
This package wraps the shared schema tables (schema_ids) and exposes a high-level parseOsmDybuf helper.
It depends on the published dybuf runtime (≥ 0.4.2). Feature IDs are packed as feature_id = sub_id * 64 + main_id (both 0–63). The parser now returns numeric featureId; use schema_ids helpers to decode debug strings or subtypes when needed. schema_ids also exports packFeatureId(...), unpackFeatureId(...), and labelFeatureId(...) for subtype-aware features such as future label overlays. Current road groups are road_express, road_major, road_minor, road_local, and road_trail.

Current Compatibility Status

parseOsmDybuf now supports the current cloud formats and keeps a legacy fallback.

Support matrix:

• Flat V1 cell file (osm.dybuf): version + payload -> supported (format="flat_v1")
• V2 overview/bundle (overview.dybuf, bundle.dybuf): version=2 + index entries -> supported (format="v2_indexed")
• Legacy V1 with region chunks: version + [region_id + region_payload]... -> supported as fallback (format="legacy_v1_regions")

inspect_dybuf.mjs now prints by detected format.

Current frontend-facing schema notes:

• admin_boundary is retained in schema IDs for compatibility, but new exports no longer send boundary lines to the frontend.
• place_label remains in schema IDs for compatibility, but current exports no longer emit place-name labels to the frontend.
• label main feature ID is now reserved for future overlay labels; sub_id is intended to carry label kind (worship, station, custom, etc.).
• Current exporter-oriented subtypes are label_worship and label_station, both derived from existing building tasks. Consumers should group all future label_* subtypes by unpackFeatureId(featureId).mainKey === "label" if they want a single overlay layer.
• Breaking change in 0.4.x: GridFeature.featureId is numeric. Consumers should map it with ID_TO_FEATURE_KIND[String(featureId)] or unpackFeatureId(featureId) when string/debug output is needed.
• road_express covers motorway corridors plus trunk ways tagged with motorroad=yes (for example many Taiwan expressway mainlines).
• water_wetland separates marsh / bog / fen / wetland polygons from deeper open water.
• Geometry payload exports currently stop at lv13, but Gridex/index helpers remain valid through lv16 for higher-detail indexing metadata.

Usage

# from another project (Next.js, Node, etc.)
npm install @truelies/osm-dybuf

import { parseOsmDybuf } from "@truelies/osm-dybuf";
import {
  FEATURE_KIND_IDS,
  ID_TO_FEATURE_KIND,
  MAIN_FEATURE_IDS,
  unpackFeatureId,
  REGION_NUMERIC_CODES,
} from "@truelies/osm-dybuf/schema_ids";

const data = await fetch("/tiles/08/213/161/osm.dybuf").then((res) => res.arrayBuffer());
const cell = { level: 8, londex: 213, latdex: 161 };
const parsed = parseOsmDybuf(data, cell);

if (parsed.format === "flat_v1") {
  console.log(parsed.features.length);
  console.log(parsed.features[0].featureId); // numeric feature id
  console.log(ID_TO_FEATURE_KIND[String(parsed.features[0].featureId)]);
  console.log(unpackFeatureId(parsed.features[0].featureId));
  console.log((parsed.features[0].featureId & 0x3f) === MAIN_FEATURE_IDS.label);
} else if (parsed.format === "v2_indexed") {
  console.log(parsed.entries.length);
} else {
  console.log(parsed.regions.length);
}

• CLI inspection (development only):

  # Preferred: only provide file path.
  # For v1 osm.dybuf, the tool auto-infers level/londex/latdex from .../<lv>/<x>/<y>/osm.dybuf
  node js/inspect_dybuf.mjs \
      --file /path/to/osm.dybuf \
      --limit 5
  
  # Optional fallback when path does not contain cell coordinates (needed for v1):
  node js/inspect_dybuf.mjs \
      --file /path/to/osm.dybuf \
      --level 8 --londex 213 --latdex 161 --limit 5

Feature ID usage notes:

• Use FEATURE_KIND_IDS.* for exact feature comparison, for example featureId === FEATURE_KIND_IDS.waterbody.
• Use MAIN_FEATURE_IDS.* with featureId & 0x3f when you want to group all subtypes under the same main feature, for example (featureId & 0x3f) === MAIN_FEATURE_IDS.label.
• Use unpackFeatureId(featureId) for debug output or subtype-aware logic where readability matters more than raw hot-path comparisons.

Grid helpers (grid_index)

grid_index.mjs exposes Gridex helpers (integers, pole triangles) aligned with the exporter:

import {
  GridUnit,
  Gridex,
  INT_COORD_SCALE,
  CANONICAL_CODE_LEVEL,
  gridexAtLonLat,
  gridCodeAtLonLat,
  gridCodeNeighbors,
  gridCodeComponentsAtLonLat,
  lonCodeFromLondex,
  latCodeFromLatdex,
  londexPrefixCodeFromLonCode,
  lonLatCodeRangeForGridex,
  encodeAxisCode,
  decodeAxisCode,
  lonCodeMagnitudeBits,
} from "@truelies/osm-dybuf/grid_index";

const unit = new GridUnit(8); // level 8
const cell = new Gridex(213, 161);
const [minLon, minLat, maxLon, maxLat] = unit.boundsOfGrid(cell);

// parent/child conversion (same rules as Python GridUnit)
const children = unit.toLowerLevelGridexes(cell);   // level 9, 4 cells
const parent = new GridUnit(9).toUpperLevelGridex(children[0]); // back to level 8

// GPS -> gridex / lonCode / latCode / gridCode
const gridex = gridexAtLonLat(16, 121.56, 25.03);
const parts = gridCodeComponentsAtLonLat(16, 121.56, 25.03);
console.log(gridex.londex, gridex.latdex);
console.log(parts.lonCode, parts.latCode, parts.gridCode);

// gridex <-> code
const lonCode = lonCodeFromLondex(16, -123);
const latCode = latCodeFromLatdex(16, 456);
console.log(lonCode, latCode);
console.log(gridCodeAtLonLat(16, 121.56, 25.03));
console.log(gridCodeNeighbors(9, gridCodeAtLonLat(9, 121.56, 25.03)));

// canonical prefix / range for coarser level query
const lonPrefix = londexPrefixCodeFromLonCode(15, lonCode, CANONICAL_CODE_LEVEL);
const range = lonLatCodeRangeForGridex(15, -3, 4, CANONICAL_CODE_LEVEL);
console.log(lonPrefix, range.lonCode.from, range.lonCode.to);

// axis code roundtrip
const lonBits = lonCodeMagnitudeBits(CANONICAL_CODE_LEVEL);
const axisCode = encodeAxisCode(-123, lonBits);
console.log(decodeAxisCode(axisCode, lonBits)); // -123

• toLowerLevelGridexes(gridex):
  • returns 4 child cells at level + 1
  • throws when current unit is already max level (16)
• toUpperLevelGridex(gridex):
  • returns the parent cell at level - 1
  • throws when current unit is level 0
• gridexAtLonLat(level, lon, lat):
  • converts GPS directly to a Gridex
• gridCodeComponentsAtLonLat(level, lon, lat):
  • returns { gridex, lonCode, latCode, gridCode }
• lonCodeFromLondex(level, londex) / latCodeFromLatdex(level, latdex):
  • converts axis indices to canonical axis codes
• londexPrefixCodeFromLonCode(targetLevel, lonCode, sourceLevel?):
  • trims a finer lonCode into a coarser-level prefix view
• lonLatCodeRangeForGridex(level, londex, latdex, sourceLevel?):
  • expands any lv <= sourceLevel cell into the corresponding canonical axis-code range
• gridCodeAtLonLat(level, lon, lat) / gridCodeFromGridex(level, gridex):
  • returns the packed gridCode
• GridUnit.neighbors(gridex) / gridexNeighbors(level, gridex, radius?) / gridCodeNeighbors(level, gridCode, radius?):
  • returns one-ring neighbors for an existing cell/code
  • currently only supports radius=1
  • crosses the zero axis without returning zero-index cells; longitude wraps and latitude clamps at poles
• encodeAxisCode(index, magnitudeBits) / decodeAxisCode(code, magnitudeBits):
  • encodes grid index as sign bit + (abs(index) - 1)

Collectible types

collectible_types.mjs exposes stable numeric collectible type IDs for Firestore / frontend use:

import {
  COLLECTIBLE_TYPE_IDS,
  COLLECTIBLE_TYPES_BY_ID,
  TELEPORT_SUBTYPE_IDS,
} from "@truelies/osm-dybuf/collectible_types";

console.log(COLLECTIBLE_TYPE_IDS.teleport); // 1
console.log(TELEPORT_SUBTYPE_IDS.rail); // 1
console.log(COLLECTIBLE_TYPES_BY_ID[2].nameZh); // "聖晶石"

Current built-in IDs:

• 1: 傳送點
• 2: 聖晶石

Current teleport subtypes:

• 1: 鐵路
• 2: 陸路
• 3: 海路
• 4: 空路

Local Development

cd js
npm install      # installs dybuf runtime for this package
npm test         # runs unit tests (node:test)

Unit test example (tests/grid_index.test.mjs) covers:

• gridexesAt near axis (no zero index cell)

• toLowerLevelGridexes and toUpperLevelGridex roundtrip

• level boundary errors (lv0 has no upper level, lv16 has no lower level)

• canonical axis-code encode/decode roundtrip

• GPS -> gridex / lonCode / latCode / gridCode

• lv15 cell -> canonical code range expansion correctness

• npm pack: build a tarball so other repos can install via npm install ./path/to/osm-dybuf-*.tgz

• npm link: link the package locally (npm link here, then npm link @truelies/osm-dybuf in the consumer repo)

• Publish to npm (requires @truelies scope):

  cd js
  npm version <new-version> --no-git-tag-version
  NPM_CONFIG_CACHE=/tmp/npm-cache npm login --auth-type=web --registry=https://registry.npmjs.org/ --scope=@truelies
  NPM_CONFIG_CACHE=/tmp/npm-cache npm whoami
  NPM_CONFIG_CACHE=/tmp/npm-cache npm publish --access public

  • If ~/.npm has permission issues (root-owned cache/logs), keep using NPM_CONFIG_CACHE=/tmp/npm-cache.
  • If the scope is not available, rename package.json → "name": "truelies-osm-dybuf" and publish without a scope.

Files

• osm_dybuf.mjs / .d.mts: DyBuf parser entry points
• schema_ids.mjs / .d.mts: shared identifiers (geometry, feature, region, segment roles)
• region_numeric_codes.json: latest region ID table (generated via scripts/dump_region_codes.py)
• inspect_dybuf.mjs: CLI tool that uses the package locally

When schema IDs or DyBuf layouts change in the exporter, remember to bump the version here and re-publish/install so frontends and tools stay in sync. Version 0.4.3 aligns the package with the current lv0~13 export range, the current frontend feature set (water_wetland, building, derived label_*, no exported place_label), the numeric featureId parser contract, and the canonical grid conversion helpers shared by Firestore / Redis / frontend indexing.