@haot/3dtiles-native

Native OSGB to 3D Tiles conversion for Node.js, based on a pinned fanvanzh/3dtiles core at revision acbcf603f33fdfe3c34b704a8b019c4fd32a8376.

One conversion owns one native subprocess. Model resources and output use callbacks; no model files, temporary directories or filesystem caches are used. The caller owns ZIP access, authorization, durable storage and publication. The release targets are Windows x64 and Linux x64 glibc (Debian Bookworm), with Node 24. Both platform builds pass the native stream/texture checks.

npm install @haot/3dtiles-native

convertOsgb accepts dataset-relative resource names, asynchronous read and write callbacks, concurrency/size limits, progress and an AbortSignal. Output callback completion acknowledges consumption, not durable publication. The returned promise settles only after the subprocess closes.

Development

src/ contains the Node API; native/ contains the subprocess transport and adapter; patches/ documents changes to the pinned core; scripts/ contains build and packaging entrypoints. .build/ and dist/ are disposable. No install script downloads or builds executable code for consumers.

Both release builds use the checksum-pinned vcpkg baseline. Linux includes the required Bookworm OpenGL dispatch libraries for OSG linking; conversion needs no GPU, display server or rendering window. Only the dependency closure is packaged, including MSVC redistributable libraries on Windows. System libc/libstdc++ remain provided by Bookworm. The package records library provenance and includes notices.

The native program loads only installed binaries/configuration from disk. GDAL and PROJ network access is disabled; coordinates remain CGCS2000 with assumed ellipsoidal height. Conversion options do not accept local model paths.

Install maintainer tools with mise install and Node dependencies with npm ci. Windows builds need the MSVC v143 C++ toolchain and a Windows SDK; run from its Developer PowerShell. Linux builds target Debian Bookworm with GCC 12 and require build-essential git curl zip unzip tar pkg-config autoconf automake libtool python3 nasm and OpenGL development headers. Docker is optional for an isolated Linux build. Consumers need none of these build tools.

HAOT_NATIVE_BUILD_DIR optionally puts maintainer build caches on a fast local filesystem. This is a build-only setting, not a runtime model cache. In Docker Desktop, avoid compiling dependency sources on a Windows bind mount; keep this directory inside the build container and copy only release artifacts out.

npm run build downloads checksum-pinned sources, builds the current platform, and builds the Node API. Both platform folders must be assembled under dist/native/ before packaging. There is no install-time download or compilation. Use npm run build:js, npm run typecheck, npm run lint, npm test and npm run format:check for API development. npm pack refuses missing platform artifacts. Assemble and pack releases on Linux so npm preserves executable permissions; a Windows-created tarball can strip the Linux program's execute bit. Review dependency notices and real-model acceptance before publishing.

import { convertOsgb } from "@haot/3dtiles-native";

const result = await convertOsgb({
  resources: ["metadata.xml", "Data/BlockA/BlockA.osgb" /* selected ZIP inventory */],
  read: (key, maxBytes, signal) => source.readEntry(key, maxBytes, signal),
  write: (key, bytes, contentType, signal) => archive.add(key, bytes, contentType, signal),
  concurrency: 2,
  readConcurrency: 4,
  signal: controller.signal,
});
console.log(result.tilesetPath, result.statistics);

The example's source and archive are caller-owned resource/stream adapters. There is no local input/output path API or storage SDK dependency. The caller must validate the completed archive before publishing it. Each conversion handles one dataset, with metadata.xml at its resource root. Block entrypoints take precedence over Model.osgb. Missing, escaping or cyclic references fail explicitly.

Maintainer checks generate a tiny OSGB scene entirely through memory streams to verify relative PagedLOD references, JPEG behavior and transparent PNG pixels. native-fixture is development-only and never enters the npm package. Format C++ with clang-format-14 -i native/*.cpp native/*.h tests/*.cpp; JS/TS uses Oxfmt.

API limits and lifetime

Optional optimization (0.2.0)

optimization contains optional textureCompression, draco, and simplify objects. Each has an enabled boolean (default false). Omitting optimization preserves 0.1.0 behavior. normalizeOptimization() validates and resolves a full snapshot; the conversion result returns that snapshot and actual output triangles, vertices, and geometries counts.

| Group | Parameter | Default | Range | | -------------------- | -------------- | ------: | ---------------------------- | | textureCompression | quality | 128 | Integer 1–255 | | draco | positionBits | 11 | Integer 10–24 | | draco | normalBits | 10 | Integer 8–16 | | draco | texCoordBits | 12 | Integer 8–16 | | simplify | ratio | 0.5 | 0.01–1 | | simplify | error | 0.01 | 0–1, relative to mesh extent |

Texture compression emits sRGB KTX2 / ETC1S with mipmaps, using Basis Universal 1.60. Dimensions not divisible by four are resampled to the next multiple of four over the same UV domain. RGB/BGR, RGBA/BGRA and luminance inputs respect row alignment and alpha. Maximum encoding dimensions are 16384 per axis. ETC1S is lossy: tiny, high-contrast textures can change substantially, and encoding can take longer than JPEG. Disabled compression retains JPEG/transparent PNG output. An encoder failure fails the task; there is no silent format fallback.

Geometry optimization first separates primitive groups and normalizes supported triangle strips, fans and quads, preserving DrawArrays offsets. Optional meshoptimizer 1.0 simplification precedes Draco 1.5.7 encoding (speed 5/5). Non-triangle and multi-draw geometry is rejected when geometry optimization is enabled; supported commands are DrawArrays and 8/16/32-bit DrawElements. Simplification may not reach its target ratio. Quantization and simplification are lossy and do not guarantee centimetre accuracy; tile bounds and geometric errors conservatively include their estimated displacement. Encoder internals use one thread; task concurrency controls parallel roots.

optimization: {
  textureCompression: { enabled: true, quality: 128 },
  draco: { enabled: true, positionBits: 11, normalBits: 10, texCoordBits: 12 },
  simplify: { enabled: true, ratio: 0.5, error: 0.01 },
}

resources lists a single dataset relative to its metadata.xml. read(key, maxBytes, signal) must return at most maxBytes, and write must consume its byte view before resolving. Both callbacks must honor cancellation. The package owns one child process per call; stdout carries length-bounded binary frames, never logs or Base64. On abort it requests cancellation and force-kills an unresponsive child after five seconds. Parent-pipe closure terminates the native program. A rejected conversion does not authorize publication of partial output.

| Option | Default | Range | | -------------------- | -----------: | --------------------------- | | concurrency | 2 | 1–32 threads | | readConcurrency | 4 | 1–32 reads | | resourceCacheBytes | 268435456 | 0 disables caching | | maxResourceBytes | 67108864 | Positive bytes, up to 1 GiB | | maxTileBytes | 536870912 | Positive bytes, up to 1 GiB | | maxTotalBytes | 214748364800 | Positive safe integer | | maxEntries | 500000 | 1–1000000 | | maxDepth | 64 | 1–256 |

Cache and per-buffer limits are not a total RSS bound. Statistics report native peak RSS separately from the Node host. elapsedMs is wall time; resourceReadMs, decodeMs, conversionMs and outputWaitMs aggregate thread time and can exceed wall time when conversion runs concurrently. Callback waits do not include later archive verification or database publication performed by the caller.

Validation and performance

Version 0.2.0 passes all eight optimization combinations on complete City and School subtrees. Decoding checks cover Draco attributes/indices/quantization, KTX2 alpha and mipmaps, pixel layouts and complete primitive groups. With all options disabled, representative output maps match the preserved 0.1.0 package byte for byte. Complete City and School texture-only and all-enabled results also pass near/far review in CesiumJS and Three.js, including LOD transitions, texture orientation, footprint and shoreline checks.

The complete School dataset passes texture-only and all-enabled conversion and near/far review in CesiumJS and Three.js. All-enabled output contains 44,038,643 triangles and 1,425,763,467 bytes, compared with 86,605,948 triangles and 4,182,287,637 bytes for texture-only. The all-enabled run took 4,275.838 seconds in the native process and 4,332.325 seconds through application publication; native peak RSS was 390,516,736 bytes. Eight threads were configured, limited to four block roots. Other conversion work ran concurrently, so these times are acceptance observations, not controlled throughput comparisons. The original source holes and outlines remain; compression and simplification lose detail.

The complete City texture-only run passes near/far review in both engines and a same-camera comparison with 0.1.0. It emits 17,849,716,427 bytes and 356,342,271 triangles across 17,594 nodes. Native wall time was 8,124.515 seconds, application publication took 8,554.652 seconds, and native peak RSS was 629,112,832 bytes. Eight root threads were configured; another conversion overlapped this run.

The full Linux City all-enabled run emits 6,117,029,157 bytes and 182,813,306 triangles across the same 17,594 nodes. Native wall time was 13,686.068 seconds, the application pipeline took 14,673.146 seconds, and native peak RSS was 987,987,968 bytes. The Node host and worker peaked at 994,095,104 bytes. This read-only Bookworm container used eight CPUs and a 4 GiB memory limit; concurrent Windows work makes this unsuitable for a platform speed comparison.

On the Windows host below, one 35-resource School subtree took 3.19 seconds with all options disabled, 98.67 seconds with textures only, and 106.73 seconds with all options enabled. Output sizes were 32,835,115 / 34,530,555 / 12,965,402 bytes. These are individual development runs, not throughput guarantees. ETC1S targets GPU texture storage and includes mipmaps; its archive payload can exceed JPEG. Callers should choose an explicit task deadline suitable for their data and measure encoding time before enabling it for large jobs.

The 0.1.0 candidate was exercised on full School (4,278 OSGB resources), City (17,593), International Finance (135) and Three Centers (297) datasets, with CesiumJS and Three.js review of the resulting 3TZ archives. School geometry, indices, UVs and JPEG payloads were compared with retained upstream output; maximum vertex-coordinate difference was 0.000030517578125 metres. Source holes are retained; the converter does not reconstruct missing geometry.

On an Intel Core Ultra 9 275HX / Windows host, sequential full-School runs at 1/2/4 threads took 377/239/167 seconds inside the native process and 478/352/281 seconds through ZIP access, archive upload and validation. Native peak RSS was 73/96/125 MiB; the Node host including its archive worker peaked at 664/818/796 MiB. Each run read 3.51 GB and emitted 4.34 GB. These are single warm-service measurements of one dataset, not a promised throughput or a total-memory bound.

The packed Linux program was also installed in a clean, non-root Bookworm image with a read-only root and no writable mounts. OSGB conversion, direct archive import in the consuming application and authenticated viewing completed without container filesystem changes. Critical checks cover transparent texture pixels, deterministic multi-block output at 1/2/4 threads, framing, limits, backpressure, failed spawn, cancellation and parent-pipe closure. The consuming application also verified worker failure, storage failure, lease loss and hot reload cleanup.