⚠️ This is a platform-specific binary package.

You should not install @fast-fs-hash/fast-fs-hash-node-linux-x64-musl directly. Instead, install the main package which will automatically select the correct binary for your platform:

npm install fast-fs-hash

See fast-fs-hash on npm for documentation.

fast-fs-hash

If you ever needed to check whether a set of files changed — to invalidate a cache, skip redundant builds, or trigger incremental CI — fast-fs-hash is for you.

It hashes hundreds of files in milliseconds using xxHash3-128 via a native C++ addon with SIMD acceleration.

xxHash3 is a non-cryptographic hash function — it is not suitable for security purposes, but it is more than enough for cache invalidation, deduplication, and change detection, which is what this library is designed for.

Zero external dependencies. Requires Node.js >= 22.

Installation

npm install fast-fs-hash

Requires Node.js >= 22.

The native addon is prebuilt for common platforms via platform-specific optional dependencies. When you run npm install, npm automatically installs only the package matching your current OS and architecture.

Supported platforms: macOS, Linux (glibc & musl), Windows, FreeBSD — both x64 and arm64.

On x64, optimized variants for AVX2 and AVX-512 are included and selected automatically at load time via native CPUID detection. Set FAST_FS_HASH_ISA=avx2|avx512|baseline to override.

CI note: Some CI configurations disable optional dependencies by default (e.g. npm install --no-optional or --omit=optional). To get the native addon in CI, either allow optional dependencies or install the platform package explicitly:

npm install @fast-fs-hash/fast-fs-hash-node-linux-x64-gnu

FileHashCache — Binary cache invalidation

FileHashCache reads, validates, and writes a compact binary cache file that tracks per-file stat metadata (inode, mtime, ctime, size) and content hashes (xxHash3-128).

On the next run it re-stats every tracked file and compares — files whose stat matches are skipped entirely (no re-read), giving near-instant validation for large file sets.

Why use FileHashCache?

Build systems, code generators, and CI pipelines often produce output that depends on many input files. Recomputing that output on every run is expensive — even when nothing changed.

FileHashCache solves this by persisting a fingerprint of all input files between runs. On the next invocation, it checks whether any input changed in sub-millisecond time (stat-only, no re-reading). If nothing changed, you skip the expensive step entirely.

Common use cases:

• Incremental builds: track source files → skip compilation when inputs are unchanged
• Generated output caching: store a compiled bundle, generated types, or processed assets alongside the cache — rebuild only when dependencies change
• CI artifact caching: validate whether a cached artifact is still fresh before uploading or downloading a new one
• Multi-step pipelines: each stage writes its own cache file, checked independently
• Standalone hashing & compression: the same fast native xxHash3-128 and LZ4 that power the cache are exported directly — hash buffers, strings, or files, or LZ4-compress data on their own

The cache file also supports user data — opaque binary payloads stored alongside the file hashes. This lets you embed build output manifests, dependency graphs, or configuration snapshots directly in the cache, so a single open() tells you both "did anything change?" and "what was the previous result?" — no separate metadata files needed.

Why it's fast — and safe:

• Stat-first validation — a warm check is one stat() per file (O(n)), not a read + hash of every byte (O(Σ file sizes)); only files whose stat changed are re-hashed. Typically 10-100× faster for warm caches.
• Fast native XXH3 + LZ4 — SIMD xxHash3-128 (AVX2/AVX-512 auto-dispatch) and LZ4, run on a work-stealing thread pool in a single open → fstat → read pass with overlapped readahead and zero-copy buffers — multiple GB/s per core. Both are exported as standalone APIs too.
• Concurrency-safe — an OS-level flock (exclusive to write, shared to read) means no build ever sees a half-written cache, with cancellable, lockTimeoutMs-bounded acquisition and no lockfiles to leak. lock() exposes the same lock as a standalone cross-process mutex.

FileHashCache benchmarks (705 files, ~24 MiB)

Native (C++ addon):

| Scenario | Mean | Hz | Files/s | Throughput | | ------------------ | -------------------- | ---------- | ----------------- | ---------- | | no change | 0.5 ms (491.2 µs) | 2 036 op/s | 1 435 194 files/s | — | | 1 file changed | 0.8 ms (765.6 µs) | 1 306 op/s | 920 810 files/s | — | | many files changed | 2.3 ms (2 289.4 µs) | 437 op/s | 307 941 files/s | 10.8 GB/s | | no existing cache | 10.0 ms (9 967.5 µs) | 100 op/s | 70 730 files/s | 2.5 GB/s | | overwrite | 9.9 ms (9 926.7 µs) | 101 op/s | 71 020 files/s | 2.5 GB/s |

Node.js v26.5.0, Vitest 4.x — Apple M4 Max, macOS 25.5.0 (arm64), with anti-virus.

Results vary by hardware, file sizes, and OS cache state.

FileHashCache API

A long-lived cache that tracks file content hashes with exclusive OS-level locking. Create the instance once, then call open() on each build cycle. Configuration (files, version, fingerprint) is set via the constructor, setters, or configure().

Example: Build cache with dynamic file list

The typical usage: the file list is only known after a build step. Open without files (reuses the list from the previous cache on disk), then set the new file list before writing. Use compressedPayloads (LZ4-compressed inside the cache body) or uncompressedPayloads (stored raw, readable without decompression) to store arbitrary build metadata alongside the cache.

import { FileHashCache } from "fast-fs-hash";

const cache = new FileHashCache({
  cachePath: ".cache/build.fsh",
  rootPath: ".",
  version: 1,
});

export async function build() {
  using session = await cache.open();

  if (session.status === "upToDate" && session.compressedPayloads.length > 0) {
    return JSON.parse(session.compressedPayloads[0].toString()); // cached result
  }

  const result = await runBuild();

  cache.configure({ files: result.getSourceFiles().map((f) => f.fileName) });

  await session.write({
    compressedPayloads: [Buffer.from(JSON.stringify(result.output))],
  });

  return result.output;
}

Example: Simple build cache with known files

When the file list is known upfront, pass it to the constructor:

import { FileHashCache } from "fast-fs-hash";
import { globSync } from "node:fs";

const cache = new FileHashCache({
  cachePath: ".cache/build.fsh",
  rootPath: ".",
  files: globSync("src/**/*.ts"),
  version: 1,
});

using session = await cache.open();

if (session.status === "upToDate") {
  console.log("Build cache is fresh — skipping.");
} else {
  console.log("Files changed — rebuilding...");
  await runBuild();
  await session.write();
}

API Reference

Constructor: new FileHashCache({ cachePath, files?, rootPath?, version?, fingerprint?, lockTimeoutMs? })

Cache configuration (mutable between opens):

• configure(opts) — set multiple config fields at once: files, rootPath, version, fingerprint, lockTimeoutMs
• Setters: cache.files, cache.rootPath, cache.version, cache.fingerprint, cache.lockTimeoutMs
• needsOpen — true when config changed since last open, or cache was never opened

Cache properties (read-only):

• cachePath — the cache file path
• fileCount — number of configured files
• busy — true while an async cache operation (open/lock/overwrite) is in flight
• activeSession — the currently-open FileHashCacheSession, or null

Cache methods:

• open(signal?) — acquires an exclusive lock, reads from disk, validates version/fingerprint, stat-matches entries. Returns a FileHashCacheSession.
• lock(signal?) — acquires the exclusive lock without reading or parsing the cache file (no entries, no payloads, no stat loop). Returns a lock-only FileHashCacheSession (status === 'missing', lockOnly === true). Use it when you only need cross-process mutual exclusion on the cache path. Calling write(options) on the returned session writes a fresh cache reusing the already-held lock — equivalent to overwrite() but without a release / re-acquire round-trip. See Lock-only sessions.
• overwrite(options?) — writes a brand-new cache without reading the old one. Options: payloadValue0..3, compressedPayloads, uncompressedPayloads, signal, lockTimeoutMs.
• invalidate(paths) / invalidateAll() — mark files as dirty for the next open (watch mode).
• isLocked() / waitUnlocked(timeout?, signal?) — check or wait for lock.
• checkCacheFile() — sync stat check if the cache file on disk changed since last open.

Session properties (read-only, from disk):

• status — see Cache status below
• lockOnly — true when opened via cache.lock(): the session holds only the OS lock and carries no cache data. Its disk-backed accessors (compressedPayloads, uncompressedPayloads) return empty, needsWrite is false, and resolve() throws. files still reflects the configured file list (status is 'missing')
• cache — the parent FileHashCache
• needsWrite — true if the session holds the lock and the status indicates changes
• configChanged — true if cache config was modified since this session was opened
• wouldNeedWrite — true if either files changed on disk or config changed
• busy / disposed — async operation state
• files, fileCount, version, rootPath
• diskVersion — the user version (u32) read from the on-disk header. Differs from version only when status === 'staleVersion'; lets migration code identify which old format to parse.
• payloadValue0..3 — four f64 numeric values read from disk
• compressedPayloads — array of LZ4-compressed binary Buffer payloads read from disk
• uncompressedPayloads — array of raw binary Buffer payloads readable without LZ4 decompression

Cache status

| Status | Disk readable? | Entries trustable? | Payloads readable? | Action | | ---------------- | ----------------- | ------------------------------ | ------------------ | ---------------------------------------------------------------------- | | 'upToDate' | Yes | Yes | Yes | None | | 'statsDirty' | Yes | Yes (content unchanged) | Yes | Rewrite cache (stats refreshed) | | 'changed' | Yes | Partially (some files changed) | Yes | Re-hash changed files, write | | 'stale' | Yes (well-formed) | No (fingerprint mismatch) | Yes | Discard entries, optionally migrate payloads, write | | 'staleVersion' | Yes (well-formed) | No (version differs) | Yes | As above, plus check session.diskVersion for version-aware migration | | 'missing' | No | — | No | Write fresh | | 'lockFailed' | — | — | — | Retry later |

'stale' and 'staleVersion' both keep the on-disk buffer readable so callers can migrate compressedPayloads / uncompressedPayloads / payloadValueN across config changes. 'missing' covers genuinely unreadable files (no file, truncated header, bad magic, corrupt body) — never thrown, always reported via this status so callers can recover by writing fresh.

Session methods:

• write(options?) — hashes unresolved entries, compresses, writes to disk, releases lock. Can only be called once. Options: payloadValue0..3, compressedPayloads, uncompressedPayloads, signal. On a lock-only session (lockOnly === true) it writes a fresh cache from the supplied options, reusing the already-held lock.
• resolve(signal?) — completes stat + hash for ALL files, returns FileHashCacheEntries. Can be called before write(). See below.
• close() — releases the lock. Also called automatically by using.

Static methods:

• FileHashCache.isLocked(cachePath) — check if locked by another process
• FileHashCache.waitUnlocked(cachePath, lockTimeoutMs?, signal?) — wait for unlock

Lock behavior:

• Cross-process exclusive lock via flock(2) (POSIX) / LockFileEx (Windows)
• Cross-thread safe: per-OFD semantics on POSIX and per-handle on Windows make worker_threads in the same process serialize correctly against each other
• Crash-safe: automatically released when the process dies
• lockTimeoutMs: -1 = block forever (default), 0 = non-blocking, >0 = timeout ms
• When lock fails: status === 'lockFailed'. Calling write() falls back to overwrite().
• Cancellable via AbortSignal on open(), overwrite(), and waitUnlocked()

Inspecting per-file changes with resolve()

After open(), the session knows the aggregate status but not which specific files changed. Call resolve() to complete stat + hash for every file and get per-file metadata.

Note: resolve() stats and hashes every unresolved file on the thread pool. This has a cost proportional to the number of changed files. Use it only when you need per-file information — for simple "changed → rebuild all" workflows, just check session.status.

using session = await cache.open();

if (session.status !== "upToDate") {
  const entries = await session.resolve();

  for (const entry of entries) {
    if (entry.changed) {
      console.log(
        `Changed: ${entry.path} (${entry.size} bytes, hash: ${entry.contentHashHex})`,
      );
    }
  }

  await session.write();
}

Each FileHashCacheEntry provides:

• path — absolute file path
• size — file size in bytes
• mtimeMs / ctimeMs — modification / change time in ms
• changed — true if content differs from the cached version (or is a new file)
• contentHash — 16-byte xxHash3-128 as a Buffer (zero-copy view)
• contentHashHex — 32-char hex string (lazy, computed on first access)

FileHashCacheEntries supports get(index), find(path), and iteration. The result is cached — subsequent calls to resolve() return the same snapshot.

Lock-only sessions

cache.lock() acquires the cross-process lock without reading or parsing the cache file — no entries, no payloads, no stat loop. Use it when you only need mutual exclusion on the cache path, or to write a fresh cache while holding the lock the whole time (no release / re-acquire between deciding and writing).

using session = await cache.lock();

if (session.status === "lockFailed") {
  // another holder owns the lock — retry later
} else {
  // session.lockOnly === true, session.status === "missing"
  // write a fresh cache reusing the already-held lock:
  await session.write({ payloadValue0: Date.now() });
}

The returned session carries no disk-backed cache data: compressedPayloads and uncompressedPayloads return empty, needsWrite is false, and resolve() throws. files still reflects the configured file list (status is 'missing'). close() (or using) releases the lock as usual.

xxHash128 — Direct hashing

When you don't need a persistent cache file — or you want raw xxHash3-128 digests to compare yourself — use the digest functions directly. FileHashCache uses them under the hood, but they are fully usable on their own.

File hashing benchmarks

large file (~197.3 KB):

| Scenario | Mean | Hz | Throughput | Relative | | -------------------- | ----------------- | ----------- | ---------- | --------------- | | native | 0.04 ms (41.6 µs) | 24 030 op/s | 4.7 GB/s | 6.6× faster | | Node.js crypto (md5) | 0.3 ms (274.3 µs) | 3 645 op/s | 719 MB/s | baseline |

medium file (~49.9 KB):

| Scenario | Mean | Hz | Throughput | Relative | | -------------------- | ----------------- | ----------- | ---------- | --------------- | | native | 0.03 ms (27.3 µs) | 36 681 op/s | 1.8 GB/s | 4.0× faster | | Node.js crypto (md5) | 0.1 ms (109.0 µs) | 9 171 op/s | 458 MB/s | baseline |

small file (~1.0 KB):

| Scenario | Mean | Hz | Relative | | -------------------- | ----------------- | ----------- | --------------- | | native | 0.03 ms (27.7 µs) | 36 107 op/s | 2.0× faster | | Node.js crypto (md5) | 0.06 ms (55.9 µs) | 17 896 op/s | baseline |

Parallel file hashing (705 files)

| Scenario | Mean | Hz | Throughput | Relative | | -------------------- | --------------------- | ------- | ---------- | --------------- | | native | 10.3 ms (10 274.0 µs) | 97 op/s | 2.4 GB/s | 3.4× faster | | Node.js crypto (md5) | 35.4 ms (35 431.2 µs) | 28 op/s | 697 MB/s | baseline |

In-memory buffer hashing

64 KB buffer:

| Scenario | Mean | Hz | Throughput | Relative | | ------------------ | ----------------- | ------------ | ---------- | ---------------- | | native XXH3-128 | 0.001 ms (1.4 µs) | 711 505 op/s | 46.6 GB/s | 47.4× faster | | Node.js crypto md5 | 0.07 ms (66.6 µs) | 15 013 op/s | 984 MB/s | baseline |

1 MB buffer:

| Scenario | Mean | Hz | Throughput | Relative | | ------------------ | ------------------- | ----------- | ---------- | ---------------- | | native XXH3-128 | 0.02 ms (21.9 µs) | 45 762 op/s | 48.0 GB/s | 48.4× faster | | Node.js crypto md5 | 1.1 ms (1 056.9 µs) | 946 op/s | 992 MB/s | baseline |

Hash files

import { digestFilesParallel, hashToHex } from "fast-fs-hash";

const digest = await digestFilesParallel([
  "package.json",
  "src/index.ts",
  "src/utils.ts",
]);
console.log("Aggregate:", hashToHex(digest));

Sequential variant (feeds files into a single running hash):

import { digestFilesSequential, hashToHex } from "fast-fs-hash";

const digest = await digestFilesSequential(["package.json", "src/index.ts"]);
console.log(hashToHex(digest));

Hash a single file

import { digestFile, hashToHex } from "fast-fs-hash";

const digest = await digestFile("package.json");
console.log(hashToHex(digest));

Hex string convenience

import { digestFileToHex, digestFilesToHexArray } from "fast-fs-hash";

// Single file → 32-char hex string
const hex = await digestFileToHex("package.json");

// Multiple files in parallel → per-file hex strings
const hexes = await digestFilesToHexArray(["src/a.ts", "src/b.ts"], 8);

| Function | Description | | ----------------------------------------------------------- | ----------------------------------------------------------------------- | | digestFileToHex(path, throwOnError?) | Hash a file → 32-char hex string. Wrapper around digestFile + hashToHex | | digestFilesToHexArray(paths, concurrency?, throwOnError?) | Hash files in parallel → per-file hex strings. Default concurrency 8 |

Hash buffers and strings

import { digestBuffer, digestString } from "fast-fs-hash";

const d1 = digestBuffer(myBuffer);
const d2 = digestString("hello world");
console.log(d2.toString("hex"));

Range variant hashes a slice: digestBufferRange(input, offset, length?).

Zero-allocation …To variants

Every digest* function has a …To sibling that writes the 16-byte digest into a caller-provided out buffer (at outOffset, default 0) instead of allocating a fresh Buffer, and returns out. Use these on hot paths to avoid per-call allocation — e.g. packing many digests into one large buffer.

import { digestBufferTo, digestFileTo } from "fast-fs-hash";

const out = Buffer.allocUnsafe(32);
digestBufferTo(a, out, 0); // first 16 bytes
digestBufferTo(b, out, 16); // next 16 bytes
await digestFileTo("package.json", out, 0); // async file variant

| Function | Writes into out the digest of… | | ----------------------------------------------------------------- | -------------------------------------------- | | digestBufferTo(input, out, outOffset?) | a whole buffer | | digestBufferRangeTo(input, offset, length, out, outOffset?) | a buffer slice | | digestStringTo(input, out, outOffset?) | a UTF-8 string | | digestFileTo(path, out, outOffset?, throwOnError?) | a file (async) | | digestFilesSequentialTo(paths, out, outOffset?, throwOnError?) | files hashed sequentially (async) | | digestFilesParallelTo(paths, out, outOffset?, concurrency?, …?) | files hashed in parallel, aggregated (async) |

Streaming class

For combining file hashes with extra data (config, environment, etc.):

import { XxHash128Stream } from "fast-fs-hash";

const h = new XxHash128Stream();
h.addString("my-config-v2");
await h.addFiles(["src/index.ts", "src/utils.ts"]);
console.log(h.digest().toString("hex"));

Busy guard: Async methods (addFile, addFiles, addFilesParallel) mark the instance as busy while the native worker thread is processing. During this time, calling any synchronous method or starting another async operation will throw an error. Always await each async call before invoking another method. Use the busy getter to check:

const h = new XxHash128Stream();
const promise = h.addFile("large.bin");
console.log(h.busy); // true — async operation in flight
// h.addString("oops"); // would throw!
await promise;
console.log(h.busy); // false — safe to use again

Seeding: new XxHash128Stream(seedLow?, seedHigh?) seeds the hash with two 32-bit halves (default 0); the current seed is exposed via the seedLow / seedHigh fields.

Full method list:

| Method | Description | | ------------------------------------------- | ------------------------------------------------------------------- | | addBuffer(input) | Feed a whole buffer (sync) | | addBufferRange(input, offset, length?) | Feed a buffer slice (sync) | | addString(input) | Feed a UTF-8 string (sync) | | addFile(path, throwOnError?) | Read and feed a file (async) | | addFiles(paths, throwOnError?) | Read and feed files sequentially (async) | | addFilesParallel(paths, concurrency?, …?) | Read and feed files in parallel (async) | | digest() | Finalize → 16-byte Buffer (does not consume state) | | digestTo(out, outOffset?) | Finalize into a caller-provided buffer (zero-alloc), returns out | | reset(seedLow?, seedHigh?) | Reset to empty, optionally reseed — reuse the instance | | clone() | Deep-copy the current hashing state into a new independent instance | | XxHash128Stream.hash(input) (static) | One-shot digest of a buffer or string |

The class also mirrors the standalone functions as statics (XxHash128Stream.digestBuffer, .digestFile, .digestFilesParallelTo, …). The IXxHash128Functions type describes this shared surface.

LZ4 Block Compression

fast-fs-hash exposes the LZ4 block compression API used internally for the cache file format. Both synchronous and asynchronous (pool-thread) variants are available.

LZ4 block format does not embed the uncompressed size — the caller must store it alongside the compressed data and pass it to the decompression function.

compress 64 KB:

| Scenario | Ratio | Mean | Hz | Throughput | Relative | | ----------------------- | ----- | ----------------- | ------------ | ---------- | --------------- | | native LZ4 | 0.7% | 0.003 ms (3.0 µs) | 334 654 op/s | 21.9 GB/s | 8.5× faster | | Node.js deflate level=1 | 1.0% | 0.03 ms (25.4 µs) | 39 311 op/s | 2.6 GB/s | baseline |

decompress 64 KB:

| Scenario | Mean | Hz | Throughput | Relative | | --------------- | ----------------- | ------------ | ---------- | --------------- | | native LZ4 | 0.002 ms (2.1 µs) | 473 807 op/s | 31.1 GB/s | 4.0× faster | | Node.js deflate | 0.008 ms (8.4 µs) | 118 896 op/s | 7.8 GB/s | baseline |

compress 1 MB:

| Scenario | Ratio | Mean | Hz | Throughput | Relative | | ----------------------- | ----- | ----------------- | ----------- | ---------- | ---------------- | | native LZ4 | 0.4% | 0.03 ms (34.3 µs) | 29 124 op/s | 30.5 GB/s | 10.1× faster | | Node.js deflate level=1 | 0.7% | 0.3 ms (346.3 µs) | 2 888 op/s | 3.0 GB/s | baseline |

decompress 1 MB:

| Scenario | Mean | Hz | Throughput | Relative | | --------------- | ----------------- | ----------- | ---------- | --------------- | | native LZ4 | 0.03 ms (29.5 µs) | 33 951 op/s | 35.6 GB/s | 3.0× faster | | Node.js deflate | 0.09 ms (86.9 µs) | 11 504 op/s | 12.1 GB/s | baseline |

import {
  lz4CompressBlock,
  lz4DecompressBlock,
  lz4CompressBound,
} from "fast-fs-hash";

const input = Buffer.from("Hello, LZ4!");
const compressed = lz4CompressBlock(input);
const decompressed = lz4DecompressBlock(compressed, input.length);
console.log(decompressed.toString()); // "Hello, LZ4!"

LZ4 API

| Function | Description | | -------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------ | | lz4CompressBlock(input, offset?, length?) | Sync compress → new Buffer | | lz4CompressBlockTo(input, output, outputOffset?, inputOffset?, inputLength?) | Sync compress into pre-allocated buffer → bytes written | | lz4CompressBlockAsync(input, offset?, length?) | Async compress on pool thread → Promise<Buffer> | | lz4DecompressBlock(input, uncompressedSize, offset?, length?) | Sync decompress → new Buffer | | lz4DecompressBlockTo(input, uncompressedSize, output, outputOffset?, inputOffset?, inputLength?) | Sync decompress into pre-allocated buffer → bytes written | | lz4DecompressBlockAsync(input, uncompressedSize, offset?, length?) | Async decompress on pool thread → Promise<Buffer> | | lz4CompressBound(inputSize) | Max compressed size for pre-allocation | | lz4ReadAndCompress(path) | Read a file and LZ4-compress it on pool thread → Promise<{data, uncompressedSize}> | | lz4DecompressAndWrite(compressedData, uncompressedSize, path) | Decompress and write to file on pool thread (creates dirs) → Promise<boolean> |

Note: LZ4 block compression supports inputs up to ~1.9 GiB (LZ4_MAX_INPUT_SIZE = 0x7E000000). lz4ReadAndCompress caps the source file at 512 MiB; lz4DecompressAndWrite caps uncompressedSize at 256 MiB.

Read and compress a file

lz4ReadAndCompress reads a file and LZ4-block-compresses it in a single pool-thread operation — no JS-thread I/O, no intermediate Buffer allocation visible to the event loop.

import {
  lz4ReadAndCompress,
  lz4DecompressAndWrite,
  lz4DecompressBlock,
} from "fast-fs-hash";

const { data, uncompressedSize } = await lz4ReadAndCompress("large-file.bin");
console.log(`Compressed ${uncompressedSize} → ${data.length} bytes`);

// Decompress back to a file (creates parent directories if needed)
await lz4DecompressAndWrite(data, uncompressedSize, "restored-file.bin");

// Or decompress to a buffer in memory
const original = lz4DecompressBlock(data, uncompressedSize);

File Comparison

Compare two files for byte-equality asynchronously on a native pool thread. Opens both files, compares sizes via fstat, then reads in lockstep chunks with memcmp. Returns false if either file cannot be opened/read or if sizes differ — never throws.

equal files (~49.9 KB):

| Scenario | Mean | Hz | Throughput | Relative | | ---------------------------------- | ----------------- | ----------- | ---------- | --------------- | | native | 0.04 ms (39.9 µs) | 25 033 op/s | 1.2 GB/s | 2.7× faster | | Node.js (fs.open + read + compare) | 0.1 ms (106.3 µs) | 9 406 op/s | 469 MB/s | baseline |

equal files (~197.3 KB):

| Scenario | Mean | Hz | Throughput | Relative | | ---------------------------------- | ----------------- | ----------- | ---------- | --------------- | | native | 0.05 ms (49.3 µs) | 20 264 op/s | 4.0 GB/s | 3.4× faster | | Node.js (fs.open + read + compare) | 0.2 ms (165.6 µs) | 6 038 op/s | 1.2 GB/s | baseline |

different content, same size (~49.9 KB):

| Scenario | Mean | Hz | Throughput | Relative | | ---------------------------------- | ----------------- | ----------- | ---------- | --------------- | | native | 0.04 ms (35.4 µs) | 28 229 op/s | 1.4 GB/s | 2.8× faster | | Node.js (fs.open + read + compare) | 0.1 ms (100.5 µs) | 9 949 op/s | 496 MB/s | baseline |

different sizes (early exit):

| Scenario | Mean | Hz | Relative | | ---------------------------------- | ----------------- | ----------- | --------------- | | native | 0.04 ms (37.0 µs) | 27 015 op/s | 2.3× faster | | Node.js (fs.open + read + compare) | 0.08 ms (84.3 µs) | 11 865 op/s | baseline |

import { filesEqual } from "fast-fs-hash";

if (await filesEqual("output.bin", "expected.bin")) {
  console.log("Files are identical");
} else {
  console.log("Files differ (or one doesn't exist)");
}

| Function | Description | | -------------------------- | ------------------------------------------------------------- | | filesEqual(pathA, pathB) | Async byte-equality check on pool thread → Promise<boolean> |

Find Project Root

Walk the parent chain from a start path and locate project markers in a single pass: .git, package.json, tsconfig.json, and node_modules/. Reports nearest* (first hit walking up) and root* (last hit, bounded by the enclosing .git) for each marker, plus gitRoot and gitSuperRoot for submodule/worktree awareness.

The walk stops at the filesystem root, the user's home directory (or any ancestor of it), an optional stopPath, and a depth cap of 128 (symlink-loop defense). Tolerant of missing paths — if startPath doesn't exist, the walk begins from its longest existing ancestor and missing markers are returned as null rather than thrown.

shallow (3 levels deep):

| Scenario | Mean | Hz | Relative | | --------------------------- | ----------------- | ----------- | --------------- | | native (sync) | 0.05 ms (45.0 µs) | 22 206 op/s | 5.8× faster | | native (async) | 0.06 ms (56.3 µs) | 17 767 op/s | 4.6× faster | | Node.js (sync, fs.statSync) | 0.1 ms (119.6 µs) | 8 364 op/s | 2.2× faster | | Node.js (async, fs.stat) | 0.3 ms (259.3 µs) | 3 857 op/s | baseline |

deep (12 levels deep):

| Scenario | Mean | Hz | Relative | | --------------------------- | ----------------- | ----------- | --------------- | | native (sync) | 0.09 ms (92.4 µs) | 10 823 op/s | 7.5× faster | | native (async) | 0.1 ms (105.0 µs) | 9 527 op/s | 6.6× faster | | Node.js (sync, fs.statSync) | 0.4 ms (350.7 µs) | 2 852 op/s | 2.0× faster | | Node.js (async, fs.stat) | 0.7 ms (692.9 µs) | 1 443 op/s | baseline |

missing start path (tolerant fallback):

| Scenario | Mean | Hz | Relative | | --------------------------- | ----------------- | ----------- | --------------- | | native (sync) | 0.05 ms (54.4 µs) | 18 393 op/s | 2.7× faster | | Node.js (sync, fs.statSync) | 0.1 ms (144.8 µs) | 6 904 op/s | baseline |

import { findProjectRoot, findProjectRootSync } from "fast-fs-hash";

// Sync (recommended for startup-time / build-tool use)
const info = findProjectRootSync(import.meta.dirname);
console.log(info.gitRoot, info.rootPackageJson, info.nearestTsconfigJson);

// Async — runs on the native thread pool, useful on cold or networked filesystems
const info2 = await findProjectRoot("/some/deep/file.ts");

// Optional stopPath: halt when the walker reaches this directory (or any ancestor of it)
const info3 = findProjectRootSync(start, "/workspace");

| Function | Description | | ------------------------------------------ | ------------------------------------------------------------ | | findProjectRootSync(startPath, stopPath) | Walk parent chain for project markers (sync) → ProjectRoot | | findProjectRoot(startPath, stopPath) | Walk parent chain on pool thread → Promise<ProjectRoot> |

The returned ProjectRoot object has these fields (each string | null):

| Field | Description | | --------------------- | ------------------------------------------------------------------------------ | | gitRoot | Innermost .git (dir or file). Matches git rev-parse --show-toplevel. | | gitSuperRoot | Outermost .git directory. Non-null only in submodules / nested worktrees. | | nearestPackageJson | First package.json walking up. | | rootPackageJson | Last package.json walking up, bounded by gitRoot. | | nearestTsconfigJson | First tsconfig.json walking up. | | rootTsconfigJson | Last tsconfig.json walking up, bounded by gitRoot. | | nearestNodeModules | First node_modules/ directory walking up (also detects when started inside). | | rootNodeModules | Last node_modules/ walking up, bounded by gitRoot. |

Find Nearest Project Files

A trimmed-down sibling of findProjectRoot that finds only the nearest package.json, tsconfig.json, and node_modules/ and exits the walk as soon as all three are populated. No .git probe, no gitRoot boundary, no root* fields — faster than findProjectRoot when callers don't need them.

The walk stops at the filesystem root, the user's home directory (or any ancestor of it), an optional stopPath, and a depth cap of 128. Tolerant of missing paths — missing fields are returned as null rather than throwing.

shallow (3 levels deep):

| Scenario | Mean | Hz | Relative | | --------------------------- | ----------------- | ----------- | --------------- | | native (sync) | 0.03 ms (31.1 µs) | 32 153 op/s | 4.0× faster | | native (async) | 0.04 ms (41.0 µs) | 24 404 op/s | 3.0× faster | | Node.js (sync, fs.statSync) | 0.06 ms (58.5 µs) | 17 097 op/s | 2.1× faster | | Node.js (async, fs.stat) | 0.1 ms (124.5 µs) | 8 033 op/s | baseline |

deep (12 levels deep):

| Scenario | Mean | Hz | Relative | | --------------------------- | ----------------- | ----------- | --------------- | | native (sync) | 0.07 ms (70.0 µs) | 14 286 op/s | 6.6× faster | | native (async) | 0.08 ms (81.0 µs) | 12 340 op/s | 5.7× faster | | Node.js (sync, fs.statSync) | 0.2 ms (227.1 µs) | 4 403 op/s | 2.0× faster | | Node.js (async, fs.stat) | 0.5 ms (462.4 µs) | 2 163 op/s | baseline |

missing start path (tolerant fallback):

| Scenario | Mean | Hz | Relative | | --------------------------- | ----------------- | ----------- | --------------- | | native (sync) | 0.03 ms (34.1 µs) | 29 284 op/s | 2.2× faster | | Node.js (sync, fs.statSync) | 0.08 ms (76.5 µs) | 13 077 op/s | baseline |

import {
  findNearestProjectFiles,
  findNearestProjectFilesSync,
} from "fast-fs-hash";

// Sync (recommended for startup-time / build-tool use)
const info = findNearestProjectFilesSync(import.meta.dirname);
console.log(info.packageJson, info.tsconfigJson, info.nodeModules);

// Async — runs on the native thread pool
const info2 = await findNearestProjectFiles("/some/deep/file.ts");

// Optional stopPath: halt when the walker reaches this directory (or any ancestor of it)
const info3 = findNearestProjectFilesSync(start, "/workspace");

| Function | Description | | -------------------------------------------------- | -------------------------------------------------------------------- | | findNearestProjectFilesSync(startPath, stopPath) | Walk parent chain for nearest markers (sync) → NearestProjectFiles | | findNearestProjectFiles(startPath, stopPath) | Walk parent chain on pool thread → Promise<NearestProjectFiles> |

The returned NearestProjectFiles object has these fields (each string | null):

| Field | Description | | -------------- | ------------------------------------------------------------------------------ | | packageJson | First package.json walking up. | | tsconfigJson | First tsconfig.json walking up. | | nodeModules | First node_modules/ directory walking up (also detects when started inside). |

Utility Functions

| Function | Description | | ---------------------------------------------------- | -------------------------------------------------------------------- | | hashToHex(digest) | Convert a 16-byte digest to a 32-char hex string | | hashesToHexArray(digests) | Convert an array of digests to hex strings | | findCommonRootPath(files, baseRoot?, allowedRoot?) | Longest common parent directory of file paths | | normalizeFilePaths(rootPath, files) | Resolve, sort, deduplicate paths relative to root | | toRelativePath(rootPath, filePath) | Single path → clean unix-style relative path (or null) | | threadPoolTrim() | Wake idle native pool threads so they self-terminate and free memory |

Environment Variables

| Variable | Default | Description | | ----------------------------------- | ----------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------- | | FAST_FS_HASH_ISA | auto-detect | Override SIMD variant: avx512, avx2, or baseline (x64 only) | | FAST_FS_HASH_POOL_IDLE_TIMEOUT_MS | 15000 | Idle timeout for native pool threads (1–3600000 ms). Threads self-terminate after this duration with no work. They respawn automatically when new work arrives. |

Acknowledgements

The native C++ backend uses:

• xxHash by Yann Collet — xxHash3-128 hashing (BSD 2-Clause License)
• LZ4 by Yann Collet — block compression (BSD 2-Clause License)

See NOTICES.md for full license texts.

Building from source

Prerequisites

| Tool | Version | Install | | -------------- | ------------------------------- | --------------------------------------------------------------------------- | | Node.js | >= 22 | nodejs.org | | npm | >= 9 | bundled with Node.js | | CMake | >= 3.15 | brew install cmake / apt install cmake / cmake.org | | C++20 compiler | Clang 14+ / GCC 12+ / MSVC 2022 | Xcode CLT / build-essential / Visual Studio |

Quick start

git clone --recurse-submodules https://github.com/SalvatorePreviti/fast-fs-hash.git
cd fast-fs-hash
npm install
npm run build:all   # compile C++ addon + TypeScript
npm test            # run tests
npm run bench       # run benchmarks

Note: git clone --recurse-submodules is required to pull deps/xxHash (the xxHash source used by the native addon).

Git submodule (xxHash)

The deps/xxHash/ directory is a git submodule pointing to xxHash v0.8.3.

If you cloned without --recurse-submodules, initialize the submodule manually:

git submodule update --init --recursive

See package.json for the full list of available build scripts.

Release process

• main — development branch. CI runs lint, typecheck, tests, and builds native binaries for all platforms on every push and PR.
• publish — release branch. Pushing to publish triggers the full CI pipeline. After all builds and tests pass, a dry-run publish verifies all packages. An admin must then manually approve the publish job (via the npm-publish GitHub environment) to publish to npm, create a git tag, and deploy docs.

npm packages are published with provenance attestations via GitHub Actions OIDC — no npm tokens are stored in CI.

License

MIT — Copyright (c) 2025-present Salvatore Previti