Market Data Freshness

Pure, bounded checks for candle coverage and reconciliation freshness. A recent cache tail cannot conceal an interior gap.

Experimental v0.1.1 · TypeScript · ESM · Node.js 22+ · MIT · zero runtime dependencies

API reference · Changelog · Documentation map · Report an issue

When to use this

Gate a backtest or data pipeline on candle coverage, distinguish cache collection from reconciliation delay, and expose missing or uncertain market data without inventing healthy zero values.

Freshness is not proof of execution or complete price history. A FRESH result can still have missing tail bars within the configured budget.

Get started

Install the ESM package:

npm install @farhadarjmand/market-data-freshness

To build and test from source:

git clone https://github.com/farhad-arjmand/market-data-freshness.git
cd market-data-freshness
npm ci --ignore-scripts
npm run check

import { assessFreshness } from '@farhadarjmand/market-data-freshness';

const result = assessFreshness([
  { openTime: 0, closeTime: 60000, contentHash: 'synthetic-a' },
  // [60000, 120000) is missing.
  { openTime: 120000, closeTime: 180000, contentHash: 'synthetic-c' },
], {
  nowMs: 180000,
  intervalMs: 60000,
  coverageThrough: 0,
  budgetMs: 180000,
});
console.log(result.status); // STALE
console.log(result.issues); // includes INTERIOR_GAP

Package: @farhadarjmand/market-data-freshness.

Four concepts that should not be confused

• Market clock: latest closed candle boundary.
• Collection: which closed bars were actually available by now.
• Reconciliation: the last successfully processed boundary.
• Publication/process health: a separate clock, not proof of market coverage.

The caller supplies all clocks. The module never reads Date.now(), a file timestamp, the filesystem, or the network.

Time contract

All timestamps are nonnegative integer epoch milliseconds. Bars use [openTime, closeTime) and have exactly intervalMs duration on an anchorMs grid (default 0). Convert inclusive provider end timestamps explicitly.

availableAt, when supplied, must be at least closeTime; future receipts and unclosed bars are ignored. If omitted, availability at close is an assumption, not a measured delivery guarantee.

coverageThrough is an exclusive, grid-aligned reconciliation watermark. Null means unknown. The checker trusts that watermark; it does not verify previously reconciled history. Passing a fabricated watermark can produce a fabricated freshness result.

Verdicts and counts

| Status | Meaning | | --- | --- | | FRESH | Complete classified scan, no interior gaps/integrity issues, and collection/reconciliation tail ages within the supplied budget. | | STALE | Missing required interior data, unavailable data for a nonempty required window, or a lag violation. | | UNKNOWN | Missing watermark, bounded incomplete scan, unknown calendar state, or duplicates lacking comparable payload hashes. | | INVALID | Malformed bars/watermark, conflicting duplicates, or calendar contradicting received data. |

Integrity errors take priority over UNKNOWN, which takes priority over STALE. Every discovered issue is retained. A capped scan may still discover some gaps, but its counts are partial and its lag estimates remain null.

FRESH is a budget-based timeliness verdict, not proof of zero missing tail bars. Missing interior bars are always stale; a not-yet-collected tail can be within budget and is explicitly counted. Inspect missingTailBars, gaps, and contiguousThrough if your application requires complete coverage through the latest close.

The default maxBars is 10,000; maxReportedGaps is 100. Scan work is bounded, but indexing still reads the supplied array in O(n); callers must bound input size. No backfill or fabricated bars are produced.

Calendars and duplicate payloads

sessionAt(openTime, closeTime) returns OPEN, CLOSED, or UNKNOWN for the whole slot. Default is OPEN for every slot. Only a positive CLOSED result excuses missing bars. UNKNOWN never certifies freshness. Calendar exceptions propagate rather than becoming healthy. No exchange calendar is bundled; supply the correct venue/instrument calendar.

Tradeable lag excludes classified CLOSED slots; wall-clock lag is reported separately. A bar arriving in a slot your calendar declares closed is a contradiction, not silently accepted evidence.

contentHash is an optional caller-computed digest of the full canonical candle payload (not a secret). Duplicate rows with differing hashes are invalid. Duplicates without hashes are unknown because timestamps cannot prove payload equality. The library does not compute or authenticate hashes. Verdicts and duplicate counters are permutation-invariant, including mixtures of missing and conflicting hashes. See the API reference for counter definitions.

API

• latestClosedBoundary(nowMs, intervalMs, anchorMs = 0)
• assessFreshness(bars, options)
• stageLatency(start, end): same-clock duration; missing => UNKNOWN, inverted/invalid => INVALID, never silently zero.

Coverage/lag values are null when unmeasurable. Gap through is exclusive. omittedGapRanges counts ranges omitted by the reporting bound, separate from the scan bound.

There is deliberately no order, fill, TP/SL, account, calendar-provider, or trading-system adapter. Freshness does not settle a position and does not prove that a price barrier did or did not trade.

Verification

Synthetic tests cover fresh-tail/interior-hole cases, latency clocks, bounds, duplicates, calendar uncertainty and contradictions, receipt delays, watermark errors, and order-independent input.

See Provenance, Contributing, and MIT license.

Integration and reproducibility

ESM named imports only; tested on Node.js 22 and 24. Types are bundled. Browser and CommonJS support are not claimed. The package includes docs/API.md and llms.txt so humans and coding assistants can inspect the installed version's contract offline. A documentation map does not guarantee search ranking or AI indexing.

For contributors, npm run test:package installs a freshly packed tarball in a temporary consumer, executes the README example, checks a functional assertion and type-checks imports by the public package name. Pin the package version and retain your input identity and options when comparing results.