@lp-labs/forecast-evaluator
v0.1.0
Published
Bounded deterministic forecast-evaluator utilities with explicit validation and zero runtime dependencies.
Readme
@lp-labs/forecast-evaluator
Ordered time splits, training-only naive baselines, paired error metrics, and explicit feature-timing leakage diagnosis. Original independent implementation, zero runtime/peer dependencies, Node >=18, version 0.1.0. Evaluation is not predictive certainty.
npm install @lp-labs/forecast-evaluatorRunnable example
import {
splitTimeSeries,
evaluateBaseline,
evaluateForecast,
diagnoseLeakage,
} from '@lp-labs/forecast-evaluator';
const split = splitTimeSeries(
[
{ time: 1, value: 2 },
{ time: 2, value: 4 },
{ time: 3, value: 5 },
],
2,
);
const baseline = evaluateBaseline(split, 'last');
const metrics = evaluateForecast([5], baseline.predictions);
const leakage = diagnoseLeakage([{ name: 'price', sourceTime: 2, availableAt: 3 }], 2);
if (metrics.mae !== 1 || leakage[0].reason !== 'late-availability')
throw new Error('Unexpected evaluation');
console.log(baseline, leakage);CommonJS supports the same runtime functions through require('@lp-labs/forecast-evaluator').
ESM and CommonJS include matching declarations.
API and types
TimePoint:{ time: number; value: number }, both finite. Time units/epochs and value units are caller-defined, consistent numeric scales.splitTimeSeries(input: Iterable<TimePoint>, trainCount, maxPoints = 100000): TimeSplit. Requires strictly increasing times, rejecting ties/reversals rather than silently sorting.trainCountis an integer strictly between zero and total length.TimeSplitcontains isolatedtrainandtestarrays; neither partition is empty.evaluateForecast(actual, predicted, maxPoints = 100000): ForecastMetrics. Equal nonempty finite numeric arrays, positionally paired; no implicit alignment.ForecastMetrics:count,mae(mean absolute error),rmse(root mean squared error),bias(mean predicted minus actual),mape(mean absolute percentage error multiplied by 100),percentageCount. Zero actuals are excluded from MAPE;mapeis null if every actual is zero. Negative actuals use absolute denominators.evaluateBaseline(split: TimeSplit, kind: 'last' | 'mean' = 'last', maxPoints = 100000): BaselineReport. Revalidates strict ordering across the training/test boundary. Predicts every test value using the last training value or training arithmetic mean only.BaselineReportcontainspredictions: number[]andmetrics: ForecastMetrics. Test outcomes do not influence predictions; there is no fitted model or refitting.FeatureTiming:{ name: string; sourceTime: number; availableAt: number }.diagnoseLeakage(features: Iterable<FeatureTiming>, forecastOrigin, maxFeatures = 10000): LeakageIssue[]. For each unique feature emitsfuture-sourceif sourceTime > origin andlate-availabilityif availableAt > origin, in input order, source first. Both issues can occur. Equality is permitted.LeakageIssuehasnameandreason. Metadata is supplied by you; the function does not inspect datasets, features, target joins, preprocessing, overlap windows, or causal relationships.
All listed interfaces are exported as types; the four functions are runtime exports.
Errors, customization and limits
TypeError: nonfinite/nonnumeric values, nonarray metric or partition inputs,
unknown baseline kinds, or duplicate/empty/over-128-unit feature names.
RangeError: invalid split, nonincreasing timestamps, unmatched/empty metric arrays,
capacity exhaustion, invalid limits, or intermediate/final arithmetic overflow.
All limits are positive safe integers <=100,000; iterables stop at limit + 1
consumed items. Arithmetic is IEEE-754; squared error overflow is rejected even
if a differently scaled implementation could represent the final RMSE.
Functions have no retained state or external mutation; errors return no partial report. External iterable/getter exceptions propagate. Inputs should be plain trusted data and may not be concurrently mutated during evaluation. O(N) time/storage for splits/baselines/diagnosis; paired metrics use O(N) time and O(1) working storage. Feature diagnosis produces at most two issues per feature.
Customize count-based holdout sizes, baseline kind, limits, timing origins, and units. Use multiple ordered holdouts yourself for rolling evaluation. MAPE is unstable near zero; baseline performance does not imply generalization, calibrated uncertainty, statistical significance, causal inference, or future success.
Related design context: virtual-time's ordered time, performance-budgets' honest metrics, incremental-aggregation's bounded numeric checks, unit-system's explicit units. No imports or dependencies on those packages.
Development
npm run build, npm run typecheck, npm test, npm run format, npm run format:check.
Dev-only TypeScript ^5.9.3, Prettier ^3.6.2; strict ESM/CommonJS declaration builds.
Tests include seeded arithmetic oracles, time-boundary leakage races, baseline
independence from test outcomes, overflow and generator bounds.
Support
If this library is useful to you, consider supporting its maintenance with a donation via PayPal.
Author
Created by Libin Prasanth. Connect on LinkedIn for updates and discussion.
