llm-date-guard

An LLM will tell you, with complete confidence, that there are 2 years between 12/03/2024 and 23/09/2024. There are 6 months and 11 days. It will not hedge, and it will not mark the figure as uncertain.

This is a small, dependency-free guard that makes date arithmetic deterministic again:

1. The model stops computing. A prompt rule makes it emit {{duree:12/03/2024|23/09/2024}} instead of a number, and the runtime substitutes the exact duration.
2. A safety net catches what slips through. When the model states a duration in plain text anyway, the guard recomputes it and rewrites it — but only under strict conditions, because a wrong "correction" is worse than the original error.
3. It works inside a token stream. A wrong duration streamed token by token is already on screen by the time you could check it. The guard holds back only risky segments, so the rest of the answer still streams at full speed.

The text being guarded is French. The API and docs are English.

Install

npm install llm-date-guard

Node ≥ 20, ESM only, no runtime dependencies.

Quickstart

import { applyDurationGuard, DATE_ARITHMETIC_RULE_FR } from "llm-date-guard";

const completion = await model.chat({
  system: `${yourSystemPrompt}\n\n${DATE_ARITHMETIC_RULE_FR}`,
  messages,
});

const { text, corrections, markers } = applyDurationGuard(completion.text);

// text        → what you display and persist
// markers     → { resolved: 3, dropped: 0 }
// corrections → durations the model got wrong, for your metrics

corrections is the interesting part in practice. The guard fixes things silently — which is the right product behaviour, and also the thing that would make you blind to how often the model is wrong. Log it.

The two layers

Layer 1 — markers

DATE_ARITHMETIC_RULE_FR (or _EN) forbids mental arithmetic and requires a marker. resolveDurationMarkers replaces it:

resolveDurationMarkers("Le premier courrier précède le second de {{duree:12/03/2024|23/09/2024}}.");
// → "Le premier courrier précède le second de 6 mois et 11 jours."

Separators |, → and -> are accepted; a bare - is not, since it is already a date separator. Both duree and duration work as keywords.

A marker whose bounds cannot be parsed degrades to du <a> au <b> rather than disappearing: the information is not hidden, only the duration is refused. stripLeftoverMarkers then guarantees that no {{…}} — including a malformed one the model invented — ever reaches the reader.

Layer 2 — the safety net

Models comply with the prompt most of the time, then write « soit environ 2 ans » anyway. correctDurationClaims recomputes such a claim, but only when all four conditions hold:

| Condition | Why | | --- | --- | | The segment (sentence, list item, line) holds at least two dates | A duration is never paired with dates from another sentence — the link would be speculative | | A trigger word presents the duration as that gap (délai, écart, soit, écoulés, séparent, …) | Without it, the number may be anything | | An interval connector explicitly ties two dates (du … au …, entre … et …, de … à …, or two dates in one parenthesis) — and the bounds are read from the connector | See below | | The discrepancy is material: > 25% and > 20 days | A good-faith rounding is not an error worth rewriting |

Miss one and the text is returned untouched. That is the whole design: the guard would rather leave a wrong duration in place than corrupt a right one.

So these are all left alone, on purpose:

La salariée avait 3 ans d'ancienneté lors de l'avertissement du 12/03/2024, et a été licenciée le 23/09/2024.
Le délai de prescription est de 5 ans ; les faits datent du 12/03/2024 et la saisine du 23/09/2024.
Du 12/03/2024 au 23/09/2024, soit environ 7 mois.

Seniority is not the gap between the two dates. A statutory deadline is not either. And « environ 7 mois » is a fair rounding of 6 months and 11 days.

Streaming

import { guardTokenStream } from "llm-date-guard";

for await (const delta of guardTokenStream(result.textStream, {
  onStats: ({ corrections }) => metrics.record(corrections),
  onComplete: (fullText) => db.save(fullText),
})) {
  send(delta);
}

guardTokenStream takes any AsyncIterable<string> and yields sanitised text. Output deltas do not line up with input tokens — risky text is re-chunked — so concatenate, never index.

What counts as risky: a date, a number followed by a time unit, an open {{, or a buffer that ends on something still becoming one of those (12/03/2, 6 mo, {{dur). That last test is what closes the real hole: without it, a token splitting a date in two would let it out before any check. Everything else flows through untouched, so on an answer with no dates the stream is byte-identical to the original.

Two escape hatches:

• withDurationGuard(stream, { format }) wraps an SSE byte stream directly, relaying non-token events (sources, metadata, [DONE]) untouched and flushing held text before them so ordering survives. Two framings ship ({"token":"…"} and {"type":"token","content":"…"}); anything else is a two-function object. It cannot preserve per-chunk metadata of an OpenAI-style payload, because one output delta may span several input events — use guardTokenStream there.
• DurationGuardBuffer is the state machine on its own: push(token) returns what is safe to emit, flush() releases the rest. Drive it from any transport. Always call flush(), including on the error path, or held text is text nobody reads.

Limits

• French only. Month names, number words, connectors and trigger words are French. The output wording is French too ("6 mois et 11 jours"). Adding a locale means replacing the lexicons in dates.ts and claims.ts — the arithmetic in duration.ts is language-agnostic already.
• Layer 2 is a heuristic, and a deliberately timid one. It will miss wrong durations rather than risk rewriting right ones. Layer 1 is where the reliability actually comes from; treat the net as a net.
• Two-digit years are ignored (12/03/24). Too ambiguous to rewrite text on.
• Durations past one year stop at months ("2 ans et 3 mois"). Residual days at that scale are noise, and printing them would imply precision the wording does not have.
• No timezone handling. Dates are parsed as UTC midnight, so a duration never depends on where the code runs. If you need calendar-day semantics in a specific zone, convert before calling.
• Streaming adds latency on risky sentences only — a sentence holding a date waits for its closing punctuation, bounded by a 2000-character ceiling so a model writing without punctuation cannot freeze the display.

API

| Export | Purpose | | --- | --- | | applyDurationGuard(text) | Both layers plus cleanup. Returns { text, markers, corrections }. Idempotent. | | guardDurations(text) | Same, text only. | | resolveDurationMarkers(text) | Layer 1 alone. | | stripLeftoverMarkers(text) | Neutralise leftover {{…}}. | | correctDurationClaims(text) | Layer 2 alone. | | guardTokenStream(source, opts?) | Guard an AsyncIterable<string>. | | withDurationGuard(stream, opts?) | Guard an SSE ReadableStream<Uint8Array>. | | DurationGuardBuffer | The buffering state machine, for custom transports. | | calendarDuration(a, b) | { years, months, days, totalDays }, order-insensitive. | | formatDurationBetween(a, b) | French wording of the gap. | | parseFrenchDate(raw) | Date or null — never a guess. | | findDates(text) | Every date with its offsets. | | DATE_ARITHMETIC_RULE_FR / _EN | The prompt half of the contract. |

Development

npm install
npm test
npm run typecheck
npm run build

50 tests, no network, no fixtures. The observed failure case (12/03/2024 → 23/09/2024 claimed as "2 ans") is a test.

Provenance

Extracted from the AI layer of a production SaaS where a wrong duration is a professional liability, not a cosmetic bug. Two defects surfaced during extraction and are fixed here:

• Interval bounds were taken as the min/max of every date in the segment. With three dates — « Du 12/03/2024 au 23/09/2024, soit 2 ans, alors que l'accord du 01/01/2020 … » — that spanned 2020→2024 and replaced a wrong duration with another wrong one. Bounds now come from the connector that was actually matched.
• Compound durations were compared piecewise. Given an already-correct « soit 6 mois et 11 jours », the trailing « 11 jours » looked like a wildly wrong claim on its own and got rewritten, producing « 6 mois et 6 mois et 11 jours » — a visible corruption of a correct answer. A claim is now matched and summed as a whole.

Both were found by the same test: the guard must be idempotent.

License

MIT