@leo-renard/lien

v1.0.0-20260713020823

Published

3 days ago

Block-level translation/reconcile engine and content-repo CLI for renard.

0High
0Medium
0Low

lien

@leo-renard/lien — the block-level translation reconcile engine. Given an edit in one locale of a multilingual document, it produces the proposed update for every sibling locale without silently discarding a human-refined translation. Named for the synchronization it keeps between locales.

Why it exists

The site is bilingual today (en + zh-CN, more later) and you can edit any locale; the others must follow. The naive approach — re-translate the whole document on every save — destroys hand-refined translations: a human-polished Chinese paragraph gets clobbered by a fresh machine rendering just because someone fixed a typo in the English three paragraphs away.

lien exists to make exactly that not happen. Its load-bearing invariant:

A human-refined translation is never silently dropped or overwritten. When a change can't be applied safely, the engine surfaces a conflict for explicit review — nothing is written live.

How a conflict is surfaced is the consumer's call. In jardinier — the admin's only surface — it appears as an inline needs-confirm step the author resolves in place (re-submit with accept=true to commit to drafts). The admin never touches git, branches, or pull requests; everything happens in jardinier. The engine itself just returns a non-empty conflicts list for the caller to handle.

Everything below is in service of that one guarantee.

Who consumes it

| Consumer | When | What it uses | | ------------------------------------ | ---------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | jardinier (admin) | every save | reconcile, blocks, ai, manifest, types — reconciles siblings at write time; commits the edited source, any changed/reconciled sibling files, and the manifest in one atomic commit | | content repo CI (the lien bin) | publish / CI | the lien CLI (promote/promote-singleton/demote/finalize) — pure git file-copy publish + tag-label/manifest finalize. sync/baseline/backfill/translate modes ship too (migration + jardinier translate-now). |

Per the v2.1 design, translation reconcile happens at jardinier save time, not in a CI workflow. The promote/finalize tooling only mirrors drafts→main and finalizes (tag labels, refresh) what jardinier already reconciled.

Two consumption paths, deliberately different:

In the monorepo, jardinier imports the engine as workspace source (@leo-renard/lien/blocks → ./src/blocks.ts), so there is no build step in the dev/test/typecheck loop.
The content repo consumes only the CLI, via pnpm dlx @leo-renard/lien@latest <subcommand>. So the published npm package is a single self-contained CLI bundle — dist/cli.js (CommonJS) with every dependency inlined by tsup/esbuild, plus a generated dist/package.json (bin: { lien }). It ships no library subpaths and no runtime dependencies; bundling also freezes in the workspace-pinned, patched js-yaml 4.x, so the published CLI never resolves a vulnerable transitive dependency.

CI publishes a new 1.0.0-<UTC commit time> on every push to main that touches lien/, via OIDC trusted publishing (no stored npm token).

The model: blocks → align → reconcile

A document is markdown with frontmatter. The body is split into blocks — top-level elements (paragraphs, headings, code fences, lists, tables, quotes) separated by blank lines. Unknown/unsupported nodes become opaque raw blocks so round-tripping never loses content.

When one locale is edited, reconcile runs in three stages:

1. blocks — decompose, byte-stably

parseDoc splits the exact frontmatter from the body and parses the body into blocks, recording each block's exact source span (string offsets into the source). Reassembly never re-serializes markdown (that isn't byte-stable) — it slices the original, so untouched regions, including inter-block whitespace, survive byte-for-byte.

Two hashes serve two different jobs:

alignHash — line-ending-insensitive (normalizes CRLF→LF). Used for diff/alignment, so a re-save that differs only in line endings doesn't read as a changed block. (It normalizes only line endings — it deliberately does not strip other whitespace, since trailing whitespace is semantic in markdown: two trailing spaces is a hard line break.)
blockHash — byte-exact (hashes the raw span). Used for provenance and the byte-stability check.

2. align — diff and classify

alignAndClassify diffs the edited locale's previous vs. new blocks with an LCS over alignHash, and pairs each new block to the sibling locale's corresponding block. Because the LCS anchors on unchanged content, inserting or reordering blocks shifts nothing — only genuinely changed blocks are flagged, never a positional mass-rewrite. Each new block is classified:

unchanged — identical (including a reordered/moved block, matched by content);
modified — an in-place edit between the same anchors;
added — new content with no prior match.

If the previous edited doc and the sibling don't share a 1:1 block structure — or a structural change is genuinely ambiguous (e.g. duplicate-identical blocks reordered, where it can't know which translation carries which) — align reports diverged: true and reconcile raises a conflict rather than guess.

3. reconcile — carry, judge, or flag

reconcileSibling walks the aligned blocks and decides each sibling block's fate:

unchanged → carry byte-for-byte. The existing sibling translation (human refinement and all) is copied verbatim. No AI call.
added → translate. Only this new block is sent to the AI.
modified → judge first, then decide. The AI is asked whether the current translation still conveys the same meaning as the new source (a looks-good / needs-change verdict with an entailment flag). Wording-only differences keep the human translation; a real meaning change re-translates. Guards make a keep conservative:
- if the block contains risky tokens (numbers, dates, negation, modals) → conflict, even on looks-good;
- a second independent judge must also agree the current translation is entailed;
- a meaning change on a hand-refined block → conflict (never a live overwrite).
(On the re-translate path, a needs-change verdict with low confidence on a machine-owned block also flags a conflict rather than applying a shaky rewrite — confidence gates the re-translate branch, not the keep branch.)

The output carries the proposed new sibling body (so a flagged conflict always has a real diff to review), per-block provenance (mtBlocks: machine hash, or null for a preserved human block), and a conflicts list. A non-empty conflicts means do not write live — surface it for review; in jardinier the admin resolves it inline (a needs-confirm step, not a pull request). An ordered byte-stability verifier confirms every "kept" block actually survived byte-for-byte in the reassembled output.

Provenance: how it knows what's human-refined

Each post has a manifest sidecar at content/.sync/<collection>/<slug>.json. Per locale it records the current block hashes plus mtBlocks — the hashes of the last machine-produced output. A sibling block is machine-owned only if its current hash still equals its recorded mtBlocks hash; any divergence (or an absent mtBlocks, as in a baseline migration) means a human touched it, so it is treated as hand-refined and never auto-clobbered.

A worked example

A two-block English post, already translated to Chinese (the Chinese second block was hand-polished):

en (before)            zh (current, block 2 refined)
─────────────          ──────────────────────────────
Welcome to my blog.    欢迎来到我的博客。
I write about Go.      我在这里分享 Go 语言的点滴。   ← human refinement

The author edits the English first block only:

en (after)
─────────────
Welcome to my new blog.   ← modified
I write about Go.         ← unchanged

Reconcile:

Block 1 (Welcome…) is modified → the AI judges; "new" is a wording change, not a meaning change, but if the existing zh still conveys it, looks-good keeps it; otherwise it re-translates just this block.
Block 2 (I write about Go.) is unchanged → the hand-polished 我在这里分享 Go 语言的点滴。 is carried byte-for-byte. It is never sent to the AI, never re-translated.

Edit only block 2's English to add a number ("I write about Go and 3 side projects.") and the risky-token guard raises a conflict instead of overwriting the refined Chinese live.

Public surface

Exports (see package.json for the exact map):

| Entry point | Purpose | | ---------------------- | -------------------------------------------------------------------------------------------------------------------------------- | | ./types | Block, ParsedDoc, Manifest, SyncConfig, DEFAULT_CONFIG, locale/collection types | | ./blocks | parseDoc / parseBody, assemble, blockHash / alignHash / docHash, blocksOf | | ./align | alignAndClassify — LCS diff + per-block classification | | ./reconcile | reconcileSibling — the keep-or-retranslate engine | | ./ai | the multi-provider CallTool seam: translateBlock, verdictForModifiedBlock, secondJudge, hasRiskyTokens, realCallTool | | ./manifest | manifest read/write + state builders (humanState, machineState, baselineManifest) | | ./frontmatter-policy | which frontmatter fields translate, per collection | | ./validate | validateSlugs — pipeline-side slug parity check | | ./tag-labels | localized tag-label types/helpers |

Purity and the injected AI seam

The reconcile core (blocks, align, reconcile, and ai's orchestration) does no I/O of its own. All model access goes through one injectable CallTool seam — a function (req) => Promise<object> that forces a single structured tool call. The live caller (realCallTool) is multi-provider:

primary = AI_PROVIDER (default anthropic), reading that provider's standard key env (ANTHROPIC_API_KEY / OPENAI_API_KEY / GEMINI_API_KEY / OPENROUTER_API_KEY);
an optional AI_FALLBACK_PROVIDER is tried when the primary fails;
all adapters are fetch-based (no SDK — smallest supply-chain surface);
all providers failing → TranslationUnavailable → the caller surfaces a conflict (no live write).

Because the seam is injectable, tests pass a fake call and need no API keys or network. (The manifest and validate entry points are filesystem helpers for the content-repo CLI — those touch node:fs; the reconcile core does not.)

Tests

pnpm --filter @leo-renard/lien test       # vitest, no API keys required

The suite covers the block decomposition, LCS alignment edge cases (insert / remove / reorder / duplicate-ambiguity), the keep-vs-retranslate verdict logic, the risky-token and second-judge guards, byte-stable reassembly, and the multi-provider fallback — all with injected fakes, no network.

Build & publish

pnpm --filter @leo-renard/lien build   # tsup bundles src/cli.ts → dist/cli.js (CJS, all deps inlined),
                                   # then scripts/make-publish-package.mjs writes dist/package.json

The workspace package.json stays private with exports → ./src/*.ts (jardinier consumes raw TS; vitest/tsc --noEmit untouched). The published artifact is the generated dist/ directory, not the workspace package: tsup (CommonJS, noExternal) emits a self-contained dist/cli.js, and make-publish-package.mjs writes a minimal dist/package.json (name, stamped version, bin, no dependencies). CI (.github/workflows/ci.yml, publish-lien job) stamps 1.0.0-<UTC commit time>, builds, and runs npm publish from lien/dist via OIDC trusted publishing on every push to main that changes lien/ — nothing is committed or tagged back. (CommonJS because the bundled gray-matter dependency does a dynamic require, which an ESM bundle can't satisfy.)