regex-railroad

A modern-syntax JavaScript regex parser core — parse(pattern, flags) returns a normalized AST — built as the pure foundation for a future railroad-diagram visualizer + plain-English explainer.

Status: staged incubator project. NOT domain-wired, NOT published, NOT deployed. The pure cores have landed: parse/parseFlags (AST), renderSvg (SVG railroad), explain/describe (plain-English), findMatches (ReDoS-guarded match tester), and substituteAll (find-and-replace preview), plus a client-side SVG→PNG export in the bundled demo page. Going live to a domain or npm is a human-gated proposal — see GO-LIVE.md.

Install

npm install @verifyhash/regex-railroad

The @verifyhash/regex-railroad scope/name shown here is a placeholder for publish-readiness — the final published scope and name are the owner's call and may change before any actual publish. Nothing here is published yet.

// One require, the whole landed API:
const { parse, parseFlags, renderSvg, explain, describe, findMatches, substituteAll } =
  require('@verifyhash/regex-railroad'); // main => index.js

const ast = parse('a(?<year>\\d{4})-\\d{2}', '');
console.log(ast.names);                  // ['year']
console.log(explain(ast).text.slice(0, 12)); // "The text 'a'"
console.log(renderSvg(ast).slice(0, 5)); // "<svg "
console.log(findMatches('\\d+', 'g', 'x12 y7').matches.length); // 2

What it is

Paste a regex, and the eventual tool draws a railroad/"train-track" diagram of what it matches, explains it clause-by-clause in plain English, and lets you export an SVG/PNG or a shareable link. The diagram is the product — the thing you drop into a pull-request review, a Notion runbook, or a Confluence page to explain a gnarly pattern to teammates.

This repository currently contains step one only: the regex → AST parser that everything else will render from. Getting the parse right, and pinning it with hand-verified golden vectors, before drawing a single line, is deliberate.

Why this exists

The canonical incumbent, Regexper (source), was archived on 2018-06-06 and is read-only. It predates the ES2018 regex era, so it largely cannot render the syntax modern JavaScript engines added:

| ES2018+ construct | Example | Regexper (2018) | regex-railroad core | | --------------------------- | -------------- | --------------- | ------------------- | | Named capture groups | (?<year>\d+) | no | yes | | Lookbehind | (?<=\$)\d+ | no | yes | | Negative lookbehind | (?<!foo)bar | no | yes | | dotAll s flag | /a.b/s | no | yes (recorded) | | Unicode property escapes | \p{L}+ | no | yes |

(regex101 debugs and matches, but draws no diagram — so the "picture you can share" niche is genuinely open.)

Who it's for

Developers and power users who read and review regular expressions: anyone who has ever pasted a 60-character pattern into a code review and typed "…I think this is right?".

How to run the tests

npm test

That runs the golden-vector suite (test/golden.test.js) with plain Node — no install step, zero runtime dependencies. Requires Node (developed on v23; anything with modern String.prototype.codePointAt works). The suite exits non-zero on any failure.

Each valid vector is additionally checked against the native new RegExp(...) engine, so a golden AST can never silently drift away from real, accepted JavaScript syntax.

Using the core

const { parse } = require('@verifyhash/regex-railroad'); // main => index.js

const ast = parse('a(?<year>\\d{4})-\\d{2}');
// ast.type === 'Root'
// ast.names === ['year'], ast.groupCount === 1
// ast.body is a Sequence of: Literal 'a', named Group, Literal '-', Quantifier{2,2}

parse('a(');        // throws: "Invalid regular expression ... Unterminated group"
parse('\\p{L}+', 'u').flags.unicode; // true
parse('a.b', 's').flags.dotAll;      // true

parse is pure: no DOM, no network, no filesystem, no global mutation. It reads only its two arguments and returns a plain-object AST (or throws an Error).

The match tester (match.js)

The diagram tells you what a pattern means; the match tester tells you what it does to real text — the regex101-style companion Regexper never had. It is a second pure core:

const { findMatches } = require('./match.js');

findMatches('(?<y>\\d{4})-(\\d{2})', '', '2026-07 and 1999-01');
// {
//   matches: [
//     { index: 0,  length: 7, text: '2026-07',
//       groups: [ { name:'y', index:0, length:4, text:'2026' },
//                 { name:null, index:5, length:2, text:'07' } ] },
//     { index: 12, length: 7, text: '1999-01', groups: [ ... ] }
//   ],
//   truncated: false
// }

What it guarantees

• Builds a native RegExp from (pattern, flags), always forcing the global flag internally so it walks the whole subject via a lastIndex loop over exec(), and the indices (d) flag so every group reports its own index/length.
• Returns matches in document order, each with a groups array covering every numbered and named capture — a non-participating optional group surfaces as { name, index:null, length:null, text:null }.
• Zero-width safety: an empty match (e.g. /a*/g or /(?:)/g against non-empty text) cannot spin forever — lastIndex is manually advanced by one code point (a full surrogate pair under the u/v flag) after every empty hit.
• Honest errors: if new RegExp(pattern, flags) throws (bad syntax / unsupported flag), findMatches returns { error: <message> } (it does not throw), so a UI can render the message on the same code path as a normal result.

Work bounds — and the ReDoS caveat you must respect

findMatches(pattern, flags, subject, opts) accepts opts = { maxSubjectLength = 100000, maxMatches = 10000 }. If either cap is hit the walk stops and the result carries truncated: true (the subject is clipped to maxSubjectLength before matching). These bound the count and length of work only.

A synchronous native regex cannot be interrupted from within its own thread, so the core cannot rescue you from catastrophic backtracking — a pattern like /(a+)+$/ against "aaaaaaaaaaaaaaaaaaaaX" can hang for seconds-to-forever inside a single exec() call, before any count/length cap is ever consulted. Therefore any UI MUST run findMatches inside a Web Worker and impose a hard wall-clock timeout, terminating the worker if it overruns. Do not call it on the main thread with untrusted patterns. (The bundled index.html/app.js demo runs it inline for trusted, hand-typed input; a shipped product would move it to a worker as described here.)

Run its golden vectors with the same one command as everything else — npm test runs test/match.test.js alongside the parser/render/explain suites.

The substitution / replace preview (substitute.js)

A third pure core, substituteAll(pattern, flags, testString, replacement), answers the other everyday regex question: what does my find-and-replace actually produce? It returns the fully replaced string and how many replacements it made:

const { substituteAll } = require('./substitute.js');

substituteAll('(?<y>\\d{4})-(\\d{2})-(\\d{2})', '', '2026-07-21', '$3/$2/$<y>');
// { output: '21/07/2026', count: 1, truncated: false }

substituteAll('a', 'g', 'banana', 'X');   // { output: 'bXnXnX', count: 3, ... }
substituteAll('z', 'g', 'banana', 'X');   // { output: 'banana',  count: 0, ... }  (identity)

It expands the standard ECMAScript replacement grammar honestly and explicitly — these tokens, and only these, are interpreted; anything else is copied literally:

| Token | Expands to | | ----------- | ----------------------------------------------------------------------- | | $$ | a literal $ | | $& | the whole matched substring | | $` | the subject text before the match | | $' | the subject text after the match | | $1…$n | the n-th numbered group (two digits tried first). A non-participating group → ''; a number past the group count is left literal. | | $<name> | the named group name. Non-participating/unknown name → ''; if the regex declares no named group, $<…> is left literal (exactly as JS does). |

Rather than delegate to String.prototype.replace (which hides the count and the grammar), it drives the match walk with the same forced-global + zero-width-advance helpers that match.js uses (imported from it — not a second copy), so a zero-width pattern such as /(?=x)/g can never infinite-loop, and the same { maxSubjectLength, maxMatches } caps and ReDoS caveat apply. On an invalid pattern/flag it returns { error } (never throws), like findMatches. The golden vectors in test/substitute.test.js cross-check every output against native String.prototype.replace.

AST shape

parse(pattern, flags) returns a Root node:

Root         { type, source, flags, groupCount, names[], body }

flags is { hasIndices, global, ignoreCase, multiline, dotAll, unicode, unicodeSets, sticky, raw }. body is one node from this set:

| Node | Fields | | ------------------ | ------------------------------------------------------------- | | Sequence | elements[] (concatenation, may be empty) | | Alternation | alternatives[] (the branches of a|b|c) | | Literal | value (a single ordinary character) | | Char | value, codePoint, escaped?, raw? (an escaped char) | | AnyChar | — (the . metachar) | | Quantifier | min, max (null = unbounded), greedy, body | | Group | capturing, name (or null), number (or null), body | | Lookaround | direction (lookahead/lookbehind), negative, body | | Anchor | kind (start/end/wordBoundary/nonWordBoundary) | | CharacterClass | negated, items[] | | Range | from, to, fromCodePoint, toCodePoint (a class range) | | CharClassEscape | kind (digit/word/whitespace), negated, raw | | UnicodeProperty | negated, property (or null), value, raw | | Backreference | name or number |

Container nodes (Sequence, Alternation, Quantifier, Group, Lookaround, CharacterClass) nest other nodes; the rest are leaves.

Coverage & honest limits

Covered: literals & escapes (\n \t \xHH \uHHHH \u{...} and escaped metachars), character classes with ranges and negation, the shorthand classes \d \w \s (and negated \D \W \S), quantifiers * + ? {n} {n,} {n,m} and their lazy ? variants, groups (capturing, non-capturing (?:…), and named (?<name>…)), alternation |, anchors ^ $ \b \B, lookahead (?=…)/(?!…), lookbehind (?<=…)/(?<!…), the recorded dotAll s flag, Unicode property escapes \p{…}/\P{…}, and back-references (\1, \k<name>).

Not yet: this parser records the syntax tree; it does not itself validate every semantic edge (e.g. it does not confirm a \k<name> back-reference resolves to a declared group, and it treats a Unicode property name as an opaque string rather than checking it against the Unicode database). It targets JavaScript regex specifically — not PCRE recursion, atomic groups, or possessive quantifiers, which JS does not support. Malformed input (unbalanced group/class, reversed range, dangling quantifier) throws an honest Error rather than guessing.

Roadmap (each item is a FUTURE, human-gated incubator task)

Graduation to a live domain, an npm package, or any publish is owner-only — proposed via state/NEEDS-HUMAN.md, never self-actioned. Planned follow-on tasks, in order:

1. SVG railroad renderer — walk this AST into a deterministic, standalone SVG train-track diagram (the shareable artifact).
2. Plain-English explainer — a clause-by-clause natural-language description generated from the same AST.
3. Export & share — SVG/PNG download plus a canonical shareable URL that round-trips the pattern (fully client-side, no backend).
4. Standalone-site candidate — assemble the above into a single self-contained page; propose graduation to the owner (a new domain is human-gated).

API

The package entry (index.js) re-exports seven pure functions. Full TypeScript signatures live in index.d.ts.

| Function | Signature | Returns | | --- | --- | --- | | parse | parse(pattern: string, flags?: string) | A Root AST node { type:'Root', source, flags, groupCount, names[], body }. Throws on invalid pattern/flags. | | parseFlags | parseFlags(flags?: string) | A flags object { hasIndices, global, ignoreCase, multiline, dotAll, unicode, unicodeSets, sticky, raw }. Throws on an unknown/duplicate flag or the mutually-exclusive u+v. | | renderSvg | renderSvg(ast, opts?: { palette }) | An inline SVG string (no xmlns, no external URL). Accepts a Root node or a bare body node; opts.palette overrides { stroke, accent, node, group, text }. | | explain | explain(ast) | { clauses: [{ node, text }], text } — one clause per top-level element plus one per active flag, and a single joined sentence. | | describe | describe(node) | A short English string for a single AST node (e.g. 'the text a'); 'an unrecognized construct' for anything unknown. | | findMatches | findMatches(pattern, flags, subject, opts?: { maxSubjectLength=100000, maxMatches=10000 }) | On success { matches: [{ index, length, text, groups:[{ name, index, length, text }] }], truncated }; on a bad pattern/flag { error } (no matches key — it returns, never throws). See the ReDoS caveat above before running on untrusted input. | | substituteAll | substituteAll(pattern, flags, testString, replacement, opts?: { maxSubjectLength=100000, maxMatches=10000 }) | On success { output, count, truncated } — output is the subject with every match replaced ($1…$n, $<name>, $&, $`, $', $$), count the number of replacements; on a bad pattern/flag { error } (no output/count key — returns, never throws). Same forced-g, zero-width-advance and ReDoS caveat as findMatches. |

PNG export (browser-only, in the demo page)

PNG export is not part of the require()-able module API — it is a client-side feature of the bundled index.html/app.js demo (exportPng() in app.js). It rasterizes the same SVG that renderSvg produces: the self-contained SVG document is loaded into an <img> via a data: URL (no network, so the canvas is never tainted), painted onto an offscreen <canvas> at 2× scale over a solid theme-colour background (a PNG has no page CSS behind it), then downloaded as regex-railroad.png via canvas.toBlob(...). Because it depends on the DOM (Image, <canvas>), it runs only in a browser, not under Node — hence it lives in app.js rather than in the module exports above. The upstream SVG string it rasterizes is available headlessly through renderSvg.

License

MIT. Zero runtime dependencies — nothing is vendored, so there is no third-party license to record.