@venn-lang/core

The Venn compiler: grammar, parser, type checker, expression compiler, decorator expansion, and the Problem model.

Everything that turns .vn text into something another package can run, check or draw lives here. The kernel is deliberately small: it knows flows, steps, expressions, types and decorators, and it knows nothing about HTTP, browsers or files. Protocols arrive as plugins built on @venn-lang/sdk; the core only ever sees the tree they decorate.

@venn-lang/core never imports node:*. It is built with platform: "neutral" so the same bundle runs in Node, in the CLI, and inside the Web Worker that hosts the language server. Its runtime dependencies are Langium (parsing) and @venn-lang/types (the published type vocabulary).

Usage

import { checkTypes, parse } from "@venn-lang/core";

const uri = "file:///hello.vn";
const source = `flow "Hello" {
  step "Ping" {
    http.get "https://example.com/health"
    expect res.status == 200
  }
}`;

const { ast, problems } = parse(source, { uri });
const checked = checkTypes(ast, { uri });

for (const problem of [...problems, ...checked.problems]) {
  console.log(`${problem.code} ${problem.title} (${problem.span.line}:${problem.span.column})`);
}

parse is synchronous and touches no filesystem. Error recovery keeps a partial AST, so a file with a syntax error still produces a tree the editor can work with.

Module map

| Module | Responsibility | | --- | --- | | grammar/ | venn.langium, the whole grammar of the language. The single source the parser is generated from. | | generated/ | Langium output: the AST interfaces, the isX type guards, the grammar object and the services module. Regenerated by pnpm --filter @venn-lang/core langium:generate. | | lang/ | The Langium services parse uses, plus VennLexer, which makes newlines significant between statements and suppresses them inside ( ) and [ ]. | | parse/ | parse and parseExpression. Turns text into a Document plus VN1xxx problems. | | ast/ | Small readers over the generated tree: walkAst, dottedPath, callArgs, splitCall, isRunnable. | | problem/ | The Problem shape shared by compile diagnostics and runtime failures, with Span, Diff, Severity, ProblemError and buildDiff. | | codes/ | CODES, the catalog of every VNxxxx the kernel can raise, and buildProblem. | | expand/ | The expansion phase. Runs every @decorator over the tree once, before anything else reads it. | | typecheck/ | Hindley-Milner inference with generics, the Type algebra, the plugin TypeCatalog, cross-file importedTypes, and the prose the editor shows on hover. | | compile/ | Compiles each Expr into a Thunk, a closure over the environment. Memoised per node, so the tree is walked once and not once per evaluation. | | expr/ | The evaluator surface: evaluate, closures and frames, invoke, memberValue, the built-in method tables, and the prelude values. | | value/ | What a runtime value is, and the three questions asked of one: truthy, strictEquals, isNumeric. | | units/ | Unit-typed values (300ms, 2mb, 50%, ISO instants) and the arithmetic that keeps units honest. | | interpolation/ | ${…} placeholders, described once for both the evaluator and the editor. | | format/ | formatText, shared by venn fmt and the language server so both agree on the result. | | events/ | The event envelope the runner emits and the UI consumes. Types only, no runtime. | | plan/ | stepTitlesOf: every step title a flow can reach, so a reporter can draw the tree before anything runs. | | module/ | Which of the three kinds an import specifier is: relative, alias or package. |

API

Parsing

| Export | What it gives you | | --- | --- | | parse(text, { uri? }) | ParseOutput: the Document AST and the VN1xxx problems. | | parseProblems({ result, uri, text }) | The same VN1xxx problems for a parse somebody else ran, so the language server publishes what venn check prints instead of Chevrotain's own words. | | parseExpression(source) | An Expr for a standalone expression, or undefined when it is not one. | | EXPRESSION_OFFSET | How far parseExpression shifts CST offsets, so a caller can map back onto the ${…} it came from. | | fileOf(node) | Which file a node was parsed from, recorded by parse. For code that has to place a Problem with no uri to thread: the compiler builds a fn body once, away from the document that asked. Empty for a tree that came from anywhere else. | | vennServices() / createVennServices() | The Langium services, cached and fresh respectively. | | VennLexer | The lexer that suppresses newlines inside brackets. | | VennGeneratedModule, VennGeneratedSharedModule, VennLanguageMetaData | The generated Langium modules, for a host that builds its own services. |

Everything from generated/ast.ts is re-exported: Document, FlowDecl, StepDecl, ActionCall, ExpectStmt, FnDecl, DecoDecl, Expr and the rest, each with its isX guard.

The AST readers exist because the grammar spells some things two ways:

| Export | What it answers | | --- | --- | | callArgs(call) | The arguments of a call as one list, whether it was written http.get "url" or conn.close(). | | splitCall(args, takes) | Which trailing { … } is an options map and which is a real argument. | | dottedPath(expr) | The dotted name an expression spells (http.get), or undefined when it is not a plain chain. | | isRunnable(decl) | Whether a top-level node runs or merely defines. | | walkAst(root) | Every descendant, depth first. | | insideAnnotation(node) | Whether a node is written inside a @name(…), where a bare name is a word and not a reference. |

Problems and codes

Every failure, at compile time or at run time, is a Problem: a stable code, a severity, a one-line title in the user's vocabulary, a span, and optional help, related, diff, note and docs. One shape means one renderer serves the terminal, the editor and the UI.

import { buildProblem, CODES, ProblemError } from "@venn-lang/core";

throw new ProblemError(
  buildProblem({
    spec: CODES.VN6001_ASSERTION_FAILED,
    span: { uri, offset: 0, length: 0, line: 1, column: 1 },
    title: "The response did not arrive before the deadline",
  }),
);

| Family | Covers | | --- | --- | | VN1xxx | Lexical and syntactic. | | VN2xxx | Name resolution, imports, decorators, capability negotiation. | | VN3xxx | Types and units. | | VN4xxx | Concurrency and isolation. | | VN5xxx | Lint. | | VN6xxx | Assertions, at run time. | | VN7xxx | Actions and protocols, at run time. | | VN8xxx | Timeouts and resource limits. |

CODES holds the codes the kernel itself raises, from VN1001_LEX to VN7004_HOOK_FAILED. The VN8xxx family is numbered above but raised elsewhere: RUN_CODES.VN8002_STILL_RUNNING is the runner's, in @venn-lang/runtime, and grepping the number here finds nothing. Plugins add their own in the same families. buildDiff turns two compared values into a structured Diff, walking them field by field when they line up, so a failure names the field that moved instead of printing two renderings side by side. formatValue renders a single value for a report. UNLOCATED is the one span for a failure nobody could place: an empty uri, which every reader takes as "print no location" rather than as the file's first line. spanOf(node, uri) places a node, ${…} included, and slotSpan({ slot, host, uri }) places the placeholder itself. orPhrase(items) says a list the way a title has to: "a", "b" or "c", no serial comma, no trailing stop.

Type checking

import { checkTypes, showType } from "@venn-lang/core";

const { problems, types, slots } = checkTypes(ast, { uri, catalog, decos, imports });

checkTypes runs Hindley-Milner inference over the whole document. Functions are generalised, so one fn id(x) => x serves every type; dynamic unifies with everything and never errors, so a plugin result or a parsed HTTP response places no annotation burden on the pure parts of a file. It returns the problems, every expression's inferred type keyed by node (for hover), and the expressions parsed out of each string literal's ${…} slots.

CheckTypesOptions is how the checker learns about the world outside the file: catalog (what the loaded plugins publish), decos (the pub decos the imports reach) and imports (what each imported name turned out to be).

| Export | Purpose | | --- | --- | | Type, PrimType, ExactType, ListType, RecordType, FnType, UnionType, OpaqueType | The type algebra. ExactType is the type of exactly one value, as "GET" is where a type is written. | | DYNAMIC, literal, opaque, union | Constructors for the ones a caller builds by hand. | | prune | Follow a type variable to what it has been solved to. | | showType, showTypes | Render types for a hover or a diagnostic. showTypes names variables across a group, so two unrelated parameters are not both called a. | | TypeCatalog | The two questions the core asks about plugins: typeOf(name) and signatureOf(target). | | specToType, ResolveRef | Read a published TypeSpec from @venn-lang/types into the checker's own type. | | importedTypes, ImportedTypes | The types of the names a document imports, worked out from the files it names. Cycles answer with nothing rather than looping. | | memberType, resolveMember | The type of xs.map, s.length, p.name. | | createContext | Fresh per-check inference state, so two checks never share variable ids. | | BUILTIN_TYPES, isBuiltinType | string, number, bool, duration, size, percent, instant and friends, each with a line of documentation and an example. | | PRELUDE_SPECS, isPrelude, PreludeSpec, PreludeArg | The prelude, described once: types for the checker, prose for the editor. | | MEMBER_DOCS, MemberDoc, memberKind | What each built-in member does, for completion and hover. | | KIND_SPECS, KIND_TYPES | What a decorator target handle offers, published as types. |

Compiling and evaluating expressions

An expression is compiled once into a Thunk, a function of the environment. Everything the source settles (which operator, which literal, which slot a name lives in) is decided at compile time.

import { evaluate, type EvalEnv } from "@venn-lang/core";

const env: EvalEnv = { lookup: (name) => (name === "res" ? { status: 200 } : undefined) };
evaluate(subject, env); // true, for `res.status == 200`

| Export | Purpose | | --- | --- | | compileExpr(expr) | The compiled Thunk. Memoised per node. | | closureOfDecl(decl, env) | A top-level fn as a callable value bound into an environment. | | evaluate(expr, env) | Compile if needed, then run. The seam every caller uses. | | EvalEnv | One method, lookup(name). Actions and matchers are not here: they belong to the runtime registry. | | childEnv(parent, bindings) | A nested scope. | | Cell, CellEnv, hasCells | Bindings addressed by cell, which is what lets a recursive fn capture itself. | | invoke, invoke1, callClosure, isCallable | Calling a Venn callable, with fixed arities for the hot paths. | | Closure, isClosure, NativeFn, nativeFn, isNativeFn | The two kinds of callable. | | memberValue(receiver, member) | What .x means on any value: a map's own data first, then the built-in member tables. | | indexValue(receiver, at) | What x[k] means, which is the same reading with the key worked out first. | | MEMBER_NAMES | Every member name each kind answers to, taken from the tables that answer them. | | namespaceValue, isNamespaceValue | A plugin namespace as a value, so fmt.table(rows) works inside any expression. | | PRELUDE_VALUES | range, str, typeOf, pretty, spawn. | | display(value), typeName(value) | How print renders a value, and what the language calls its type. |

value/ owns what a value is. kindOf(value) answers with one of the fourteen ValueKinds the language has, and everything that dispatches on the shape of a value asks it: the member read, the method tables, typeOf, and how a value is written into text. Value is the shape a runtime value can take, with truthy, strictEquals and isNumeric alongside it. There is no coercion: "99.00" never equals 99. isReservedKey and reservedKeyProblem are the one refusal both halves of assignment give for a key that would reach past the value into what made it.

Decorator expansion

expand is the phase between parsing and everything else. It is the only place a @name means anything. The kernel does not know what @retry is; it knows that a name written with an @ is looked up in a DecoratorSource, handed the node it sits on, and allowed to rewrite it. Decorators written in TypeScript by a plugin and decorators written in Venn with deco go through the same door.

import { expand } from "@venn-lang/core";

const { problems } = expand({ document: ast, decorators, uri, imported });

Applied innermost first, so a decorator that rewrites a body finds a body its own decorators have already finished with. An ExpandContext gives a decorator the node, its evaluated args, the same arguments as syntax in written, the parent, and four verbs: replace, remove, meta and reject.

| Export | Purpose | | --- | --- | | expand, ExpandResult, ExpandContext, DecoratorDefinition, DecoratorSource, DecoratedNode, NodeMeta | The mechanism and its contracts. | | TargetKind, TARGET_KINDS, isTargetKind, kindOf | The words a deco uses about what it decorates: Fn, Flow, Step, Binding, Type, Node. kindOf is the one place that vocabulary meets the compiler's $type names. | | makeHandle, handleSurface, TargetHandle, HandleSurface | The value a deco body holds for its target, and what it publishes. | | readSignature, DecoSignature, SignatureResult, acceptedKinds, decoTarget | Reading what a deco decorates off the type of its first parameter. | | decoDecorator, withDocumentDecos, ImportedDeco, DocumentDecoArgs, decoCannotCall, DecoBodyArgs | Turning a deco declaration, local or imported, into a definition expansion can run. | | metaOf, readMeta, writeMeta | Facts a decorator leaves on a node for the runtime. Non-enumerable, so they never land in a serialised AST. | | readDecorations, addDecoration, AROUND_KEYS, Decorations, decorateCallable | Where .wrap, .before and .after leave their closures, and how a callable picks them up. | | swapNode, spanOf | Replacing a node in its container, and locating one. | | wrongPlace, wrongKind, wrongKindTitle, wrongTargetTitle, kindWords, nodeWord, everyKindWritten | Whether a decorator belongs where it is written, and the prose behind VN2014, phrased in the author's words rather than in node type names. wrongPlace is the one question expansion and venn check both ask, so the refusal reads the same whichever found it. |

Units

A number literal may carry a unit, and the unit survives arithmetic.

const took = 250ms
const body = 1mb
const rate = 99.9%

expect took < 300ms
expect body <= 2mb

parseNumber turns a NUMBER lexeme into a plain number or a UnitValue; parseInstant turns an ISO-8601 lexeme into an Instant that keeps its source text. combine({ op, left, right }) does the arithmetic and comparison: 300ms + 1s succeeds, 300ms + 2mb returns the mismatch behind VN3012. A moment takes part too: ended - began is the duration between them, began + 2h and began - 2h are moments, and anything else involving one is a mismatch. isUnitValue and isInstant are the guards. Canonical bases are milliseconds, bytes, a ratio from 0 to 1, and epoch milliseconds.

Interpolation

scanInterpolations(text) locates every ${…} precisely enough to highlight, hover and jump to. compileTemplate(text) splits a literal once into its constant chunks and its holes, each hole carrying both its source and its parsed Expr. There is always exactly one more chunk than holes, so rejoining them cannot drop text. Both are cached by text, because a .vn file holds a fixed set of string literals.

stringifyValue(value) is how a filled placeholder reads, and the only definition of it: a list as [1, 2], a map as { hits: 0, name: "ada" }, a moment as its ISO text, a unit keeping its unit, and nothing at all for null. No value can read as [object Object].

displayValue(value) is the same definition for print and str, which is why print x and "${x}" cannot disagree about the same value. One rule differs, and only at the top: nothing reads as null, because print x asked what x is and deserves an answer, while add ${name} with no name reads better as add than as add null.

Formatting

import { DEFAULT_FORMAT, formatOptionsFrom, formatText } from "@venn-lang/core";

formatText(source, { indentWidth: 4 });
formatText(source, formatOptionsFrom(manifest.format)); // the [format] table of venn.toml

formatText re-indents by bracket depth and, by default, gathers every import into one block. It never joins or splits lines, and it is idempotent. organizeHeader and reindent are the two steps on their own.

Events and plans

The events/ module is types only: Envelope is the single contract between runner, host and UI, carrying a monotonic seq, a timestamp, a RunId, a kind and its payload. EventKind is derived from the keys of EventData, so adding an event is one edit. RunPlan is what the UI draws in grey before anything executes.

node says which step an envelope is about; Envelope.step says which RUN of it. A step inside a forEach has one NodePath and a StepId per pass, and parallel opens two at once by design, so a reporter with two openings and one later failure has no way to attribute it without the second id, and attributing by arrival order is wrong exactly when it matters.

Status has five values, not four: cancelled is neither verdict. A break, return or exit cut the step short, so it reached no verdict at all, and calling that passed counted work nobody did.

stepTitlesOf(flow, document) in plan/ is every step title a flow can reach: nested in groups, loops, branches and try, and followed through run <fragment> into the fragments of the same file. It lived in the CLI, which meant the plan a reporter drew and the plan the runner walked were two readings of one tree.

Modules

specifierKind(spec) classifies an import specifier the way Node does: ./util.vn is relative, #shared/auth.vn is an alias resolved through [paths], and a bare name is an installed package. isPackageSpecifier is the common question asked of it.

Grammar

The grammar lives in src/grammar/venn.langium and is the fixed part of the language. It knows flow, step, group, fragment, fn, deco, type, config, matrix, control flow, lifecycle hooks, expect, and one rule that absorbs every protocol present and future:

ActionCall:
    target=QualifiedName (called?='(' (call=ArgList)? ')')?
    (args+=ActionArg)* (opts=MapLit)?;

That is why http.get "url" { headers } needs no grammar change to exist. Whether a dotted target is a plugin verb or a method on something in scope is decided by what the name resolves to, not by how the call is written.

Statements are terminated by a newline or ;. The lexer drops those inside ( ) and [ ], so an argument list or a list literal may still span lines, while blocks and maps keep their newlines.

Regenerate the parser after editing the grammar:

pnpm --filter @venn-lang/core langium:generate
pnpm --filter @venn-lang/core test

See also

• @venn-lang/runtime walks the checked document and executes it.
• @venn-lang/types is the type vocabulary as plain data, shared with plugins.
• @venn-lang/lsp reuses the parser, checker and formatter to serve editors.
• docs/venn-language.md is the language specification; docs/type-system.md covers the types.