yipeo

XML-driven language runtime. Define a full language — lexer, parser, executor — in a single .yip XML file. No code generation, no compilation step.

Install

npm install          # install @xmldom/xmldom
npm link             # make `yipeo` available globally (optional)

Usage

yipeo run     <lang.yip> <source>   [--tokens] [--ast] [--json]
yipeo repl    <lang.yip>
yipeo check   <lang.yip>
yipeo tokens  <lang.yip> <source>
yipeo ast     <lang.yip> <source>
yipeo init    <name>
yipeo help
yipeo version

Quick start

yipeo init mylang
cd mylang
yipeo run mylang.yip example.mylang

.yip file structure

<yip>
  <lexer>   … token rules …   </lexer>
  <parser>  … grammar rules … </parser>
  <steps>   … custom step types (JS) … </steps>
  <exec>    … exec scripts (JS) …       </exec>
</yip>

Lexer

<lexer>
  <registery name="kw">
    <element>print</element>
    <element>let</element>
  </registery>

  <tokens>
    <token-type name="KW"    registery="kw"/>
    <token-type name="IDENT" regex="[a-zA-Z_][a-zA-Z0-9_]*"/>
    <token-type name="NUM"   regex="[0-9]+(\.[0-9]+)?"/>
    <token-type name="STR"   regex='"[^"]*"'/>
  </tokens>

  <skip>
    <element> </element>
    <element>\n</element>
    <element>\t</element>
  </skip>
</lexer>

Parser

<parser>
  <!-- name= makes the rule reusable via <sub-rule> -->
  <on type="KW" value="print">
    <node type="Print">
      <consume type="STR" as="value"/>
    </node>
  </on>
</parser>

Built-in parser tags

| Tag | Description | |-----|-------------| | <consume type="T" as="k"/> | Consume token by type | | <keyword value="v"/> | Consume token by value | | <return/> | Early exit | | <set-const as="k" value="v"/> | Store literal | | <peek-match type="T" as="k"/> | Lookahead without consuming | | <expect-end/> | Assert EOF | | <switch on="type\|value"> | Branch on token | | <if-peek type="T"><then/><else/></if-peek> | Conditional lookahead | | <optional>…</optional> | Try / backtrack | | <repeat as="k">…</repeat> | Zero-or-more | | <repeat-until type="T" as="k">… | Collect until terminator | | <choice as="k"><alt type="T">… | Ordered alternatives | | <list sep="S" as="k">…</list> | Sep-separated items | | <delimited open="O" close="C" sep="S" as="k">… | Bracketed list | | <sub-rule rule="name" as="k"/> | Recursive named rule |

User-defined step types

Define reusable parsing logic in JS:

<steps>
  <step name="consumeName">
    return expect('IDENT').value;
  </step>

  <step name="expression">
    <!-- full Pratt parser, call stack, whatever you need -->
    function led(left, tok) {
      var right = parseExpr(getPrecedence(tok));
      return { op: tok.value, left, right };
    }
    function getPrecedence(tok) {
      return { '+':10, '-':10, '*':20, '/':20 }[tok.value] || 0;
    }
    function parseExpr(minPrec) {
      var left = parsePrimary();
      while (true) {
        var t = peek(); if (!t || t.type === 'EOF') break;
        var p = getPrecedence(t); if (p <= minPrec) break;
        advance(); left = led(left, t);
      }
      return left;
    }
    function parsePrimary() {
      var t = peek();
      if (t.type === 'NUM')   { advance(); return { type:'num',   value: parseFloat(t.value) }; }
      if (t.type === 'IDENT') { advance(); return { type:'ident', value: t.value }; }
      throw new Error('Expected primary, got ' + t.type);
    }
    return parseExpr(parseInt(attrs.minPrec || '0'));
  </step>
</steps>

Call from any <node>:

<consumeName as="name"/>
<expression  as="expr" minPrec="0"/>

Step bindings: peek, advance, expect(type), expectValue(val), tokens, cursor, node, attrs, runtime

Exec

<exec>
  <script name="Print">
    console.log(node.value);
  </script>
  <script name="Assign">
    ctx.vars[node.name] = node.value;
  </script>
</exec>

Script bindings: node, ctx (ctx.vars, ctx.last), console

Programmatic API

const { buildRuntime, tokenize, parse, execute, runProgram, startRepl } = require('./yipeo-api');

const { tokens, ast, output } = runProgram(yipSrc, progSrc);