@codincod/codemirror-lang-dc

This package implements dc language support for the CodeMirror code editor: a Lezer grammar and highlighting.

dc is the reverse-Polish desk calculator that ships with every Unix, and it is a programming language because of one character: [ collects everything up to its matching ] into a string, and x executes a string.

No grammar for dc existed anywhere before this one, in the Lezer generation, in the CodeMirror 5 modes, or in tree-sitter.

Written in part for CodinCod, a competitive coding platform, where it colours the editor people solve puzzles in.

This code is released under an MIT license.

Usage

import {EditorView, basicSetup} from "codemirror"
import {dc} from "@codincod/codemirror-lang-dc"

const view = new EditorView({
  parent: document.body,
  doc: `[d lp * sp 1 - d 1 <f]Sf   [product = product * n]sz
1 Sp 50 lfx Lp p
`,
  extensions: [basicSetup, dc()]
})

What it reads

A bracket holds code, or it holds text, and dc cannot tell which until something executes one. All three uses are written the same way:

[d lp * sp 1 - d 1 <f]sf   a macro, stored to be called
[Take one down, ]P         a string, printed
[product = 1]sz            a comment: pushed, stored nowhere, forgotten

Reading all three as strings leaves a dc program with almost no colour in it, because 84% of the characters in real dc code sit inside brackets. Reading all three as code turns every sentence into a rainbow, because P, r, i, n and t are each a command.

So the tokenizer decides, on a fact about the language rather than a guess about English: a bracket holds code when every character in it is one dc has a meaning for. Take one down cannot be code, because dc has no T. mul: cannot, because it has no m. A register command with nothing left to name is not code either, which is what gives [Loop:] away as a heading. And a bracket that is printed rather than executed is text whatever it is spelled with, which is why [Fizz]P is a word and not four commands.

Registers are the other half. The character after s, l, :, ;, <, > or = is the name of a register, whatever character it happens to be: real programs use s!, S@, s. and s#, and dc will take a space if you leave one. Nothing about that character may be interpreted, which is why this is a grammar rather than a table of one-character tokens.

The alphabet is GNU's, and the commands BSD dc adds are in it where they can be: G, J, M, N and R, the four bracket comparisons (, ), { and }, and the else. An uppercase letter is rare in the middle of an English sentence and a bracket is rarer, so none of those cost anything. The BSD commands spelled with a lowercase letter (m, t, u, w, y) are left out, because a lowercase letter is what prose is made of: admitting them costs more sentences than it explains programs, and each was measured before it was left out.

The else is the exception, and the one character in dc with no token of its own. >ReS runs S when the comparison fails, so an e is a command immediately after a comparison and its register, and a letter everywhere else. An external tokenizer reads what stands behind it.

What it was measured on

Two bodies of dc, because they are written by different people for different reasons.

The first is every dc program on Rosetta Code, which is very nearly all the dc anybody has published: 101 files, 1201 lines. 97 parse with no error node. The four that do not are shell command lines someone pasted above their program (dc -e '22 7/p', echo '12345678899' | dc inc.dc), so every actual dc program in it parses clean.

The second is the dc that implementers write: Gavin Howard's bc test suite and OpenBSD's dc regression tests, 70 files of working dc. 69 parse clean. The one that does not stores into OpenBSD's extended registers, which are named by the raw byte 255 and the two bytes behind it, on 514 of its 570 lines. A raw byte is not a character and an editor holds decoded text, so that file cannot be put in a CodeMirror document at all, let alone parsed in one.

Neither figure means as much as it looks. A character dc has no answer for is a node of its own rather than a failure, so this grammar has almost nothing to fail at. 30 of the 33 files in Howard's suite that exist to be rejected by dc parse clean here. A C file gives one error node, a JSON file one, a page of English four. Treat a clean parse as evidence of very little, and 101 files as what they are: enough to say the true rate is above 97%, and nothing stronger.

The number that means something is the one the grammar exists to decide. Across the Rosetta corpus it reads 371 brackets as code and 392 as text, no dense run of commands is read as text, and 55% of the characters end up coloured as text. Nearly three quarters of that is the comment convention: a sentence pushed and dropped into a register nothing reads. dc programs really are that much English, and one colour covers all of it, because dc has no comment syntax and no way to tell a comment from a string it means to print.

The second corpus was worth fetching. It found the four commands above, and one bug: an else was read after any command that names a register rather than only after a comparison, which made code of [A = R[54] = seed] by reading the se of "seed" as a store and the e behind it as an else.

# is a comment to the end of the line in GNU dc, and almost nothing in the wild uses it. One program in the corpus contains a #, as the name of a register.