@codincod/codemirror-lang-postscript

This package implements PostScript language support for the CodeMirror code editor: a Lezer grammar, highlighting, folding and indentation.

PostScript is Adobe's page description language, and a stack language with a scanner small enough to state in a sentence and awkward enough to need three external tokenizers.

No grammar for it existed in the Lezer generation, and none in the CodeMirror 5 modes either. There is a tree-sitter grammar by Stefan Möding, which settles the same tokenizer questions and was worth reading before starting.

Written in part for CodinCod, a competitive coding platform, where it colours the editor people solve puzzles in.

This code is released under an MIT license.

Usage

import {EditorView, basicSetup} from "codemirror"
import {postscript} from "@codincod/codemirror-lang-postscript"

const view = new EditorView({
  parent: document.body,
  doc: `%!PS-Adobe-3.0
/inch { 72 mul } def
/Helvetica findfont 24 scalefont setfont
1 inch 5 inch moveto (Hello) show
showpage
`,
  extensions: [basicSetup, postscript()]
})

What it reads

Adobe's manual states the whole of the scanner in a sentence. A token is a run of characters that are neither white space nor one of the delimiters, and if that run has the syntax of a number it is a number, and otherwise it is a name.

Everything difficult follows from it.

There are no keywords. def, if and moveto are names bound to operators in a dictionary, and a program may shadow any of them. What this package colours as an operator is a claim about what a name will most likely mean, never about what it is. The tables are in src/tokens.ts, built by reading two lists against each other: what Ghostscript answers when asked for the operators in its systemdict, and Mike Williams's Vim syntax file, which sorts them by language level. Neither alone is right, because the first is missing what an interpreter writes in PostScript rather than in C and the second is missing what Ghostscript adds.

A number cannot be recognised from the left. 1e5 is a real and 1e is a name, and nothing shorter than the whole run tells them apart, so the run is read to its end and asked about afterwards. Every case in test/numbers.txt was settled by handing it to Ghostscript rather than by reading the manual: 5. and .5 are reals, 16#FG and 37#z and 1#0 are names, +42 is an integer, and + on its own is a name.

A bracket is an operator. [ pushes a mark and ] collects everything above it, so the pair is a run-time affair and need not be written inside one procedure or one file:

[ .shellarguments { ] dup length 1 eq { ... } ... } { pop } ifelse
mark encname cvn 3 -1 roll /exec load ] cvx
[ /Title (Report) /DOCINFO pdfmark

The last of those is Adobe's convention for putting anything into a PDF, and there is no ] in it because pdfmark is another of the operators that eat back to a mark. All three are in code the people who wrote the interpreter shipped. So a matched pair is read as an array and folds and indents like one, a matched group closed by a word does the same, and a bracket left over is the operator it always was.

Two percent signs in the first column are a different language. Document structuring comments are what a program tells a spooler, and a reader looking for a figure's bounding box is reading those lines and nothing else. They take a colour of their own. A rule of percent signs drawn across a page of code does not, which is what the first-column rule is for.

What the clean figure is worth

Almost nothing, and it is worth saying so.

Parsed against 400 files of real PostScript, this grammar leaves no error node in 95.50% of them. That number does not mean what the same number means for a language with keywords, because PostScript's scanner accepts almost anything: every printable run is a name, so a JSON document, a page of English and a Python program all parse without an error. What can fail is an unbalanced paren, brace or bracket, and a byte above 127 outside a string, which is usually the body of an image that got written into the file.

The figure that does mean something is how much of a file comes back carrying a highlight tag. Against CodinCod's pool of PostScript snippets that is 99.99% of the non-whitespace characters, across eight distinct colours.

The files this grammar still fails are of two kinds. Some carry a payload that is not PostScript at all, an ASCII85 blob or a run of ISO-2022 escapes, which no parser can read without executing the program that reads it. The rest open an array at the top of a file and close it inside a procedure, which the tree cannot express, and which costs one error node at the end of the file.

Development

npm test        # the parse specs, the highlighting, the folding and indentation
npm run corpus  # error nodes over a directory of real code
npm run gaps    # what failed, grouped by shape

The last two need lezer-survey beside this repository. No corpus ships here, because none of that code is ours to redistribute; Ghostscript's own library and examples, the Barcode Writer in Pure PostScript, and Vim's printing prologues are all worth pointing it at.