@codincod/codemirror-lang-ruby

[ CHANGELOG ]

This package implements Ruby language support for the CodeMirror code editor, using a Lezer grammar written for this package.

Written in part for CodinCod, a competitive coding platform, where it colours the editor people solve puzzles in.

This code is released under an MIT license.

Usage

import {EditorView, basicSetup} from "codemirror"
import {ruby} from "@codincod/codemirror-lang-ruby"

const view = new EditorView({
  parent: document.body,
  doc: `class Greeter
  def initialize(name)
    @name = name
  end

  def greet
    puts "Hello, #{@name}!"
  end
end
`,
  extensions: [basicSetup, ruby()]
})

Coverage

Ruby 3.3. Pattern matching, endless methods, hash shorthand, safe navigation, numbered block parameters, rational and imaginary literals, and everything older that is still written every day: heredocs of all four kinds, percent literals with any delimiter, singleton classes, operator methods, begin blocks with their rescue and ensure, and blocks in both spellings.

Measured against 1445 Ruby files from Rosetta Code, 95.64% parse with no error node. Of the 63 that do not, 39 are not Ruby: irb transcripts, C and shell saved with a .rb name, snippets cut off in the middle of a string, and pseudo-code with a literal ... in it. The 24 that are Ruby average two error nodes each, and no two of them are the same problem.

What a tokenizer has to settle

Almost all of Ruby can be written down as a grammar. Seven things cannot, and they are the whole of tokens.ts.

A slash is a pattern or a division. a / b divides and a =~ /b/ matches, and the same character opens both. Where a division cannot stand the parser has already ruled it out; where both could, Ruby's own rule decides, which is that a /b/ with a space in front and none behind is a call taking a pattern and a / b is arithmetic.

A brace is a block or a hash. each { |x| } passes a block and {a: 1} is a value. Ruby tells them apart by what stands in front of the brace, so the tokenizer asks the parser whether a block could stand here and lets the answer decide. The same question settles f(x) against (x), def f(x) against both, and while x do against xs.each do.

A newline ends a statement. x = 1 followed by if y is two statements; x = 1 if y is one. rescue splits the same way: on its own line it opens a clause, and after an expression it catches one. So does the end of an argument list written without parentheses. This grammar skips newlines like any other space, so each of those is decided by looking at where the line began, and the last of them is marked with a token of no width.

A heredoc's body is nowhere near its name. <<~SQL promises a body that starts on the next line, and one line may promise several. Rather than carry the promise from one token to the next, the body is read at the newline by looking back over the line that just ended, and then skipped like whitespace.

A colon is a hash key or a punctuation mark. a: written against a name is a key, and the second colon of A::B says it is not. A regular token takes the longer match and reads A::B as the key A: followed by the symbol :B, so the second colon is looked at before the first is believed.

A percent literal chooses its own delimiter. %w[a b], %i(x y), %q{...} and %r|...| are one form with the delimiter picked per use, and the bracketing ones nest.

A string holds code. "total: #{a + b}" is a string with an expression inside it, and that expression may hold another string. The tokenizer cuts the string into pieces and the grammar parses what lies between them.

What the tree does not say

Two things are deliberately shallower than the language reference.

A call without parentheses takes one argument. puts result and Marshal.load(Marshal.dump orig) are read as calls; foo a, b without brackets keeps only the first argument in the call. Ruby decides the rest by knowing whether each name is a method or a local variable, which is a question about the program rather than about the text.

Patterns are expressions. case x in {name: String} holds a shape that only appears after in. Rather than carry a second grammar for patterns, the hash and the array read as the literals they resemble, and only the pin (^x) and the alternation are written down separately.