@codincod/codemirror-lang-scala

This package implements Scala language support for the CodeMirror code editor: a Lezer grammar, highlighting, indentation, folding and completion for the core library.

The grammar is written from scratch. Scala has never had a Lezer grammar, and what a CodeMirror 6 editor could reach for until now was the clike stream mode in @codemirror/legacy-modes, which reads Scala as a curly-braced language with a longer keyword list.

Both dialects are read by one grammar. A file may brace its blocks or indent them, and may do both.

Written in part for CodinCod, a competitive coding platform, where it colours the editor people solve puzzles in.

This code is released under an MIT license.

Usage

import {EditorView, basicSetup} from "codemirror"
import {scala} from "@codincod/codemirror-lang-scala"

const view = new EditorView({
  parent: document.body,
  doc: `enum Shape:
  case Circle(radius: Double)
  case Rectangle(width: Double, height: Double)

object Shape:
  extension (shape: Shape)
    def area: Double = shape match
      case Circle(r)       => math.Pi * r * r
      case Rectangle(w, h) => w * h

@main def report(): Unit =
  val shapes = List(Shape.Circle(1.0), Shape.Rectangle(2.0, 3.0))
  for shape <- shapes do
    println(f"\${shape.area}%2.2f")
`,
  extensions: [basicSetup, scala()]
})

What it reads

Measured against the stream mode it replaces, over 1072 files of Scala from CodinCod's language-guessing pool: the stream mode carries a highlighting tag on 85.76% of the text in 7 colours, with 43.30% of it in the busiest one. This package tags 99.98% in 11 colours, with 35.71% in the busiest.

Over the same 1072 files, 86.29% parse with no error node. Over 29546 files and 59.84 MiB of cats, zio, the Scala 2 compiler and the Scala 3 compiler, 87.31% do: cats 94.85%, zio 90.51%, scala2 91.83%, scala3 84.53%. The two compilers carry their own test suites, and 7838 of those files are code that was written not to compile, so a share of the remainder is Scala nobody meant to be read.

Three things took most of the work.

• Scala 3 writes no braces. A region opens where a line stands further right than the one that opened it, and closes at the first line to the left of it. The parser is handed zero-width indent and dedent tokens for the braces nobody wrote. A region is only ever opened where the grammar could take a body, so an argument list broken over lines opens nothing, and a class header continued on the next line is read as the header it still is.
• A line break ends a statement. Scala spends few semicolons, so one is inserted wherever a break ended something that could stand alone. The newline is kept in the parser's context rather than read back out of the text, because a zero-width token leaves the text exactly as it found it, and a token that cannot tell it has already fired will fire forever.
• Scala 3 reserved nothing. end, open, inline, opaque, transparent, infix, extension and derives are all ordinary names, and in this corpus they are: end is somebody's field or method wherever a collection has one. Each is decided by one word of lookahead in the tokenizer, so where the grammar wants the keyword it gets it, and everywhere else the name survives. case is decided the same way, which is what tells case class Foo from the case Foo that opens a clause, and @inline is an annotation rather than a modifier because of the character in front of it. inline and erased modify a parameter as well as a definition, and there the word after them is somebody's parameter name, so those two are decided by the name and colon standing after them instead.

Smaller things worth naming.

• Strings are cut into their pieces. s"hi $name, ${1 + 2}" reaches the tree as an interpolator, its text, a variable and a block of code, so a theme can colour the hole differently from the string around it. A plain string takes escapes and no holes, """raw""" takes neither, and s"""...""" takes holes and no escapes. Each is a rule of its own, because the tokenizer asks the parser which kind of string it is standing in and one rule covering two of them answers wrongly.
• A name says what it is. A capitalised name is a type and a lowercase one is a value, which is the whole of what tells case Some(x) matching from case some binding. A parameter, a definition and a use each reach a different node.
• * is four tokens. Multiplication, the star that ends a parameter list, the one that spreads a sequence into an argument list, and kind-projector's Map[K, *]. The middle two are decided by what stands on either side of the star, because by the time the grammar sees it the parser has already chosen to multiply.
• An operator with an operand on neither side is a name. ??? is the standard library's value for something nobody has written yet, and it is a method spelled out of the characters an operator is made of. An infix operator has something to its left and a prefix operator something to its right, so a run with neither is neither, and reaches the tree as the name it is.
• Capture checking. IterableOnce[A]^ captures something and AnyRef^{ll} says what. dotty's own library is written in it. The caret is an ordinary method name as well, and is told apart the same way the splat star is: a capture touches the type it closes.
• A type computes. X match { case String => Char } is read by the rule a pattern match is read by, so its clauses reach the tree as the clauses they are. [X] =>> F[X] and [X] => F[X] => G[X] are one rule told apart by its arrow. A Ior B is Ior[A, B], which is how cats and zio are written.
• A list may end on a comma. Scala 2.13 lets a bracket broken over several lines carry one after its last item, and every build file does.

What it does not read

• Postfix notation. xs head is legal Scala 2 and is not read as a call. Both readings of 1 to 10 parse, and keeping the postfix one would make every x max y in the corpus ambiguous for the sake of a form Scala 3 has deprecated.
• XML literals. Dropped from Scala 3, and 0.6% of the corpus.
• inline x match. The word before an inline match is decided a word at a time like every other soft keyword, and the match that would decide this one stands after the thing being matched rather than after the word.
• Capture arrows and capture-set bounds. List[() ->{C} Unit] and [A: {Arbitrary, Ordering}]. The first is dotty-internal; the second collides with a refinement type at the brace, and is 0.2% of the corpus.
• given (using c: C): T = ..., the unnamed given whose signature is nothing but a parameter list. The named form and the one taking type parameters both work.
• Named tuples, (name: String, age: Int), which Scala 3.7 added.

One thing that looks like a gap and is not a rule. A file deep enough in braced blocks inside an indentation region loses the else of an if, and every single reduction of the case makes it parse. That is @lezer/lr pruning a stack rather than a production nobody wrote; the same file fails the same way with two of its patterns removed and passes with three.

Measuring it yourself

npm run corpus -- path/to/scala/files

No corpus ships with the package, because none of that code is ours to redistribute. Point it at your own; it counts files with no error node and groups them by directory, so a project that drags the total down names itself.