@codincod/codemirror-lang-kotlin

[ CHANGELOG ]

This package implements Kotlin language support for the CodeMirror code editor, using a Lezer grammar written for this package.

CodeMirror 6 has never shipped Kotlin, and no Lezer grammar for it existed before this one. What gets used instead is the C-family stream mode ported from CodeMirror 5, which colours a list of keywords and builds no tree, so nothing above it can fold a function, indent a when or ask what the word under the cursor is. It also predates most of the language: string templates, raw strings, labels and annotations all come out as plain text.

Written in part for CodinCod, a competitive coding platform, where it colours the editor people solve puzzles in.

This code is released under an MIT license.

Usage

import {EditorView, basicSetup} from "codemirror"
import {kotlin} from "@codincod/codemirror-lang-kotlin"

const view = new EditorView({
  parent: document.body,
  doc: `data class Greeter(val name: String) {
    fun greet(): String = "Hello, $name!"
}
`,
  extensions: [basicSetup, kotlin()]
})

Coverage

Kotlin 2.2. Classes, interfaces, objects and companions, data and value classes, sealed hierarchies, enums, annotations and their use site targets, typealiases, extension functions and properties, generics with variance and where clauses, delegation, property accessors and delegates, coroutines and suspend, lambdas and trailing lambdas, callable references, destructuring, labels and labelled jumps, when with and without a subject, and every way Kotlin writes a string, including the multi dollar raw strings 2.2 added.

Measured against 6 589 Kotlin files from six open source projects (ktor, detekt, kotlinx.coroutines, Exposed, kotlinx.serialization and okio), which is 26 MiB of code, of which 97.33% parse with no error node.

What the grammar cannot say

Four things about Kotlin are settled in src/tokens.ts rather than in the grammar, because no context free rule can decide them.

A newline ends a statement, except when it does not. Kotlin has no semicolons and no statement terminator, so a\nb is two statements and a +\nb is one. The tokenizer emits a zero width insertSemi when a line break stands between two things that could each begin a statement, and holds it back when the next line opens with something that can only continue the one before: an operator, a dot, a closing bracket, as, by, else, finally, a catch followed by its parenthesis, a where followed by a constraint, or a get or set that turns out to be an accessor rather than a call. That last list is the whole difficulty. set.addAll(x) at the start of a line is a call to a set, and set(value) { } under a property is an accessor, and only what follows the word tells them apart.

A < is a comparison or a type argument list. a < b and Map<K, V> are the same two characters. When the parser could shift a comparison the tokenizer scans ahead over what a type argument list is allowed to contain, up to two hundred characters, and only emits the bracket tokens if it reaches a matching > followed by something that could stand after a type. A * inside the brackets settles it immediately, because a star projection cannot be anything else.

A string is four different languages. "a", """a""", $"a" and their templates disagree about what ends a string, what an escape is, and whether a $ opens a hole. The tokenizer cuts each one into a start, its content and an end, and the grammar parses the expressions between the pieces. A raw string is read whole, because its content is whatever stands before three quotes.

A block comment nests. /* /* */ */ is one comment in Kotlin and two in most of its relatives, so the tokenizer counts depth rather than searching for the first */.

One more is settled in the grammar, and is worth knowing about.

Casing decides what a name is. After one identifier no amount of lookahead separates a type from a variable, so the parser follows the convention every Kotlin codebase keeps: Foo is read as a type, FOO_BAR as a constant, and foo as an ordinary name. Both readings stay alive where the grammar allows either, and the one that fits wins. The trick is @lezer/java's.

Known gaps

when guards, the if that Kotlin 2.1 allows after a branch condition, are not parsed. The guard and a branch whose condition is itself an if expression share a prefix that an LR parser cannot tell apart, and resolving it in the guard's favour cost more real files than it fixed.

Testing

npm test                       # the parse specs, then highlighting and indentation
npm run corpus -- path/to/code # error nodes over a directory of real Kotlin files

No corpus ships with the package, because none of that code is ours to redistribute. test/corpus.ts takes a directory and reports the share of files that parse with no error node, grouped by directory so that a project which drags the total down names itself.