@codincod/codemirror-lang-haskell

This package implements Haskell language support for the CodeMirror code editor, using a Lezer grammar written for this package.

CodeMirror has never shipped Haskell. What a CodeMirror 6 editor can fall back on is the stream mode from CodeMirror 5, which colours keywords and literals a token at a time and knows nothing about the shape of what it is reading. No Lezer grammar for Haskell existed anywhere before this one.

Written in part for CodinCod, a competitive coding platform, where it colours the editor people solve puzzles in.

This code is released under an MIT license.

Usage

import {EditorView, basicSetup} from "codemirror"
import {haskell} from "@codincod/codemirror-lang-haskell"

const view = new EditorView({
  parent: document.body,
  doc: `module Main where

import Data.List (sort)

main :: IO ()
main = do
  input <- getContents
  mapM_ print (sort (map read (lines input) :: [Int]))
`,
  extensions: [basicSetup, haskell()]
})

Coverage

Haskell 2010, and the extensions GHC 9 turns on for ordinary code: bang patterns, lambda case, multi-way if, GADTs, standalone and strategy deriving, type families, functional dependencies, pattern synonyms, view patterns, tuple sections, MagicHash, the arrow proc notation, and the Template Haskell quotations and splices that fit in [| |], ''Name and $( ).

Measured against 832 files of Haskell that somebody else wrote, 98.56% parse with no error node:

| | files | clean | | --- | --- | --- | | xmonad | 30 | 100.00% | | lens | 117 | 99.15% | | text | 106 | 99.06% | | pandoc | 366 | 98.91% | | aeson | 137 | 97.81% | | containers | 76 | 96.05% | | | 832 | 98.56% |

Five of the twelve files that are left are conditional on the C preprocessor, which hands GHC one of two branches and hands a parser both. Five use a quotation that cannot be told apart from ordinary Haskell without knowing which extensions are on, and are described below. The other two are a let binding with a type signature written into it.

The corpus does not ship with this package. It belongs to the projects above and carries their licences, so test/corpus.ts takes the path to your own copy of it.

What a tokenizer has to settle

Most of Haskell can be written down as a grammar. Seven things cannot, and they are the whole of tokens.ts.

Indentation opens and closes blocks. A where, a do, a let or an of opens a block at the column of the token after it, a line beginning at exactly that column starts a new item, and the first token to the left of it ends the block. Almost no Haskell writes the braces and semicolons the report says are there, so the tokenizer puts them back: it hands the parser a blockOpen, a blockSep and a blockEnd with no text in them, standing where the indentation put them. Where somebody did write the braces, they are read as the same two tokens, which is what lets one rule in the grammar serve both.

A block also closes where the parser would fail. let x = 1 in x keeps the in on the same line and nothing is indented differently, so no column says the block is over. The report closes an implicit block wherever the parser would otherwise fail, and the only thing that knows is the parser, so the tokenizer asks it: before in, then, else, of, where, a closing bracket or a comma, it checks whether the parser could shift that token, and ends the block where it could not.

A block that opens too far left is empty. A where with nothing under it, followed by a declaration at the left margin, opens a block and closes it in the same place. That is the rule that keeps the declaration below it a declaration rather than an error.

Block comments nest. {- {- -} -} is one comment rather than two, which no regular token can count. Two things ride along with them: {-# ... #-} is a pragma rather than a comment, and Haddock's -- |, -- ^, {-| ... -} and {-^ ... -} are documentation rather than either, so a highlighter can give all three different colours. A line comment is two or more dashes that no symbol character follows, which is what keeps --> an operator.

A name may carry its module. Data.Map.insert is one name in three parts, and the dots in it are not the composition operator. The tokenizer reads the module part off the front and hands it over as a token of its own, so it can be coloured as the namespace it is, and so M.map, M.Map and M.! need one rule each rather than three. Which of the three follows is decided there as well: f M.x applies a function and f M.+ y does not, and one token of lookahead is not enough to tell them apart afterwards.

Whitespace says what ! and ~ mean. m ! k looks something up and f !x = ... takes a strict argument, and the difference is the space. GHC decides it that way and so does this: a ! or a ~ with space in front of it and none behind it marks a pattern, and everywhere else it is an operator.

A name being declared looks like any other name. f x = x declares f, and xs !! n = ... declares the operator and neither name around it. Which one it is depends on what stands after the name, and an LR parser cannot hold both readings open long enough to find out, so the tokenizer looks one token ahead and decides, the same way it does for Odin. An operator behind the name means the name is not the one being declared, with three exceptions: :: opens a signature, = and | open a body.

What the tree does not say

Patterns are expressions. Haskell writes them with the same syntax, and the difference between Just x in a case and Just x in an argument is which side of the equals sign it stands on. Reading them as one thing accepts a few programs GHC rejects and refuses none that it accepts, which is the trade every highlighting grammar makes.

Types are their own small grammar. That is the opposite decision, and it is made for the highlighter: a name in a type is a type name, and the only way to say so is for the tree to know it is in a type. Types are read as a flat run of elements rather than as a tree of arrows, because nothing that reads this tree cares how a -> b -> c associates.

An operator chain is flat as well. Haskell resolves fixity after parsing, with infixl declarations the parser may never have read, so a + b * c is one run of three operands rather than a tree. A run that begins or ends on its operator is a section, a negation, or (+) on its own, and all three are the same shape here.

A lambda ends where its chain ends. \x -> a + b reads the whole of a + b as the body, which is what Haskell means, and it is why a lambda, a let and an if stand at the end of an operator chain and nowhere else in one. A do and a case end on a block instead, and the block says where they end, so a chain may carry on after those.

What is left out

Quasi-quotes. [foo| ... |] holds text in somebody else's language, and telling it apart from the list comprehension [foo|x<-xs] needs to know whether QuasiQuotes is on. A comprehension written without spaces is ordinary Haskell and a quasi-quote is not, so the comprehension wins.

The lettered and typed quotations. [e| |], [t| |], [d| |], [p| |] and [|| ||] have the same problem: [t|t<-ts] is a comprehension. The plain [| ... |], which is the one most Template Haskell writes, is read.

The C preprocessor. #if, #else and #endif lines are skipped the way comments are, which is right for a file that turns one feature on and wrong for a file that writes a declaration twice. Both branches reach the parser, and one of them is usually the error.