@codincod/comment-syntax

How 645 programming languages spell their comments and their strings, and a scanner that cuts the comments out.

npm install @codincod/comment-syntax

import { strip, forLanguage, supports } from "@codincod/comment-syntax"

strip("x = 1  # note", "python")
// "x = 1"

strip('puts("/* text */"); /* note */', "c")
// 'puts("/* text */");'

forLanguage("ocaml")
// { line: [], block: [["(*", "*)"]], strings: ['"'], nested: true, ... }

supports("befunge")
// false

Why this exists

Every entry was read off real code in the language before it was written down. That is the expensive half of a comment stripper; the scanner around it is three hundred lines and anybody could write it.

The table knows that OCaml nests its block comments, that Simula runs one to the next ;, that LOLCODE closes with TLDR, that ALGOL-M closes with the same % it opened with, that a # inside a Python docstring is text, that shell ${#arr} is not a comment, and that ABAP's * comments a line but multiplies inside one.

It was built for CodinCod's guess-the-language game, where a comment naming the language turns a round into a reading test.

The data on its own

The scanner is optional. The JSON is the artifact, and it is reachable without installing anything:

https://unpkg.com/@codincod/comment-syntax/data/syntax.json

{
  "schemaVersion": 1,
  "languages": {
    "ocaml": {
      "line": [],
      "lineStart": [],
      "block": [["(*", "*)"]],
      "strings": ["\""],
      "multilineStrings": [],
      "directives": [],
      "nested": true
    }
  },
  "aliases": { "cpp": "c++" },
  "families": [{ "name": "basic", "syntax": {} }]
}

schemaVersion changes when the shape changes, not when a language is added.

What an entry means

| Key | What it holds | | --- | --- | | line | Comments to end of line, counted at line start or after whitespace | | lineStart | Counted only where it opens the line, for a marker that is an operator elsewhere | | block | [open, close] pairs. The closer is often not a newline | | strings | Quote characters, tracked within a line | | multilineStrings | [open, close] pairs for strings that genuinely span lines, possibly asymmetric | | directives | Sequences spelled with the comment brackets that are not comments | | nested | Whether a block comment can contain another one |

Resolving a name

Three steps, in this order: trim and downcase, follow an alias (cpp is C++), then fall back to the first family whose name the language contains (FreeBASIC is a BASIC). Resolving it yourself from the raw JSON means doing all three; a consumer that reads only languages answers "unknown" for the 78 BASIC dialects the family line covers.

What this is conservative about

It was built so a wrong cut never corrupts code, which makes it careful in ways you should know about before you read a false as a bug.

• A language with no entry is returned unchanged. An unknown grammar is never guessed at.
• Some absences are deliberate. Befunge is every character an instruction on a 2D playfield, so it has no comment syntax to publish. supports("befunge") is false and always will be.
• A family fallback is a good guess, not a verified entry. Dialects in families inherit their parent's rules because they agree about comments, which is checked per family and not per dialect.
• An unterminated block drops the whole strip. A block comment still open at the end of a snippet means the grammar was misread, and acting on that reading would swallow the rest of the file. You get the original back.
• A leading shebang is always dropped, for every language, known or not, because it names the interpreter.
• Blank lines are normalised. Lines that held only a comment go away, blank lines the author wrote survive, and runs of blanks collapse to one.

Types

Everything is typed, and the language names are a union, so a literal completes and a typo is a compile error:

import type { CommentSyntax, LanguageName, SyntaxTable } from "@codincod/comment-syntax"

const language: LanguageName = "ocaml"

Functions take LanguageInput, which is that union widened with string, because aliases and family dialects resolve too and the name usually arrives from a file rather than from you typing it.

Where the table is maintained

In Elixir, in CodinCod's own tree, and exported here with mix comment_syntax.export. Adding a language is a change there and a release here. The JSON in this repository is generated and should not be hand-edited; a fix that lands only in the copy disappears at the next export.

The TypeScript scanner is a port of the Elixir one, and the test suite is differential: test/fixtures.json and test/sweep.json hold outputs produced by the Elixir implementation, one snippet for every language in the table, and the tests assert this port answers identically.

Licence

MIT.