MAQL ANTLR4TS npm package

The module bundles TypeScript MAQL lexer and parser generated from MAQL grammar, a visitor to create MAQL AST, and an abstract AST visitor to work with MAQL AST.

The lexer and parser are generated by antlr-ng and run on the antlr4ng runtime, which is declared as a dependency of this package. The runtime has no dependencies of its own and no Node.js built-in requirements, so the package works in browsers without polyfills.

Note: antlr4ng must resolve to a single copy in the consuming application. If the application depends on antlr4ng directly or through another package (for example antlr4-c3), it must use the exact same version as this package, otherwise instanceof checks between the runtime copies fail.

The package name keeps the historical antlr4ts suffix; the antlr4ts runtime is no longer used.

Example usage:

import { CharStream, CommonTokenStream } from "antlr4ng";

import { MaqlLexer, MaqlParser, MaqlAstVisitor, AstNormalizer } from "@gooddata/maql-antlr4ts";

const maql =
    "parse metric SELECT SUM(0.81 * {fact/price} * {fact/quantity}) BY {attribute/country} WITHOUT PF";
const charStream = CharStream.fromString(maql);
const lexer = new MaqlLexer(charStream);
const tokenStream = new CommonTokenStream(lexer);
const parser = new MaqlParser(tokenStream);
const parseTree = parser.maql();
const astVisitor = new MaqlAstVisitor(workspace, tokenStream);
const ast = astVisitor.visit(parseTree);
const normalizer = new AstNormalizer();
const normalizedAst = normalizer.visit(ast);

The grammar declares the caseInsensitive option and the generated lexer honors it, so MAQL keywords may be written in any letter case with a plain CharStream.

Use the exported CaseInsensitiveCharStream instead of CharStream.fromString when token positions have to match JavaScript string offsets or LSP ranges. The antlr4ng stream indexes by Unicode code point, so a character outside the Basic Multilingual Plane (an emoji, for example) shifts every following start, stop and column by one against the source string. CaseInsensitiveCharStream indexes by UTF-16 code unit, so its positions are string offsets.

Local development

Requirements: pnpm

| command | effect | | ------------------ | ------------------------------------------------------------------ | | pnpm install | install all dependencies | | pnpm build | generate source files, type-check, and compile into esm/ | | pnpm codegen | generate source files from MAQL grammar and function definitions | | pnpm antlr4 | generate lexer, parser, listener and visitor from the MAQL grammar | | pnpm funcdef | generate source files from MAQL function definitions | | pnpm build-check | type-check all sources, including tests | | pnpm build-ts | compile src/ into esm/ without type-checking | | pnpm clean | remove the esm/ output | | pnpm test | run all tests in watch mode | | pnpm test-once | run all tests once | | pnpm test-ci | run all tests once with a junit.xml output file | | pnpm lint | run oxlint and ESLint | | pnpm lint-fix | automatically fix oxlint and ESLint errors | | pnpm format | check formatting with oxfmt | | pnpm format-fix | automatically fix formatting with oxfmt | | pnpm validate | run pnpm lint and pnpm format |

The ANTLR grammar lives in ../maql-parser/src/main/antlr. Generated sources go to src/gen/ and are not committed; pnpm build regenerates them. The generator runs on Node.js, no Java is required.

Linking the package to other modules

First install the dependencies, then build the module.

pnpm install

pnpm build

Warning: Do not use pnpm link or npm link.

Symbolically linking this package can make the consuming application load a second copy of the antlr4ng runtime, which breaks instanceof checks between the two copies at runtime.

Use yalc to link the package to other repositories.

1. Run yalc publish in the current directory.
2. Run yalc add @gooddata/maql-antlr4ts in the target directory
3. Once you are done with testing run yalc remove @gooddata/maql-antlr4ts or yalc remove --all

Publishing

To publish the package, run the publish_maql_antlr4ts_npm GitHub workflow manually.

The workflow publishes a prerelease version <package.json version + 1 patch>-<commit sha>.0, for example 0.5.1-a5e4f136.0. It moves the npm latest dist-tag to this version. It does not commit the version back. To raise the base version, change version in package.json in a PR before you run the workflow.

The workflow publishes to npm with trusted publishing (OIDC), so it does not need an npm token. Trusted publishing does not support self-hosted runners, so the workflow runs on a GitHub-hosted runner.

Trusted publisher setup

The trusted publisher for the package is set once on npm. This section records how it was set up, so that you can change it or set it up again.

You can set it up in one of these ways:

• In the npm UI at https://www.npmjs.com/package/@gooddata/maql-antlr4ts/access.

• With the npm CLI. Use npm 11.10.0 or newer, which has the npm trust command. Log in to npm (npm login) as user gooddata-ci. The user must have publish access to @gooddata/maql-antlr4ts and two-factor authentication. Then run:

  npm trust github @gooddata/maql-antlr4ts \
      --file publish_maql_antlr4ts_npm.yml \
      --repo gooddata/gdc-nas \
      --allow-publish

To see the current configuration, run npm trust list @gooddata/maql-antlr4ts.