jdl-parser

The parser and the printer of the JHipster Domain Language (JDL), as a package of its own: the ones of generator-jhipster (lib/jdl/core/parsing and lib/jdl/core/printing), and its conversion of a jdl to the files of JHipster, without the generators. It depends on chevrotain, and for the conversion on @yeoman/namespace.

The parser knows the language, not JHipster: the application and deployment options, the entity options, the field types and the validations are definitions given to the runtime. generator-jhipster builds them from its generators, in lib/jdl-config; jdl-parser/jhipster has them for each release of JHipster from 8.0.0 on.

A validation of a field is a definition too, validation.configs, with the value it is written with: none, required ({ jdl: {} }); a number, a constant or a pattern, minlength(3) ({ jdl: { value: 'integer' } }); or either, { jdl: { value: 'integer', optional: true } }. required and unique are the ones of the language, definitions without them keep them.

An application or a deployment option is a definition too, application.configs and deployment.configs, with how its value is written, { jdl: { type: 'name', pattern: /^[a-z]+$/ } }, the values it takes, choices, and why it is deprecated, deprecated; a type is name, qualifiedName, string (a quoted text), integer, boolean, list or quotedList. The definitions generator-jhipster gives up to now, getDefaultJDLDefinitions() of generator-jhipster/jdl, have the options in another shape, four records by option name (validatorConfig, optionsTypes, optionsValues, quotedOptionNames): createRuntime converts them, and convertJDLDefinitions gives them in the current shape. Their entity options are given how a json entity of JHipster holds the ones it holds by another name (filter as jpaMetamodelFiltering), and their semantic rules, written for the parsed jdl of generator-jhipster, are replaced by the equivalent ones of jdl-parser/jhipster.

Usage

import { createRuntime, parseJDL } from 'jdl-parser';

const noOptions = { configs: {} };
const runtime = createRuntime({
  application: noOptions,
  deployment: noOptions,
  entity: { configs: {} },
  validation: { configs: {} },
});

const { ast, diagnostics, comments } = parseJDL('entity A {\n  name String\n}', runtime);

• parseJDL never throws: every problem is a diagnostic with its location, and the AST is built whenever the jdl lexes. The semantic rules of the language are checked, createRuntime({ rules }) adds some: a rule is { id, check(ast, runtime) }, returning its diagnostics.

• The AST is linear and generic, as ESTree: { type: 'JDL', body, source }, the statements in body in the order they are written, nothing merged nor repeated; every node has a type, a range ([start, end) offsets) and a loc ({ start: { line, column }, end }, lines from 1, columns from 0), and the nodes printed on their own their comments. An application holds its statements in body (Config, config(<namespace>) included, Entities, Option, Use), a Config and a Deployment their Entry nodes, each with its value as written (Name, String, Integer, Boolean, List, QuotedList), an entity its body (EntityBody, its fields) when it has braces.

  const { ast } = parseJDL('application { config { baseName store } }\nentity A { name String }', runtime);
  ast.body[0].body[0].entries[0]; // { type: 'Entry', key: 'baseName', value: { type: 'Name', value: 'store' }, range, loc }
  ast.body[1].body.fields[0]; // { type: 'Field', name: 'name', fieldType: 'String', validations: [], annotations: [], range, loc }

• printJDL(ast) writes the AST parseJDL returns back, its statements in the order they are written: the text of what did not change as written, a new node in the layout of the printer, every node with its comments; printJDL(ast, { format: true }) writes the whole jdl in that layout. A config value is quoted as it is written.

• The entry points are jdl-parser and jdl-parser/jhipster: the other modules are internal.

Optional rules

rules.optionalComma and rules.onePerLine, of the rules export of jdl-parser, are rules a tool adds to the ones of its definitions, suggestions about how a jdl is written, info diagnostics: between the items of a block, the fields of an entity, the options of a config or a deployment, the values of an enum, the relationships of a declaration, a comma is never needed, and an item per line reads better. The commas of a list, an entity list, an exclusion or a use statement are needed. rules.requiredComma is the inverse of rules.optionalComma, for a tool whose jdls separate the items with commas: it suggests a comma between two items written without one, none after the last item; a tool adds one or the other.

import { createRuntime, parseJDL, rules } from 'jdl-parser';
import { getDefaultJDLDefinitions } from 'jdl-parser/jhipster';

const definitions = getDefaultJDLDefinitions();
const runtime = createRuntime({ ...definitions, rules: [...definitions.rules, rules.optionalComma, rules.onePerLine] });
parseJDL('entity A {\n  name String, age Integer\n}', runtime).diagnostics;
// optional-comma: The comma after the field name of the entity A is not needed.
// one-per-line: The field age of the entity A is on the line of the previous one, one per line reads better.

The JHipster definitions

import { parseJDL } from 'jdl-parser';
import { getDefaultRuntime } from 'jdl-parser/jhipster';

const { diagnostics } = parseJDL('entity A {\n  name Strin\n}', getDefaultRuntime());
// field-type: The type Strin of the field name in the entity A is neither a field type nor an enum.

parseJDL('application { config { baseName gw routes ["blog:blog:8081"] } }', getDefaultRuntime('8.4.0')).diagnostics;
// syntax: Unknown application option: routes. (routes is a JDL option from JHipster 8.5.0 on)

jdl-parser/jhipster exports getDefaultJDLDefinitions and getDefaultRuntime, as generator-jhipster/jdl does, and jhipsterSemanticRules. They take the version of JHipster whose definitions to use, defaultJHipsterVersion (9.4.0) by default. The definitions are the ones of jhipsterVersions, every release from 8.0.0 to 9.4.0: a version takes the newest of them up to it (8.7.4 takes 8.7.3, 9.5.0-beta.1 takes 9.4.0, a prerelease coming before its release), an older one than 8.0.0 takes 8.0.0, and one from the minor after the newest, 9.5.0, throws, as an invalid version does. resolveJHipsterVersion(version) tells the one taken. A runtime of other definitions is made by createRuntime, from the ones of a version: it replaces createJDLRuntime of generator-jhipster/jdl.

JHipsterJDLModule is the type of the jdl module of generator-jhipster, as a tool imports it from an installed JHipster: getDefaultJDLDefinitions() gives the definitions of its version, its semantic rules included, and createRuntime the runtime parseJDL checks a jdl with. jdl-parser/jhipster is one; generator-jhipster checks its own against the type (satisfies JHipsterJDLModule), and a tool takes the definitions of a JHipster newer than jhipsterVersions from it.

import { createRuntime, parseJDL } from 'jdl-parser';

/** @type {import('jdl-parser/jhipster').JHipsterJDLModule} */
const jdl = await import(pathToFileURL(resolveFrom(project, 'generator-jhipster/jdl')).href);
const { diagnostics } = parseJDL(text, createRuntime(jdl.getDefaultJDLDefinitions()));

import { createRuntime } from 'jdl-parser';
import { getDefaultJDLDefinitions } from 'jdl-parser/jhipster';

const definitions = getDefaultJDLDefinitions('8.7.3');
const runtime = createRuntime({
  ...definitions,
  application: { configs: { ...definitions.application.configs, myOption: { jdl: { type: 'name' } } } },
});

The definitions are src/jhipster/definitions.ts, maintained by hand: first taken from generator-jhipster (lib/jdl-config/jdl-runtime.ts), with the descriptions of the options, their deprecations and the values they take. A version of JHipster is a patch of them, src/jhipster/versions.ts: the options, validations and field types the release does not have, has of its own, or has otherwise (its syntax, its values, paginate for the pagination option). The values of an option are the ones the release checks, else the ones it declares (jhipsterOptionValues, but a single value, a default), else the ones of the definitions. A deprecated field type of the definitions, Date and DateTime, is in every release: their generators migrate it to Instant. The semantic rules, src/jhipster/semantic-rules.ts, are the same for every version; so is the grammar, which accepts what the earlier ones did.

The patches are generated by npm run versions from definitions.ts and the definitions of each release, releases/<version>.json, committed: for the versions of generator-jhipster published to npm from generatorJhipster.firstVersion to generatorJhipster.defaultVersion of package.json, prereleases left out. A version without its file is installed in vendor/versions/<version> (a minute or so) and its definitions extracted by scripts/extract-version.mjs from its parser, where they were spread over the lexer tokens, the validator and lists of options. Run it after a change of definitions.ts, and when defaultVersion moves to a new release; a test (npm run versions -- --check, offline) fails when versions.ts is not what they make.

The conversion to files

astToFiles(ast, runtime) of jdl-parser/jhipster converts the AST of a jdl to the files of JHipster: the .yo-rc.json of each application and deployment, and the .jhipster/<Entity>.json of each entity, with the customizations of JHipster (an application named jhipster by default, blueprints named after their package, a relationship naming no field on either side bidirectional). config(jhipster) { … } is a catch-all for the options of an application the jdl does not define: they go to its generator-jhipster config unchecked; setting there an option of its config, or entities, is an error (namespace-config-conflict). It is lib/jdl/converters/ast-to-files of generator-jhipster but the compatibility defaults of its importer, which read the generators: { files, relativeRoot }, nothing written to the disk. Checked against generator-jhipster on the 88 jdl files of its repository: the same files. An entity lists the applications it is in, applications, as JHipster's import writes it: ["*"] for an entity of a jdl without application. The jdl itself, as written, and the files it gives are in #jdl of a .yo-rc.json at the root of the files that holds nothing else, { "source": "<the jdl>", "files": ["store/.yo-rc.json", "store/.jhipster/A.json"] }: the files tell what the jdl gives, not how it is written (its comments, constants, *, the order of its statements). A tool writing the files of a project leaves that file out; it serves to write the jdl back, and tells the files the jdl gave. filesToAst takes the jdl alone too, the text #jdl held before. A key starting with # is no generator namespace: no npm package name does.

The conversion back

filesToAst(files, runtime) of jdl-parser/jhipster converts the json files of JHipster back to a jdl, the reverse of astToFiles: the .yo-rc.json of an application, a deployment or a workspaces root, at the root or in a folder of it, and the .jhipster/<Entity>.json next to it. It returns { ast, jdl, diagnostics, errors, warnings }: the jdl the files are written as, its AST and its diagnostics (an option value out of its choices, for instance), and what the files hold that no jdl writes (an unknown key of an entity, a value with no jdl form, a workspaces root), as { file, path, value, message } errors, left out of the jdl, and what they hold in another form than JHipster writes it, which the jdl holds as JHipster writes it, as warnings: an integer option written as a text, "8081", is the number 8081, which astToFiles writes. The application options the jdl does not define, or deprecates (jhipsterVersion, which the generator stamps), go to config(jhipster); the applications of an entity are the entities statements of the applications. astToFiles gives the files back: checked on the jdl files of generator-jhipster. Every value of the files the jdl does not keep is an error, of an application, an entity or a deployment: the files the jdl gives back are compared with them, in the form JHipster reads (* and a list of applications, a validation value as a number or a text, blueprints by name or as { name }, any order of fields, relationships and entities), the values a more precise error tells left out.

With #jdl, the jdl the files were written from, the jdl is that one, as written, when the files are the ones it gives. When they changed since, the changes are merged into it, what the files give the same kept as written: an entity field by field, its header (javadoc, annotations, table) as a whole; an enum value by value; a relationship by the sides it gives its entities, one changed written again in place, one added after the others (the relationships written again as a whole when the files order them otherwise); an application by its configs and its entities statement (* giving an entity added since); a deployment as a whole. The option statements are kept while the files hold the options they give: one giving an entity an option its file no longer holds leaves it out, in its except for *, out of its list otherwise, and the options the files give otherwise come in statements of their own. What changed is written from the files, in place, what is new after. The merged jdl must give the files, else the jdl is written from them, with an error.

How it is built

The sources are in src/, with their history: src/parsing, src/printing, src/utils/format-utils.ts and the semantic rules of JHipster (now src/jhipster/semantic-rules.ts), then src/ast-to-files and the modules it uses (src/utils; the relationship types and the jdl metadata, moved into src/ast-to-files), were imported from generator-jhipster, its pull requests #35311 (the parsing behind an index) and #35198 (printJDL) applied together, with git filter-repo: git log --follow follows a file back into generator-jhipster. They are no longer copied from it. npm run build compiles src/ with tsc and copies the declaration files of the sources, which tsc does not emit.

A new release of JHipster is added by moving generatorJhipster.defaultVersion, then running npm run versions.

Development

npm install
npm run build
npm test

npm test runs the tests of test/ with vitest against dist/, through the exports of the package, and type checks test/types.ts as a consumer would.

Releasing

Releases are made by release-please from the conventional commits: it keeps a release pull request open, and merging it tags the version, creates the GitHub release and publishes the package to npm (.github/workflows/release-please.yml), through npm trusted publishing, with a provenance once the repository is public. Before 1.0.0, a feat bumps the minor version and a fix the patch one.

License

Apache License 2.0. The parser is the work of the JHipster authors: see NOTICE.