GitLab Query Language

The compiler is built in Rust[^1] and distributed as two packages:

• npm / WASM (@gitlab/query-language-rust): Used by GitLab's frontend for client-side query compilation.
• Ruby gem (gitlab_query_language): Used by GitLab's backend for server-side query processing.

Goals of this library include:

• Transpiling GLQL queries, including the YAML frontmatter block to a GraphQL response.
• Transforming the result of a GraphQL call using transformer functions.

npm / WASM

@gitlab/query-language-rust is an npm module that provides a simple interface for converting glql queries into other formats.

It wraps the WebAssembly build of the glql compiler, written in Rust.

Usage

Install the module:

npm install --save @gitlab/query-language-rust

Import and use the module:

<script type="module">
  import { glql } from '@gitlab/query-language-rust';

  let query = 'label != "backend" and author = currentUser() and weight = 1 and updated > today()';

  let graphql = await glql.compile(query, {
    group: 'gitlab-org',
    username: 'johnhope',
  });

  console.log(graphql);
</script>

Running the Browser Playground

Clone the repo, then install the dev dependencies, build the WASM target, and serve the npm/ directory:

npm install
cargo install wasm-pack
rustup target add wasm32-unknown-unknown
wasm-pack build --target web
npx wds -r npm/

Ruby Gem

The same Rust compiler is also available as a Ruby gem, providing native Ruby bindings for server-side use.

Installation

Add to your Gemfile:

gem 'gitlab_query_language'

Or install directly:

gem install gitlab_query_language

Usage

require 'gitlab_query_language'

query = 'label != "backend" and author = currentUser() and weight = 1'
result = Glql.compile(query, { group: 'gitlab-org', username: 'johnhope', fields: 'title' })
puts result

Interactive Console (CLI / REPL)

The Ruby gem provides a quick way to test GLQL queries from the command line without needing a browser:

cd glql_rb
bundle exec rake compile
bundle exec rake console

irb(main):001> Glql.compile('type = Issue', {group: 'gitlab-org', username: 'johnhope', fields: 'id'})
=>
{"fields" => [{"key" => "id", "label" => "ID", "name" => "id"}],
 "output" => "query GLQL(...) { ... }",
 "success" => true,
 "variables" => { ... }}

For full details on the Ruby gem, see glql_rb/README.md.

Development

Install mise (or asdf), then run:

mise install        # installs Rust, Node.js, and Ruby at the versions pinned in .tool-versions
brew install lefthook  # for managing git hooks (see below)

If using VSCode, consider installing the rust-analyzer extension.

Setting Up Lefthook

Lefthook is used to automate certain checks before pushing code to the repository, such as running tests, ensuring code is formatted, and checking for lint warnings.

To install Lefthook globally on MacOS, run:

brew install lefthook

For other platforms or methods to install, see installation docs for Lefthook.

Once lefthook is installed, enable it in your project by running the following command in the root of your repository:

lefthook install

This command will set up Lefthook to manage git hooks for pre-push checks as defined in the lefthook.yml file.

Pre-push Hooks

The following checks will run automatically before you push your code:

• Tests: Ensures all tests pass using cargo test.
• Clippy: Runs cargo clippy to check for lint warnings and ensures no warnings are present.
• Format: Ensures code is formatted properly using cargo fmt --check.
• Check: Runs cargo check to verify that the code compiles without errors.

Running Tests

Rust Tests

To run the Rust tests manually, use the following command:

cargo test

This will build and run all tests defined in your Rust project.

WASM Package Tests

To run the WASM package tests, use:

npm test

For continuous integration testing:

npm run test:ci

Code Coverage

Coverage reports are automatically generated in CI/CD and published to GitLab Pages at:

• Rust coverage: https://gitlab-org.gitlab.io/glql/coverage/

Formatting Code

To ensure your code is properly formatted using rustfmt, run:

cargo fmt -- --check

This command checks whether your code adheres to the formatting rules but does not modify any files. To automatically format your code, run:

cargo fmt

Linting with Clippy

To lint your code and catch common mistakes or inefficiencies, run:

cargo clippy -- -D warnings

This will treat all warnings as errors and prevent the code from building if any warnings are found.

End-to-End Testing

When making changes to the GLQL Rust codebase, you can follow these steps to test your modifications end-to-end in GitLab GDK. The frontend (npm/WASM) and backend (Ruby gem) both need to be rebuilt and pointed at your local checkout.

In the glql repo:

1. Check out the branch you want to test:

   git checkout my-feature-branch

2. Build the WASM package:

   wasm-pack build --target web --out-dir pkg

3. Build the npm package:

   npm install && npm run build

4. Build the Ruby gem:

   cd glql_rb && bundle install && bundle exec rake compile
   cd ..

5. Print the paths you'll need in the next step:

   echo "WASM path: $(pwd)/npm/dist/main.js"
   echo "Ruby gem path: $(pwd)/glql_rb"

In the GitLab GDK directory:

1. Update the import in app/assets/javascripts/glql/core/parser.js and app/assets/javascripts/glql/core/transformer.js:

   // change from
   import { glql } from '@gitlab/query-language-rust';
   
   // to
   import { glql } from 'file:///[WASM_PATH]';

2. Update the gitlab_query_language line in the root Gemfile to point at your local gem:

   gem 'gitlab_query_language', path: '[GEM_PATH]'

3. Reinstall gem dependencies and restart the affected services so both the Rails backend and Vite frontend pick up the local builds:

   bundle install
   gdk stop rails-web vite && rm -rf tmp/cache/vite && gdk start rails-web vite

Now you can test your changes in any markdown field by embedding a GLQL query, for example:

```glql
display: table
fields: title, labels, author
query: group = "gitlab-org" and label = "group::knowledge" and user = currentUser()
```

Note: Every time you change the GLQL Rust code, you must rebuild the WASM, npm, and Ruby gem artifacts for the changes to take effect.

Contributing

1. If you don't already have access, start onboarding by requesting access to the community forks.
2. Clone the community fork repository.
3. Create a new feature branch (git checkout -b feature/my-feature).
4. Commit your changes (git commit -am 'Add new feature').
5. Push to the branch (git push origin feature/my-feature).
6. Open an MR.

Toolchain version sync

Runtime versions are pinned in three places that must stay in sync:

| File | What it controls | | ------------------------------ | ------------------------------------------------------------- | | .tool-versions | Local dev environment (mise / asdf) | | .gitlab-ci.yml | CI pipeline images and variables (RUST_VERSION, image tags) | | .gitlab/duo/agent-config.yml | Duo Developer agent environment |

When bumping a Rust, Node.js, or Ruby version, update all three files.

Release process

[!important] Pre-requisites You need Developer permissions in the project.

On the main branch, run any of the below pipeline jobs to create a release:

• release-patch: Bump patch version and publish to NPM.
• release-minor: Bump minor version and publish to NPM.
• release-major: Bump major version and publish to NPM.

The release-* job will additionally create a release MR to also bump the Ruby gem version and dependency. This currently requires manual approval to trigger the gem release.

See the gem glql_rb/README.md for more details on the release process.

Additional Reading

Learning Rust

• The Rust book
• Rust by Example

Learning more about GLQL

• Architecture Design Document
• User facing docs

[^1]: Originally written in Haskell.