tree-sitter-sql-polyglot

A multi-dialect SQL parser for tree-sitter. It provides an ANSI SQL base plus 22 independently compiled dialect grammars, each layered on top with tree-sitter's grammar(parent, overrides) composition.

The grammar is a fork of DerekStride/tree-sitter-sql. Upstream ships a single "permissive" grammar that mixes several dialects together. This fork splits that into a strict ANSI base and one grammar per dialect, so each engine's syntax is parsed on its own terms.

Docs site and dialect coverage, which carries the per-dialect feature scores and is regenerated from the live parsers on every push to main.

Dialects

Each dialect compiles to its own <dialect>/src/parser.c and can be used independently.

| Dialect | Extends | Highlights | |---------|---------|-----------| | base (ANSI) | none | GRANT/REVOKE, GROUP BY ROLLUP/CUBE/GROUPING SETS, FETCH FIRST/OFFSET ... FETCH, WITHIN GROUP, TRIM(... FROM ...), interval qualifiers | | hana | base | CREATE COLUMN/ROW TABLE, UPSERT ... WITH PRIMARY KEY, WITH HINT (...), SQLScript procedures (LANGUAGE SQLSCRIPT, DECLARE, :=, :param) | | hive | base | LATERAL VIEW, STORED AS/STORED BY, multi-table INSERT, LOAD DATA INPATH, CLUSTER/DISTRIBUTE/SORT BY | | spark | hive | QUALIFY, PIVOT/UNPIVOT, time travel, scripting (IF/WHILE/LOOP), Iceberg, VARIANT, CREATE TABLE ... USING/OPTIONS | | databricks | spark | Delta/DLT (OPTIMIZE ... ZORDER BY, VACUUM, RESTORE), Unity Catalog (CATALOG/VOLUME/EXTERNAL LOCATION, GRANT), Iceberg CALL | | postgres | base | COPY, VACUUM, PARTITION BY/PARTITION OF, CREATE TABLE (LIKE ...), INHERITS, extensions, RLS policies, :: cast | | mysql | base | ENGINE=/CHARSET=, index hints, SHOW, DESCRIBE, LIMIT offset, count, @/@@ variables | | mariadb | mysql | INVISIBLE columns (plus inherited MySQL features) | | oracle | base | CONNECT BY, PL/SQL blocks, packages, cursors, FORALL, BULK COLLECT, numeric FOR ... IN 1..10 | | db2 | base | SQL PL (BEGIN...END, IF/WHILE/LOOP, LEAVE/ITERATE), modules, audit policies, federated objects | | tsql | base | T-SQL scripting, CROSS/OUTER APPLY, query hints, #temp/##global identifiers | | bigquery | base | INT64/STRUCT<...>/ARRAY<...> types, UNNEST, backtick identifiers, QUALIFY | | snowflake | base | scripting, LATERAL FLATTEN, time travel, @stage sources, :: cast | | sqlite | base | INSERT OR REPLACE/IGNORE, UPSERT, AUTOINCREMENT, INDEXED BY | | spanner | bigquery | trailing PRIMARY KEY, INTERLEAVE IN PARENT ... ON DELETE CASCADE, NULL_FILTERED/STORING indexes, CREATE CHANGE STREAM, ROW DELETION POLICY, STRING(n\|MAX)/BYTES(n\|MAX) | | duckdb | base | FROM-first SELECT, SELECT * EXCLUDE/REPLACE/RENAME, lambdas, struct/map/list literals, ASOF/POSITIONAL JOIN, ATTACH | | teradata | base | SEL/DEL abbreviations, SET/MULTISET/VOLATILE tables, [UNIQUE] PRIMARY INDEX/NO PRIMARY INDEX, PARTITION BY RANGE_N/CASE_N, COLLECT STATISTICS, CREATE MACRO, TOP n, QUALIFY, :param references | | trino | base | PREPARE/EXECUTE/DEALLOCATE, MATCH_RECOGNIZE, TABLESAMPLE BERNOULLI/SYSTEM, ARRAY/MAP/ROW types, lambdas | | athena | trino | UNLOAD ... TO 's3://...', MSCK REPAIR TABLE ... PARTITIONS (managed Trino plus data-lake semantics) | | redshift | base | DISTKEY/SORTKEY/DISTSTYLE/ENCODE, CREATE EXTERNAL SCHEMA/TABLE, COPY/UNLOAD, VACUUM REINDEX, APPROXIMATE COUNT | | cockroachdb | postgres | AS OF SYSTEM TIME, UPSERT INTO, BACKUP/RESTORE, IMPORT INTO ... CSV DATA, CREATE CHANGEFEED, hash-sharded indexes (USING HASH), STORING (...), SHOW JOBS/GRANTS/DATABASES | | clickhouse | base | ENGINE = MergeTree() ..., column MATERIALIZED/ALIAS/EPHEMERAL/CODEC/TTL, PREWHERE, FINAL, ARRAY JOIN, LIMIT n BY, SAMPLE, WITH TOTALS, QUALIFY, ORDER BY ... WITH FILL, LIMIT ... WITH TIES, INTO OUTFILE/FORMAT, ALTER ... UPDATE/DELETE, OPTIMIZE ... FINAL, CREATE DICTIONARY/LIVE VIEW, SYSTEM ..., Map/Tuple/Nested/LowCardinality/Nullable types | | flink | base | connector DDL (WITH (...)), WATERMARK FOR, windowing TVFs (TUMBLE/HOP/CUMULATE), MATCH_RECOGNIZE, temporal joins, CREATE CATALOG, LOAD/UNLOAD MODULE, statement sets |

Dependency chains: databricks -> spark -> hive -> base, mariadb -> mysql -> base, athena -> trino -> base, cockroachdb -> postgres -> base, and spanner -> bigquery -> base. The chains follow real dialect genealogy: CockroachDB is PostgreSQL-compatible by design, and Spanner and BigQuery share GoogleSQL. Regenerate the child when a parent grammar changes. See AGENTS.md for the full architecture.

Installation

Every dialect is compiled/loaded lazily in all six: importing or depending on the package never pulls in more than the base grammar until you actually ask for a specific dialect.

cargo add tree-sitter-sql-polyglot --features postgres   # or: --features full (all 22)
npm install @relativelyunknown/tree-sitter-sql-polyglot
pip install tree-sitter-sql-polyglot
go get github.com/relativelyunknown/tree-sitter-sql-polyglot/bindings/go/postgres
# Swift: add https://github.com/RelativelyUnknown/Tree-sitter-sql-polyglot as a package dependency
# CMake: cmake -B build -DTREE_SITTER_SQL_POSTGRES=ON && cmake --build build   (or no flag for base)

use tree_sitter_sql_polyglot::{LANGUAGE, LANGUAGE_POSTGRES};

let mut parser = tree_sitter::Parser::new();
parser.set_language(&LANGUAGE.into()).unwrap();            // base ANSI grammar
// parser.set_language(&LANGUAGE_POSTGRES.into()).unwrap(); // needs features = ["postgres"] or "full"

Each dialect's identifier (postgres, databricks, cockroachdb, ...) is the same everywhere it appears: Cargo feature, npm/Python/Go/Swift name. See the Usage page for full import examples in every language, the lazy-loading mechanism per binding, and the complete identifier reference.

Development

Prerequisites

npm install -g tree-sitter-cli

Workflow

# Regenerate the base parser after editing grammar.js or grammar/**/*.js
npm run generate

# Regenerate a single dialect (and its parent chain as needed)
npm run generate:spark

# Regenerate every parser (base + all 22 dialects)
npm run generate:all

# Run corpus tests for the base grammar
npm run test:corpus

# Run corpus tests for a specific dialect
npm run test:corpus:spark

# Check that base keywords are in sync with queries/highlights.scm
npm run test:keywords

Generation is hash-cached: npm run generate* skips tree-sitter generate when the relevant grammar sources are unchanged. Use npm run generate:force to bypass the cache.

Base grammar rules are split across grammar/ (e.g. grammar/statements/*.js, grammar/expressions.js, grammar/keywords.js). Dialect rules live under <dialect>/grammar/. A change to the base ripples to all 22 parsers, so regenerate and test all of them after editing base files.

See CONTRIBUTING.md and AGENTS.md for more detail.

References

• Wikipedia SQL syntax
• Databricks SQL reference
• Apache Spark SQL reference
• Apache Hive language manual
• Unity Catalog SQL reference
• Apache Iceberg Spark procedures
• PostgreSQL syntax
• MySQL reference manual
• MariaDB SQL statements
• Oracle PL/SQL language reference
• IBM Db2 SQL reference
• Microsoft T-SQL reference
• BigQuery SQL reference
• Snowflake SQL reference
• SQLite SQL syntax

Other SQL tree-sitter grammars

• DerekStride/tree-sitter-sql: upstream
• takegue/tree-sitter-sql-bigquery: BigQuery fork
• m-novikov/tree-sitter-sql

Fork history & attribution

This repo preserves the full git history of DerekStride/tree-sitter-sql, so GitHub's Contributors graph includes upstream authors alongside this fork's own. Current maintainership is CODEOWNERS. LICENSE carries both the original 2021 copyright notice and this fork's. General extensions are worth sending upstream if the maintainers there want them; vendor-specific ones stay here.