@doync/sqlite-parser

A native TypeScript SQLite SQL parser fully compatible with SQLite's own parser.

It parses any SQLite SQL into a typed, visitable, JSON-serializable AST – similar to the one generated by liteparser, as the project began as a TypeScript rewrite of it – but has since evolved past it. SQLite itself is the only upstream: the parser's grammar is woven from SQLite's own parse.y, and fidelity is enforced by a generated grammar corpus carrying acceptance verdicts baked from real SQLite's prepare() with anchored structural matchers, two seeded fuzzers (grammar-walk and tokenizer-lexeme), a dedicated tokenizer edge-case suite, and a clean-parse ratchet plus self-round-trip invariant over SQLite's own 22,888-statement test corpus.

• Free functions, fully synchronous
• Zero runtime dependencies, ESM + CJS, 100% tree-shakable
• Complete SQLite grammar — SELECT (compounds, CTEs, window functions), INSERT/UPSERT, UPDATE, DELETE, RETURNING, CREATE TABLE/INDEX/VIEW/TRIGGER/VIRTUAL TABLE, ALTER, DROP, PRAGMA, transactions, savepoints, ATTACH/DETACH, VACUUM/REINDEX/ANALYZE, EXPLAIN.
• Faithful to SQLite — the grammar is SQLite's own Lemon grammar with the Lemon generator retargeted to emit TypeScript, and the tokenizer is a hand-port of SQLite's.

Playground

Try the parser in the browser at closeio.github.io/doync — type SQL on one side and see its AST on the other, or edit the AST and watch it unparse back to SQL.

To run it locally against your own build (playground.html imports ./dist/index.js, so it reflects the last pnpm build):

pnpx local-web-server -d packages/sqlite-parser --static.index playground.html
# → http://localhost:8000

Installation

pnpm add @doync/sqlite-parser

Parsing

import {
  parse,
  parseAll,
  parseTolerant,
  SQLiteParserError,
} from '@doync/sqlite-parser'

// One statement -> typed AST node. Throws SQLiteParserError on invalid SQL.
const stmt = parse('SELECT a, b FROM t WHERE a > 5')
stmt.kind // 'STMT_SELECT'

// A semicolon-separated script -> all statements.
const stmts = parseAll('CREATE TABLE t (a); INSERT INTO t VALUES (1);')

// Tolerant / IDE mode: never throws, recovers at `;` boundaries.
const { stmts, errors } = parseTolerant('SELECT 1; GARBAGE; SELECT 2')
errors[0].code // 'syntax' | 'illegal_token' | 'incomplete' | 'stack_overflow'
errors[0].pos // { offset, line, col } — byte-exact source range start

try {
  parse('SELECT FROM')
} catch (e) {
  if (e instanceof SQLiteParserError) {
    e.code // 'syntax'
    e.pos // start of the offending token
    e.end_pos // end of the error range (exclusive)
  }
}

The AST is a discriminated union keyed by kind, so TypeScript narrows node types inside switches and handlers:

import type { Statement } from '@doync/sqlite-parser'

function whereOf(stmt: Statement) {
  if (stmt.kind === 'STMT_SELECT') return stmt.where // typed as Expr | undefined
}

Nodes carry byte-exact source positions (pos: { offset, line, col }), and optional fields are simply absent.

SQL output

import { parse, unparse } from '@doync/sqlite-parser'

const stmt = parse('SELECT a FROM t')

unparse(stmt) // 'SELECT a FROM t' — SQL text back from any AST node

The round-trip invariant — parse → unparse → parse yields a structurally equal AST (enforced over the whole corpus in tests).

The AST is plain data — JSON.stringify(stmt) serializes it (the in-memory node is the serialized shape). Key order is builder-insertion order and non-contractual, so compare structurally, never by string.

Traversal

import { parse, walk, Visit } from '@doync/sqlite-parser'

const stmt = parse(
  'WITH x AS (SELECT a FROM t) SELECT * FROM x JOIN u ON x.a = u.a',
)

// Pre-order walk with per-kind handlers; each handler gets its node narrowed.
const tables: string[] = []
walk(stmt, {
  FROM_TABLE(node) {
    tables.push(node.name ?? '') // node: FromTableNode
  },
  EXPR_SUBQUERY() {
    return Visit.Skip // don't descend into this subtree
  },
  '*'(node, ancestors) {
    // runs for every node; `ancestors` is a stable root-first snapshot,
    // e.g. for finding an enclosing WITH scope
  },
})

• Return Visit.Skip to prune a subtree, Visit.Stop to end the walk.
• childNodes(node) returns a node's direct children generically (and throws loudly if a future node shape drifts, instead of silently skipping).
• match(node, handlers, fallback) is typed single-level dispatch that returns a value — the building block for lowering the AST to your own IR:

import { match, parse, type Expr } from '@doync/sqlite-parser'

const select = parse('SELECT * FROM t WHERE a > 5')
const where = select.kind === 'STMT_SELECT' ? select.where : undefined

const lowered =
  where &&
  match<Expr, string>(
    where,
    {
      EXPR_COLUMN_REF: (n) => n.column ?? '?',
      EXPR_BINARY_OP: (n) => `(${n.op})`,
    },
    (n) => `unsupported: ${n.kind}`, // every unhandled kind routes here
  )

• assertNever(value) is the exhaustiveness guard for switch (node.kind): unhandled kinds become compile errors.

Fidelity notes

Fidelity means equivalence to real SQLite's own parser, at the exact version node:sqlite bundles (anchored by the repo's .nvmrc; currently SQLite 3.53.3): the parser accepts what SQLite accepts and rejects what SQLite rejects for everything decidable from the SQL text alone — including parse-time reduce-action rejections such as non-constant column DEFAULTs, unknown join types, or ORDER BY before a compound operator — with no divergence allowlist. A disagreement with SQLite in either direction is a bug.

Two prepare-time rejection families are deliberately not mirrored because they are schema-gated — whether SQLite rejects depends on what names resolve to, which a pure-syntax parser cannot know: RAISE(...) outside a trigger body, and qualified table names in trigger-body DML (rejected by SQLite only when the trigger resolves to a non-TEMP schema). Both sit on the safe side: the parser accepts, exactly as SQLite does when the schema resolves the other way.

Three SQLite parser quirks are reproduced on purpose (the test suites enforce them): TRUE/FALSE parse as column references — with the nuance that a quoted "true"/"false" carries quoted: true and stays a non-constant column ref in DEFAULT, mirroring SQLite's EP_Quoted gate; VALUES (...) lowers into a SELECT; and index / primary-key columns are ORDER_TERM nodes.

Versioning

@doync/sqlite-parser versions as 0.<SQLITE_VERSION_NUMBER>.<counter>, where the middle component is SQLite's own canonical numeric encoding of the fidelity target. SQLite 3.53.3 → 3053003, so 0.3053003.0 means "parses what SQLite 3.53.3 parses." The target is the node:sqlite build anchored by the repo's .nvmrc.

The last component is this package's own release counter. By convention the parser only ever receives patch changesets — a minor or major bump would corrupt the encoded SQLite version (0.3053003.x → 0.3053004.0 would falsely claim SQLite 3.53.4). Retargeting SQLite is a manual version edit in package.json in the same PR that moves the target; the middle component is a floor, not a signal of this package's own feature growth (read the changelog for that). The full retargeting workflow is documented in Upgrading SQLite.

/internal

@doync/sqlite-parser/internal exists so every publishable package exposes the same subpath contract. The current public surface already stands on the main entry; the internal entry is a placeholder. Anything under /internal may change in any release, including patches, with no notice. Import parse / unparse / the AST from @doync/sqlite-parser.

Contributions

Issues and pull requests welcome on closeio/doync. See the root README for install, lint, typecheck, and remaining contribution notes.