SQLAnvil

SQL workflow tool for BigQuery, Postgres, Supabase, and MySQL/MariaDB.

SQLAnvil is an open-source fork of Dataform OSS (Apache 2.0), extended with first-class PostgreSQL and Supabase support, plus adapters for other data sources. Define your data transformations in SQLX, have SQLAnvil compile them to idiomatic SQL, and run them against your warehouse.

SQLAnvil is not affiliated with or endorsed by Google. The Dataform name and related marks are trademarks of Google LLC. See NOTICE for attribution.

Features

• BigQuery — full support: partitioning, clustering, labels, materialized views, MERGE-based incremental upserts
• PostgreSQL — idiomatic DDL: native partitioning, INSERT ... ON CONFLICT upserts, btree/gin/gist/brin indexes, tablespaces, fillfactor
• Supabase — extends Postgres with RLS policies, Realtime publications, pgvector indexes, and Supabase Wrappers (coming soon)
• MySQL / MariaDB — portable MySQL DDL: CTAS tables, CREATE OR REPLACE VIEW, ON DUPLICATE KEY UPDATE incremental upserts (one adapter, validated against both engines)
• SQLX + YAML + JS — three authoring modes: SQL with config blocks, actions.yaml bulk definitions, or the JavaScript API

Quick start

npm install -g @sqlanvil/cli
sqlanvil init my-project
cd my-project
# edit workflow_settings.yaml to configure your warehouse
sqlanvil compile
sqlanvil run

workflow_settings.yaml example (Postgres):

warehouse:
  kind: postgres
  host: localhost
  port: 5432
  database: analytics
  user: sqlanvil_writer
  password: ${PG_PASSWORD}
  ssl: disable
  defaultSchema: public

First action (definitions/my_view.sqlx):

config {
  type: "view",
  description: "My first SQLAnvil view."
}

SELECT 1 AS id, 'hello' AS greeting

Documentation

Full documentation at sqlanvil.com.

• Getting Started
• BigQuery Guide
• PostgreSQL Guide
• Supabase Guide
• Configs Reference

Project layout

core/         Compiler + action types (table/view/incremental/assertion/operation/notebook/declaration)
cli/          CLI entrypoint and per-adapter glue (cli/api/dbadapters/)
protos/       Protobuf definitions for core/configs/execution/db_adapter
examples/     Sample SQLAnvil projects
scripts/      ./scripts/run is the CLI entrypoint wrapper

Building from source

SQLAnvil uses Bazel (via Bazelisk). The recommended dev path on macOS is the Docker-based build container:

# Build the proto layer
./scripts/docker-bazel build //protos:sqlanvil_proto

# Run the CLI
./scripts/docker-bazel run //packages/@sqlanvil/cli:bin -- help

# Run tests
./scripts/docker-bazel test //...

See contributing.md for full instructions.

Documentation

Reference docs and design documents live in the separate sqlanvil/docs repo, published at sqlanvil.com/docs.

License

Apache License 2.0. See LICENSE and NOTICE.

SQLAnvil is a derivative of Dataform OSS (originally developed by Dataform Co and contributed to by Google LLC). It has been fully renamed and extended with first-class PostgreSQL/Supabase support. Licensed under Apache License 2.0.