bigquery-builder-ts
v0.1.3
Published
Specify and materialise BigQuery models. Link models together in a dependency chain/DAG. All in TypeScript.
Downloads
15
Readme
bigquery-builder-ts
Orchestrates your data pipeline (model building). Lightweight and powerful.
Can only be used with BigQuery. No CLI provided, it's a TypeScript library so your interface is code. Build your own CLI on top if you like!
Very much a work in progress right now!
Implemented:
- "Full Refresh" and "External" model types
Planned:
- Ability to specify table clustering key/s
- Cost tracking (log run information to BigQuery table and ability to cap spend)
Considering:
- "Incremental" model type
- Ability to specify table partitioning
Running tests
Start up a bigquery emulator like so:
docker run -p 9050:9050 -it ghcr.io/goccy/bigquery-emulator:latest --project=local-test-project