datasette-sql-to-csv

Download all of the results of a Datasette SQL query into a CSV file.

Usage

$ datasette-sql-to-csv <url> -o data.csv

Options

url The Datasette URL for the first page of resulting data, required

--limit, -l The number of rows per page, default is 1,000 --output, -o The output CSV file, required

Motivation

Datasette is an excellent tool that I use all the time. It let's me explore my data with queries represented by stable URLs. And its sidekick sqlite-utils makes it easy to turn data from elsewhere into a SQLite database.

And if you use the Datasette interface to build queries (instead of writing a custom SQL query) you can download all of your data as a single CSV.

But if you write a custom SQL query, you'll end up with a page that doesn't let you download all of the matching records, just the first n (by default 1,000) records.

You can paginate through all of the records in the query using limit and offset but you can't just click a button and get a spreadsheet.

datasette-sql-to-csv takes the URL for the first page of resulting data, makes a series of requests to get all the pages, and saves the data to a local CSV file.