@prairielearn/csv

A few helpful wrappers on top of the functionality from csv-stringify.

Usage

Here's an example taking data from @prairielearn/postgres#queryCursor() and writing it to a file, though this should be applicable to any source and destination streams:

import { stringifyStream } from '@prairielearn/csv';
import { queryCursor } from '@prairielearn/postgres';
import { pipeline } from 'node:stream/promises';
import { createWriteStream } from 'node:fs';
import { IdSchema } from '@prairielearn/zod';
import { z } from 'zod';

const WorkspaceSchema = z.object({
  id: IdSchema,
  hostname: z.string(),
});
type Workspace = z.infer<typeof WorkspaceSchema>;

const cursor = await queryCursor('SELECT id, hostname FROM workspaces;', WorkspaceSchema);
const output = createWriteStream('workspaces.csv');

const stringifier = stringifyStream<Workspace>({
  header: true,
  columns: [
    { key: 'id', header: 'ID' },
    { key: 'hostname', header: 'Hostname' },
  ],
  // Optionally provide a function to transform each item in the stream.
  transform(record) {
    return {
      id: `workspace-${record.id}`,
      hostname: record.hostname,
    };
  },
});

// Fetch one row at a time with cursor.stream(1) for large records.
await pipeline(cursor.stream(100), stringifier, output);

stringifyStream respects backpressure and processes asynchronous transformations one at a time. For large records, also limit buffering in the source stream.

Note that this works best when the source stream is producing data asynchronously, such as though an async iterator. If you use a synchronous data source like Readable.from([...]), the conversion will still occur synchronously. If you have a large array of data in memory and want to convert it to a CSV, you can use stringifyNonblocking:

import { stringifyNonblocking } from '@prairielearn/csv';
import { createWriteStream } from 'node:fs';

const data = Array.from(new Array(100_000), (_, i) => ({ id: i }));
const output = createWriteStream('numbers.csv');
stringifyNonblocking(data, {
  header: true,
  columns: [{ key: 'id', header: 'ID' }],
}).pipe(output);

For lower-level usage, stringify and Stringifier are also re-exported from csv-stringify:

import { stringify, Stringifier } from '@prairielearn/csv';