bun-jsonl

A lightweight, type-safe JSONL (JSON Lines) file processor for Bun runtime that streams large files efficiently.

Features

• 🚀 Fast streaming: Uses Bun's native file API for optimal performance
• 💾 Memory efficient: Processes files line-by-line without loading entire file into memory
• 🔒 Type safe: Full TypeScript support with generics
• 🔄 Async/sync support: Works with both synchronous and asynchronous processor functions
• 🛡️ Error handling: Detailed error messages with line numbers
• 📦 Zero dependencies: Built specifically for Bun runtime

Installation

bun add bun-jsonl

Usage

Basic Example

import { processJsonl } from 'bun-jsonl';

// Process each line with a simple function
await processJsonl('data.jsonl', (data) => {
  console.log('Processed:', data);
  // Return anything to continue, bail to stop
});

With Type Safety

import { processJsonl } from 'bun-jsonl';

interface User {
  id: number;
  name: string;
  email: string;
}

await processJsonl<User>('users.jsonl', (user) => {
  console.log(`User: ${user.name} (${user.email})`);
});

Async Processing

import { processJsonl } from 'bun-jsonl';

await processJsonl('data.jsonl', async (data) => {
  // Simulate async processing (API calls, database operations, etc.)
  await fetch('/api/process', {
    method: 'POST',
    body: JSON.stringify(data)
  });
});

Error Handling

import { processJsonl } from 'bun-jsonl';

try {
  await processJsonl('data.jsonl', (data) => {
    if (!data.id) {
      throw new Error('Missing required ID field');
    }
    // Process valid data
    return processData(data);
  });
} catch (error) {
  console.error('Processing failed:', error);
  // Error includes line number for JSON parsing errors
}

Early Termination

import { processJsonl } from 'bun-jsonl';

// Stop processing after finding a specific item
await processJsonl('data.jsonl', (data, bail) => {
  console.log('Processing:', data);
  
  if (data.id === 'target-id') {
    console.log('Found target, stopping...');
    return bail; // Stop processing
  }
  
  // Continue processing (return anything else or nothing)
});

Filtering and Transformation

import { processJsonl } from 'bun-jsonl';

interface LogEntry {
  level: 'info' | 'warn' | 'error';
  message: string;
  timestamp: string;
}

await processJsonl<LogEntry>('logs.jsonl', (entry) => {
  // Only process error logs
  if (entry.level === 'error') {
    console.error(`[${entry.timestamp}] ${entry.message}`);
  }
  // Continue processing all entries
});

API Reference

processJsonl<T>(filePath, fn)

Processes a JSONL file line by line, calling the provided processor function for each parsed JSON object.

Parameters

• filePath (string): Path to the JSONL file
• fn ((data: T, bail?) => MaybePromise<unknown>): Function to process each parsed JSON object
  • Can be synchronous or asynchronous
  • Receives the parsed JSON object as first parameter
  • Receives optional bail value as second parameter
  • Return bail to stop processing, anything else to continue

Returns

• Promise<void>: Resolves when all lines have been processed or fn returns bail

Throws

• Error: If file doesn't exist
• Error: If JSON parsing fails (includes line number in error message)
• Error: If fn function throws an error

Exports

• processJsonl: Main processing function

JSONL Format

JSONL (JSON Lines) is a text format where each line is a valid JSON object:

{"id": 1, "name": "Alice", "email": "[email protected]"}
{"id": 2, "name": "Bob", "email": "[email protected]"}
{"id": 3, "name": "Charlie", "email": "[email protected]"}

Performance

This library is optimized for processing large JSONL files:

• ✅ Streams data instead of loading entire file into memory
• ✅ Uses Bun's native file API for maximum performance
• ✅ Processes lines as they're read from disk
• ✅ Handles backpressure automatically

Requirements

• Bun runtime (v1.0+)
• This package is specifically designed for Bun and won't work with Node.js

Contributing

1. Fork the repository
2. Create a feature branch: git checkout -b feature-name
3. Make your changes and add tests
4. Run tests: bun test
5. Submit a pull request

License

MIT

Related

• JSONL Format Specification
• Bun Documentation