Strime

A streaming JSON projection engine. Selects and extracts fields from JSON without parsing the entire structure.

Why Strime

Most JSON parsers convert the entire input to objects before you can query it. Strime inverts this: it filters during traversal, touching only the bytes you need. The result is constant memory usage regardless of file size—a 10GB file uses the same baseline memory as a 10KB file.

This isn't a marginal improvement. On high-volume workloads, Strime processes JSON at throughputs typically reserved for native implementations.

Performance

| Metric | Result | Guarantee | | ------------------------ | -------------- | -------------- | | Throughput (1GB) | 809-939 Mbps | O(N) time | | Skip-heavy (chunked) | 4,408 Mbps | 6.5x faster | | Memory | Constant | O(D) space | | 1M NDJSON rows | ~4.2s | 220k+ rows/s | | Deep nesting (1k levels) | < 1ms | Stack-safe | | Large strings (50MB) | 1.6 Gbps | No degradation |

Validated on 1GB+ files. See Performance Contract for methodology.

Install

npm install @laphilosophia/strime

Use

Query

import { query } from '@laphilosophia/strime'

const result = await query(data, '{ id, user { name, email } }')

Stream NDJSON

import { ndjsonStream } from '@laphilosophia/strime'

for await (const row of ndjsonStream(stream, '{ id, name }')) {
  console.log(row)
}

Fault-Tolerant

for await (const row of ndjsonStream(stream, '{ id, name }', {
  skipErrors: true,
  onError: (info) => console.error(`Line ${info.lineNumber}: ${info.error.message}`),
})) {
  console.log(row)
}

Real-Time Subscribe

import { subscribe } from '@laphilosophia/strime'

subscribe(telemetryStream, '{ lat, lon }', {
  onMatch: (data) => console.log(data),
  onComplete: () => console.log('Done'),
})

CLI

strime data.json "{ name, meta { type } }"
cat massive.json | strime "{ actor { login } }"
tail -f telemetry.log | strime --ndjson "{ lat, lon }"

Features

Core

• Zero-allocation tokenizer with object recycling
• String interning for repeated keys
• Integer fast-path parsing
• Binary line splitting for NDJSON

API

• Streaming and pull modes
• Parallel NDJSON processing (ndjsonParallel)
• Raw byte emission for zero-copy pipelines
• Compression sink (gzip/brotli)
• Budget enforcement for DoS protection

Error Handling

• Typed errors with position tracking
• Fault-tolerant NDJSON streaming
• Controlled termination via AbortSignal

Advanced

Chunked Execution (Large Files)

For maximum throughput on large single-buffer inputs:

import { Engine } from '@laphilosophia/strime'
import { StrimeParser } from '@laphilosophia/strime'
import { readFileSync } from 'fs'

const buffer = readFileSync('large-file.json')
const schema = new StrimeParser('{ id, name }').parse()
const engine = new Engine(schema)

// 6.5x faster on skip-heavy workloads
const result = engine.executeChunked(buffer) // 64KB chunks (default)
const result = engine.executeChunked(buffer, 32768) // Custom 32KB chunks

Low-Level Tokenizer

import { Tokenizer } from '@laphilosophia/strime'

const tokenizer = new Tokenizer()
const buffer = new TextEncoder().encode('{"key": "value"}')

// Zero-allocation callback mode
tokenizer.processChunk(buffer, (token) => {
  console.log(token.type, token.value)
})

// Iterator mode
for (const token of tokenizer.tokenize(buffer)) {
  console.log(token.type, token.value)
}

Telemetry

await query(stream, '{ id, name }', {
  sink: {
    onStats: (stats) => {
      console.log(`Throughput: ${stats.throughputMbps.toFixed(2)} Mbps`)
      console.log(`Skip ratio: ${(stats.skipRatio * 100).toFixed(1)}%`)
    },
  },
})

Raw Emission

await query(stream, '{ items { id } }', {
  emitMode: 'raw',
  sink: {
    onRawMatch: (bytes) => outputStream.write(bytes),
  },
})

Documentation

• Quick Start — Getting started
• Query Language — Syntax reference
• API Reference — Complete API
• CLI Guide — Command-line usage
• Performance — Benchmarks and guarantees
• Internals — How it works

Licensing

Strime is released under the Business Source License (BSL).

• ✅ Source code is fully available and auditable
• ✅ Free for learning, experimentation, and open-source projects
• ❌ Commercial and hosted use requires a commercial license

After the Change Date (see LICENSE file), Strime Core will automatically transition to the Apache 2.0 License.

Why this model?

Strime is a performance-critical infrastructure component. The BSL model allows transparent development while protecting early-stage sustainability, with a commitment to full open-source availability long-term.

For commercial licensing, please contact us.

Trademark Notice

"Strime" and the Strime logo are trademarks of the Licensor. Forks and derivative works must not use the Strime name or branding in a way that suggests official endorsement.

Contact

Erdem Arslan [email protected]