corpus

a functional snapshotting library for typescript. store versioned data with lineage tracking, content deduplication, and multiple backend support (memory, file, cloudflare d1/r2).

install

bun add corpus

usage

import { z } from 'zod'
import { create_corpus, create_memory_backend, define_store, json_codec } from 'corpus'

const TimelineSchema = z.object({
  items: z.array(z.object({ id: z.string(), text: z.string() })),
})

const corpus = create_corpus()
  .with_backend(create_memory_backend())
  .with_store(define_store('timelines', json_codec(TimelineSchema)))
  .build()

// typed store access - version is auto-generated
const result = await corpus.stores.timelines.put({ 
  items: [{ id: '1', text: 'hello' }] 
})

if (result.ok) {
  console.log('saved:', result.value.content_hash)
}

todo

• [ ] add gzip compression codec wrapper for large json blobs
• [ ] add encryption codec wrapper for sensitive data at rest
• [ ] implement ttl/expiration support with auto-cleanup
• [ ] add batch operations (put_many, get_many) for bulk imports
• [ ] create drizzle migration files for d1 schema setup
• [ ] add diff(v1, v2) function for comparing json snapshots
• [ ] implement data compaction (merge old versions)
• [ ] add rate limiting awareness for cloudflare api limits
• [ ] create test utilities module with helpers (create_test_corpus, seed_test_data)
• [ ] add signed url support for direct r2 access to large files
• [ ] implement garbage collection for orphaned data blobs
• [ ] add retry logic with exponential backoff for network failures