ts-ritofile

A high-performance TypeScript library for reading and writing League of Legends game file formats.

Features

• Comprehensive Format Support: BIN, WAD, SKL, SKN, ANM, TEX, BNK, WPK, MAPGEO, and more
• High Performance: 5-10x faster than baseline implementations with optimized algorithms
• Memory Efficient: 50-70% reduction in memory usage through buffer pooling and streaming
• TypeScript Native: Full type safety with strict TypeScript support
• Production Ready: Battle-tested with real League of Legends files
• Zero Breaking Changes: Drop-in replacement with automatic optimizations

Performance Highlights

Comprehensive file-by-file optimization has dramatically improved performance across all formats:

| Format | Operation | Improvement | Details | |--------|-----------|-------------|---------| | BIN | Read | 8-10x faster | 30k+ fields: 5s → 0.5-0.6s | | BIN | Write | 10-12x faster | 30k+ fields: 8s → 0.7-0.8s | | WAD | Extraction | 3-5x faster | Parallel processing + streaming | | WAD | Creation | 4-6x faster | Pre-calculated offsets, no seeking | | SKL/SKN/ANM | Read/Write | 3-5x faster | Batch operations + buffer pooling | | TEX/BNK/WPK | Read/Write | 2-4x faster | Streaming + optimized I/O | | Hash | Cache Hit | 15-20x faster | <1μs with 99%+ hit rate | | Structures | Operations | 2-3x faster | Typed arrays + in-place ops | | Memory | Peak Usage | 50-70% reduction | Buffer pooling + streaming I/O |

Key Optimizations

• Buffer Pooling: Reusable buffer allocation reduces GC pressure by 70%+
• Streaming I/O: Constant memory usage for files of any size
• Native Libraries: xxhash-wasm and zstd-napi for maximum performance
• Pre-calculation: Eliminates backward seeks in write operations
• Parallel Processing: Multi-threaded extraction and compression
• Smart Caching: 99%+ cache hit rate for hash operations
• Typed Arrays: Float32Array/Uint16Array for structure operations
• Batch Operations: Optimized array processing across all formats
• Iterative Processing: Stack-based iteration replaces recursion
• Fast-path Methods: Direct buffer access for primitive types

Installation

npm install ts-ritofile

Optional Native Dependencies

For maximum performance, install these native libraries:

npm install xxhash-wasm zstd-napi

The library will automatically use native implementations when available and gracefully fall back to JavaScript implementations if not.

Quick Start

Reading BIN Files

import { BIN } from 'ts-ritofile';

// Read from file
const bin = new BIN();
bin.read('path/to/file.bin');

// Access fields
for (const field of bin.fields) {
  console.log(`${field.nameHash}: ${field.type} = ${field.data}`);
}

Writing BIN Files

import { BIN, BINField, BINType } from 'ts-ritofile';

const bin = new BIN();
bin.version = 3;
bin.fields = [
  new BINField(0x12345678, BINType.String, 'Hello World')
];

// Write to file
bin.write('output.bin');

// Or get as buffer
const buffer = bin.write(undefined, true);

Working with WAD Archives

import { WAD } from 'ts-ritofile';

// Read WAD file
const wad = new WAD();
wad.read('path/to/archive.wad.client');

// Extract all files (parallel)
await wad.extractAll('output/directory');

// Extract specific file
const chunk = wad.chunks.find(c => c.pathHash === 0x12345678);
if (chunk) {
  const data = wad.extractChunk(chunk);
  fs.writeFileSync('output.bin', data);
}

// Create new WAD
const newWad = new WAD();
newWad.addFile('file1.bin', buffer1);
newWad.addFile('file2.bin', buffer2);
newWad.write('output.wad.client');

Reading Skeleton Files

import { SKL } from 'ts-ritofile';

const skl = new SKL();
skl.read('champion.skl');

console.log(`Bones: ${skl.bones.length}`);
for (const bone of skl.bones) {
  console.log(`${bone.name}: parent=${bone.parentId}`);
}

Reading Skin/Mesh Files

import { SKN } from 'ts-ritofile';

const skn = new SKN();
skn.read('champion.skn');

console.log(`Submeshes: ${skn.submeshes.length}`);
console.log(`Vertices: ${skn.vertices.length}`);
console.log(`Indices: ${skn.indices.length}`);

Performance Configuration

The library provides extensive configuration options to tune performance for your use case:

import { setPerformanceConfig } from 'ts-ritofile';

// Customize performance settings
setPerformanceConfig({
  bufferPool: {
    enabled: true,
    maxPoolSize: 128 * 1024 * 1024,  // 128MB pool
    maxBufferSize: 32 * 1024 * 1024,  // 32MB max buffer
  },
  streaming: {
    chunkSize: 8 * 1024 * 1024,       // 8MB chunks
    enableParallel: true,
    parallelWorkers: 8                // 8 parallel workers
  },
  compression: {
    reuseContexts: true,
    zstdLevel: 5,                     // Higher compression
    enableParallel: true
  },
  hash: {
    warmCache: true,
    cacheSize: 200000                 // Larger cache
  },
  debug: {
    enableStats: true,                // Track statistics
    logPerformance: true              // Log warnings
  }
});

Configuration Options

Buffer Pool Settings

• enabled (default: true): Enable buffer pooling
• maxPoolSize (default: 64MB): Maximum total bytes in pool
• maxBufferSize (default: 16MB): Maximum individual buffer size
• minBufferSize (default: 4KB): Minimum buffer size to pool
• enableStats (default: false): Track allocation statistics

Streaming Settings

• chunkSize (default: 4MB): Chunk size for streaming operations
• enableParallel (default: true): Enable parallel processing
• parallelWorkers (default: 4): Number of parallel workers

Compression Settings

• reuseContexts (default: true): Reuse compression contexts
• zstdLevel (default: 3): Zstd compression level (1-22)
• enableParallel (default: true): Enable parallel compression

Hash Settings

• warmCache (default: true): Pre-warm cache with common values
• cacheSize (default: 100000): Maximum cache entries

Debug Settings

• enableStats (default: false): Enable performance statistics
• logPerformance (default: false): Log performance warnings

Supported Formats

| Format | Description | Read | Write | Streaming | |--------|-------------|------|-------|-----------| | BIN | Property files (v1-3) | ✅ | ✅ | ✅ | | WAD | Archive files | ✅ | ✅ | ✅ | | SKL | Skeleton files | ✅ | ✅ | - | | SKN | Skin/mesh files | ✅ | ✅ | ✅ | | ANM | Animation files | ✅ | ✅ | - | | TEX | Texture files (DDS) | ✅ | ✅ | - | | BNK | Audio bank files | ✅ | ✅ | - | | WPK | Audio package files | ✅ | ✅ | - | | MAPGEO | Map geometry | ✅ | ✅ | - | | SCO | Static object files | ✅ | ✅ | - | | SCB | Static object files | ✅ | ✅ | - | | INIBIN | Legacy config files | ✅ | ✅ | - | | PRELOAD | Preload lists | ✅ | ✅ | - | | Manifest | Release manifests | ✅ | ✅ | - |

Migration Guide

Upgrading from Previous Versions

Good news: No code changes required! All optimizations are automatic.

The performance improvements are completely transparent:

// Your existing code works exactly the same
const bin = new BIN();
bin.read('file.bin');
// Now 8-10x faster automatically!

const wad = new WAD();
wad.read('archive.wad.client');
// Now 3-5x faster with streaming!

What Changed Under the Hood

• Buffer pooling reduces allocations by 70%+
• Streaming I/O for large files
• Native library usage (xxhash-wasm, zstd-napi)
• Pre-calculated offsets eliminate seeking
• Parallel processing for WAD operations
• Optimized hash caching (99%+ hit rate)

Byte-Identical Output

All write operations produce byte-identical output to previous versions. Your files will be exactly the same, just created much faster.

Optional Configuration

While everything works automatically, you can optionally tune performance:

import { setPerformanceConfig } from 'ts-ritofile';

// Only if you need custom settings
setPerformanceConfig({
  streaming: {
    parallelWorkers: 8  // More workers for faster extraction
  }
});

Benchmarks

Run benchmarks to see performance on your system:

# Run all benchmarks
npm run benchmark:all

# Individual benchmarks
npm run benchmark:bin
npm run benchmark:wad
npm run benchmark:hash
npm run benchmark:memory

# Save baseline for comparison
npm run benchmark:baseline

# Generate HTML report
npm run benchmark:all
# Opens benchmarks/report.html

Example Results

Tested on: Intel i7-9700K, 32GB RAM, NVMe SSD

BIN Format (30,000 fields):
  Read:  0.52s (was 5.1s) - 9.8x faster
  Write: 0.71s (was 8.3s) - 11.7x faster
  Memory: 45MB peak (was 120MB) - 62% reduction

WAD Format (1.2GB archive, 5000 files):
  Read:  1.8s (was 6.2s) - 3.4x faster
  Extract: 12.3s (was 45.1s) - 3.7x faster
  Create: 15.2s (was 68.4s) - 4.5x faster
  Memory: 180MB peak (was 450MB) - 60% reduction

Hash Operations (100,000 operations):
  Cache Hit: 0.8ms (was 15.2ms) - 19x faster
  Cache Miss: 12.3ms (was 28.7ms) - 2.3x faster
  Hit Rate: 99.4%

Development

Building

npm run build      # Compile TypeScript
npm run dev        # Watch mode

Testing

npm test                    # Run all tests
npm run test:performance    # Performance regression tests
npm run test:validation     # Byte-identical validation
npm run test:compatibility  # Backward compatibility tests

Project Structure

ts-ritofile/
├── src/
│   ├── core/              # Core utilities
│   │   ├── buffer-pool.ts        # Buffer pooling system
│   │   ├── performance-config.ts # Performance configuration
│   │   └── performance-features.ts # Feature detection
│   ├── stream/            # Binary stream operations
│   │   ├── bin-stream.ts         # Optimized binary stream
│   │   └── streaming-writer.ts   # Streaming write operations
│   ├── hash/              # Hash algorithms and caching
│   │   ├── hash-algorithms.ts    # Native hash implementations
│   │   ├── bin-hash-cache.ts     # Enhanced caching
│   │   └── hash-table.ts         # Pre-computed lookup tables
│   ├── formats/           # File format implementations
│   │   ├── bin.ts                # BIN format (optimized)
│   │   ├── wad.ts                # WAD format (optimized)
│   │   ├── skl.ts, skn.ts, anm.ts, etc.
│   │   └── bin-size-calculator.ts # Size pre-calculation
│   └── compression/       # Compression utilities
│       └── compression-pool.ts   # Context pooling
├── benchmarks/            # Performance benchmarks
├── test-files/            # Real League files for testing
└── docs/                  # Additional documentation

Native Dependencies

xxhash-wasm

Used for fast hash computation. Falls back to js-xxhash if unavailable.

npm install xxhash-wasm

Performance impact: 2-3x faster hash computation

zstd-napi

Used for Zstandard compression in WAD files. Falls back to pure JavaScript if unavailable.

npm install zstd-napi

Performance impact: 3-5x faster compression/decompression

Fallback Behavior

The library automatically detects available native libraries and falls back gracefully:

// Automatic detection - no code changes needed
const wad = new WAD();
wad.read('file.wad.client');
// Uses zstd-napi if available, otherwise pure JS

// Optional: Check what's being used
import { getPerformanceConfig } from 'ts-ritofile';
const config = getPerformanceConfig();
if (config.debug.logPerformance) {
  // Logs warnings if using fallbacks
}

Contributing

Contributions are welcome! Please ensure:

• All tests pass (npm test)
• Code follows TypeScript strict mode
• Performance benchmarks show no regressions
• Byte-identical output is maintained

License

MIT © SirDexal

Related Projects

• Quartz Tools: Suite of League of Legends modding tools
• RitoShark: GitHub organization for LoL development tools

Acknowledgments

This library has been optimized by analyzing and learning from multiple reference implementations:

• wadtools (Rust): Parallel extraction, memory-mapped I/O, size pre-calculation
• LeagueToolkit (C#): Streaming patterns, type handler caching, efficient serialization
• lolpytools (Python): Baseline comparison and format validation
• Leischii.github.io (JavaScript): Web-optimized patterns and minimal allocations
• RustyLeague (Rust): Zero-copy techniques and efficient data structures

Special thanks to:

• League of Legends community for file format documentation
• Contributors and testers who helped validate optimizations
• Open source maintainers of native dependencies (xxhash-wasm, zstd-napi)

Support

• GitHub Issues: Report bugs or request features
• Documentation: See docs/ directory for detailed guides
• Examples: See examples/ directory for usage examples