@synet/fs-memory

In-Memory FileSystem Adapter - Lightning-fast memory-based storage for testing, caching, and ephemeral data operations with full sync/async support.

Perfect for unit testing, development environments, caching layers, and any scenario requiring fast, temporary file operations without disk I/O.

Features

• Dual API Support: Both synchronous and asynchronous interfaces
• Lightning Fast: Pure in-memory operations with zero disk I/O
• Full FileSystem API: Complete implementation of standard filesystem operations
• UTF-8 Support: Handle text content, JSON, and binary-like data
• Path Intelligence: Automatic directory creation and proper path handling
• Testing Optimized: Perfect for unit tests with instant setup/teardown
• Memory Efficient: Built on proven memfs library
• TypeScript First: Full type safety with comprehensive interface definitions

Installation

# npm
npm install @synet/fs-memory

Quick Start

Basic Usage (Sync)

import { MemFileSystem } from '@synet/fs-memory';

const memFs = new MemFileSystem();

// Write and read files
memFs.writeFileSync('/config.json', JSON.stringify({ app: 'myapp' }));
const config = JSON.parse(memFs.readFileSync('/config.json'));

// Directory operations
memFs.ensureDirSync('/data/cache');
memFs.writeFileSync('/data/cache/user-123.json', '{"name": "John"}');

// Check existence and stats
if (memFs.existsSync('/data/cache/user-123.json')) {
  const stats = memFs.stat('/data/cache/user-123.json');
  console.log('File size:', stats.size);
}

Async Operations

import { MemFileSystem } from '@synet/fs-memory/promises';

const memFs = new MemFileSystem();

async function example() {
  // Async file operations
  await memFs.writeFile('/async-data.txt', 'Hello async world!');
  const content = await memFs.readFile('/async-data.txt');
  
  // Concurrent operations
  await Promise.all([
    memFs.writeFile('/file1.txt', 'content 1'),
    memFs.writeFile('/file2.txt', 'content 2'),
    memFs.writeFile('/file3.txt', 'content 3'),
  ]);
  
  const files = await memFs.readDir('/');
  console.log('Files:', files); // ['file1.txt', 'file2.txt', 'file3.txt']
}

Advanced Configuration

Unit Testing Setup

import { MemFileSystem } from '@synet/fs-memory';
import { describe, it, beforeEach, afterEach } from 'vitest';

describe('My App Tests', () => {
  let memFs: MemFileSystem;

  beforeEach(() => {
    memFs = new MemFileSystem();
    // Setup test data
    memFs.writeFileSync('/test-config.json', JSON.stringify({
      environment: 'test',
      database: 'memory'
    }));
  });

  afterEach(() => {
    // Clean up (optional - memory is automatically released)
    memFs.clear('/');
  });

  it('should load configuration', () => {
    const config = JSON.parse(memFs.readFileSync('/test-config.json'));
    expect(config.environment).toBe('test');
  });
});

Async Test Patterns

import { MemFileSystem } from '@synet/fs-memory/promises';

describe('Async Operations', () => {
  let memFs: MemFileSystem;

  beforeEach(() => {
    memFs = new MemFileSystem();
  });

  it('should handle concurrent writes', async () => {
    const operations = [];
    for (let i = 0; i < 100; i++) {
      operations.push(memFs.writeFile(`/data-${i}.json`, `{"id": ${i}}`));
    }
    
    await Promise.all(operations);
    
    const files = await memFs.readDir('/');
    expect(files).toHaveLength(100);
  });
});

Cache Implementation

import { MemFileSystem } from '@synet/fs-memory/promises';

class MemoryCache {
  private fs = new MemFileSystem();
  
  async set(key: string, data: any, ttl?: number): Promise<void> {
    const cacheEntry = {
      data,
      timestamp: Date.now(),
      ttl: ttl ? Date.now() + ttl : null
    };
    
    await this.fs.writeFile(`/cache/${key}.json`, JSON.stringify(cacheEntry));
  }
  
  async get(key: string): Promise<any | null> {
    try {
      if (!await this.fs.exists(`/cache/${key}.json`)) {
        return null;
      }
      
      const content = await this.fs.readFile(`/cache/${key}.json`);
      const entry = JSON.parse(content);
      
      // Check TTL
      if (entry.ttl && Date.now() > entry.ttl) {
        await this.fs.deleteFile(`/cache/${key}.json`);
        return null;
      }
      
      return entry.data;
    } catch {
      return null;
    }
  }
  
  async clear(): Promise<void> {
    await this.fs.clear('/cache');
  }
}

// Usage
const cache = new MemoryCache();
await cache.set('user-123', { name: 'John', role: 'admin' }, 3600000); // 1 hour
const user = await cache.get('user-123');

API Reference

Synchronous Interface

File Operations

• writeFileSync(path: string, data: string): void - Write file content
• readFileSync(path: string): string - Read file content
• deleteFileSync(path: string): void - Delete file (idempotent)
• existsSync(path: string): boolean - Check if path exists

Directory Operations

• ensureDirSync(path: string): void - Create directory recursively
• deleteDirSync(path: string): void - Delete directory recursively (idempotent)
• readDirSync(path: string): string[] - List directory contents

Metadata Operations

• stat(path: string): FileStats - Get file/directory statistics
• chmodSync(path: string, mode: number): void - Change permissions

Utility Operations

• clear(path: string): void - Clear directory contents

Asynchronous Interface

File Operations

• writeFile(path: string, data: string): Promise<void> - Write file content
• readFile(path: string): Promise<string> - Read file content
• deleteFile(path: string): Promise<void> - Delete file (throws on missing)
• exists(path: string): Promise<boolean> - Check if path exists

Directory Operations

• ensureDir(path: string): Promise<void> - Create directory recursively
• deleteDir(path: string): Promise<void> - Delete directory recursively (idempotent)
• readDir(path: string): Promise<string[]> - List directory contents

Metadata Operations

• stat(path: string): Promise<FileStats> - Get file/directory statistics
• chmod(path: string, mode: number): Promise<void> - Change permissions

Utility Operations

• clear(path: string): Promise<void> - Clear directory contents

FileStats Interface

interface FileStats {
  isFile(): boolean;
  isDirectory(): boolean;
  isSymbolicLink(): boolean;
  size: number;
  mtime: Date;  // Modified time
  ctime: Date;  // Created time
  atime: Date;  // Accessed time
  mode: number; // File permissions
}

Testing

# Run all tests
npm test

# Run tests in watch mode
npm run dev:test

# Run tests with coverage
npm run coverage

# Type checking
npm run type-check

Test Coverage

The memory adapter includes comprehensive tests covering:

• File Operations: Read, write, delete, existence checks
• Directory Operations: Creation, deletion, listing, recursive operations
• Path Handling: Root files, nested paths, special characters
• Content Types: UTF-8, JSON, multiline, binary-like content
• Error Handling: Missing files, invalid operations, graceful degradation
• Concurrent Operations: Parallel reads/writes, Promise patterns
• Memory Management: Cleanup, isolation behavior
• Edge Cases: Empty files, large content, special paths

Use Cases

1. Unit Testing

// Fast, isolated test environments
const testFs = new MemFileSystem();
testFs.writeFileSync('/test-data.json', JSON.stringify(mockData));

2. Development Environment

// Quick prototyping without file system setup
const devFs = new MemFileSystem();
devFs.ensureDirSync('/mock-api/users');
devFs.writeFileSync('/mock-api/users/123.json', '{"name": "Test User"}');

3. Caching Layer

// Fast in-memory caching for computed results
const cache = new MemFileSystem();
await cache.writeFile(`/cache/result-${hash}.json`, JSON.stringify(expensiveResult));

4. Temporary Storage

// Processing pipelines with intermediate files
const temp = new MemFileSystem();
await temp.writeFile('/temp/step1.json', processedData);
await temp.writeFile('/temp/step2.json', furtherProcessed);

5. Mock File Systems

// Simulating complex directory structures
const mockFs = new MemFileSystem();
mockFs.ensureDirSync('/app/config');
mockFs.ensureDirSync('/app/data/users');
mockFs.ensureDirSync('/app/logs');

Performance

Benchmarks

In-memory operations are 100-1000x faster than disk I/O:

| Operation | Memory | Disk | Speedup | |-----------|--------|------|---------| | Small file read (1KB) | 0.01ms | 1-5ms | 100-500x | | Large file read (1MB) | 0.1ms | 10-50ms | 100-500x | | Directory listing | 0.01ms | 1-10ms | 100-1000x | | File creation | 0.01ms | 1-5ms | 100-500x |

Memory Usage

• Minimal Overhead: Only stores actual file content in memory
• Efficient Structure: Based on proven memfs implementation
• Automatic Cleanup: Memory released when instance is garbage collected
• Scalable: Handles thousands of files efficiently

Best Practices

// ✅ Good: Clear specific directories when done
await memFs.clear('/temp');

// ✅ Good: Use for temporary operations
const tempResult = await processInMemory(memFs);

// ❌ Avoid: Storing very large files (>100MB) for extended periods
// ❌ Avoid: Using as permanent storage replacement

Error Handling

Sync Error Patterns

try {
  const content = memFs.readFileSync('/missing-file.txt');
} catch (error) {
  if (error.message.includes('Failed to read file')) {
    console.log('File not found');
  }
}

// Idempotent operations don't throw
memFs.deleteFileSync('/maybe-exists.txt'); // Safe to call
memFs.deleteDirSync('/maybe-exists-dir'); // Safe to call

Async Error Patterns

try {
  const content = await memFs.readFile('/missing-file.txt');
} catch (error) {
  console.log('Read failed:', error.message);
}

// Using promises
memFs.readFile('/missing.txt')
  .then(content => console.log(content))
  .catch(error => console.log('File not found'));

// Graceful degradation
const content = await memFs.exists('/config.json') 
  ? await memFs.readFile('/config.json')
  : '{}'; // Default config

Development

Building

npm run build

Linting

npm run lint
npm run lint:fix

Formatting

npm run format

License

MIT License - see LICENSE file for details.

Contributing

Contributions are welcome! Please read our Contributing Guide for details on our code of conduct and the process for submitting pull requests.

Development Setup

git clone https://github.com/synthetism/fs-memory.git
cd fs-memory
npm install
npm test

Related Packages

• @synet/fs - Core filesystem abstraction and Unit Architecture
• @synet/fs-azure - Azure Blob Storage adapter
• @synet/fs-gcs - Google Cloud Storage adapter
• @synet/fs-s3 - AWS S3 storage adapter

Built with ❤️ by the Synet Team