@firesystem/indexeddb

IndexedDB implementation of the Virtual File System (VFS) interface. Provides persistent storage in web browsers with an intuitive file system API.

Features

• 🌐 Browser Native - Uses IndexedDB for persistent storage
• 📦 Large Storage - Store gigabytes of data (browser limits apply)
• 🔄 Full VFS API - Implements complete @firesystem/core interface
• 🚀 Async Performance - Non-blocking operations
• 🔍 Indexing Support - Fast file lookups and queries
• 💾 Persistent - Data survives browser restarts
• ⚡ Native Events - Built-in reactive event system for all operations
• 📊 Progress Tracking - Monitor initialization and file loading
• 🔔 Real-time Updates - Get notified of all file system changes
• 🔌 Workspace Ready - Optional integration with @workspace-fs/core
• 🔄 Multi-tab Sync - Changes sync across browser tabs

Installation

npm install @firesystem/indexeddb
# or
yarn add @firesystem/indexeddb
# or
pnpm add @firesystem/indexeddb

Usage

import { IndexedDBFileSystem } from "@firesystem/indexeddb";

// Create a new file system instance
const fs = new IndexedDBFileSystem({
  dbName: "my-app-storage", // Optional, defaults to "vfs"
  version: 1, // Optional, defaults to 1
});

// Use it like any other VFS implementation
await fs.writeFile("/data.json", { message: "Hello, IndexedDB!" });
const file = await fs.readFile("/data.json");
console.log(file.content); // { message: "Hello, IndexedDB!" }

// NEW: Native event support
fs.events.on("file:written", ({ path, size }) => {
  console.log(`File ${path} saved to IndexedDB (${size} bytes)`);
});

Configuration Options

interface IndexedDBConfig {
  dbName?: string; // Database name (default: "vfs")
  version?: number; // Database version (default: 1)
  storeName?: string; // Object store name (default: "files")
  enableExternalSync?: boolean; // Enable cross-tab sync and external change detection (default: true)
  deepChangeDetection?: boolean; // Check content even if date hasn't changed (default: false)
}

const fs = new IndexedDBFileSystem({
  dbName: "my-app",
  version: 1,
  storeName: "documents",
  enableExternalSync: true, // Enable external change detection
});

Event System

IndexedDBFileSystem includes a native event system that emits events for all operations:

Basic Events

import { IndexedDBFileSystem, FileSystemEvents } from "@firesystem/indexeddb";

const fs = new IndexedDBFileSystem({ dbName: "my-app" });

// Listen to file operations
fs.events.on(FileSystemEvents.FILE_WRITTEN, ({ path, size }) => {
  console.log(`File written: ${path} (${size} bytes)`);
});

fs.events.on(FileSystemEvents.FILE_DELETED, ({ path }) => {
  console.log(`File deleted: ${path}`);
});

// Listen to directory operations
fs.events.on(FileSystemEvents.DIR_CREATED, ({ path }) => {
  console.log(`Directory created: ${path}`);
});

// Operation tracking with timing
fs.events.on(FileSystemEvents.OPERATION_END, ({ operation, duration }) => {
  console.log(`${operation} completed in ${duration}ms`);
});

Initialization with Progress

Load all existing files from IndexedDB with progress tracking:

const fs = new IndexedDBFileSystem({ dbName: "my-app" });

// Track initialization progress
fs.events.on(FileSystemEvents.INIT_PROGRESS, ({ loaded, total, phase }) => {
  console.log(`${phase}: ${loaded}/${total} files loaded`);
});

// Get notified of each file during initialization
fs.events.on(FileSystemEvents.FILE_READ, ({ path, size }) => {
  console.log(`Loaded: ${path} (${size} bytes)`);
});

fs.events.on(FileSystemEvents.DIR_READ, ({ path }) => {
  console.log(`Found directory: ${path}`);
});

// Initialize and load all existing files
await fs.initialize();

Integration with State Management

// Example with React and Zustand
import { create } from 'zustand';
import { IndexedDBFileSystem, FileSystemEvents } from '@firesystem/indexeddb';

const useFileStore = create((set) => ({
  files: [],
  isLoading: true,
  progress: { loaded: 0, total: 0 },

  initializeFS: async () => {
    const fs = new IndexedDBFileSystem({ dbName: "my-app" });

    // Update progress
    fs.events.on(FileSystemEvents.INIT_PROGRESS, ({ loaded, total }) => {
      set({ progress: { loaded, total } });
    });

    // Populate files as they load
    fs.events.on(FileSystemEvents.FILE_READ, ({ path, size }) => {
      set(state => ({
        files: [...state.files, { path, size, type: 'file' }]
      }));
    });

    fs.events.on(FileSystemEvents.DIR_READ, ({ path }) => {
      set(state => ({
        files: [...state.files, { path, type: 'directory' }]
      }));
    });

    // Mark as loaded when done
    fs.events.on(FileSystemEvents.INITIALIZED, () => {
      set({ isLoading: false });
    });

    // Initialize and load all files from IndexedDB
    await fs.initialize();
  }
}));

// In your component
function FileExplorer() {
  const { files, isLoading, progress, initializeFS } = useFileStore();

  useEffect(() => {
    initializeFS(); // Load all files from IndexedDB
  }, []);

  if (isLoading) {
    return <div>Loading files: {progress.loaded}/{progress.total}</div>;
  }

  return <FileTree files={files} />;
}

Storage Events

// Monitor storage operations
fs.events.on(FileSystemEvents.STORAGE_CLEARING, () => {
  console.log("Clearing all files...");
});

fs.events.on(FileSystemEvents.STORAGE_CLEARED, ({ duration }) => {
  console.log(`Storage cleared in ${duration}ms`);
});

fs.events.on(FileSystemEvents.STORAGE_SIZE_CALCULATED, ({ size }) => {
  console.log(`Total storage used: ${size} bytes`);
});

Workspace Integration

IndexedDBFileSystem can be integrated with @workspace-fs/core through the IndexedDBWorkspaceProvider:

Standalone Usage (Default)

import { IndexedDBFileSystem } from "@firesystem/indexeddb";

// Use directly without workspace
const fs = new IndexedDBFileSystem({ dbName: "my-app" });
await fs.initialize();
await fs.writeFile("/data.json", { message: "Hello!" });

With Workspace

import { WorkspaceFileSystem } from "@workspace-fs/core";
import { indexedDBProvider } from "@firesystem/indexeddb/provider";

// Register the provider
const workspace = new WorkspaceFileSystem();
workspace.registerProvider(indexedDBProvider);

// Create persistent projects
const project = await workspace.loadProject({
  id: "my-project",
  name: "My Project",
  source: {
    type: "indexeddb",
    config: {
      dbName: "project-storage",
      version: 1,
      enableExternalSync: true,
    },
  },
});

// Work with the project - data persists across sessions
await workspace.writeFile("/src/app.js", "// Application code");
await workspace.writeFile("/data/users.json", JSON.stringify([]));

// Switch between multiple persistent projects
const devProject = await workspace.loadProject({
  id: "dev",
  name: "Development",
  source: { type: "indexeddb", config: { dbName: "dev-db" } },
});

const prodProject = await workspace.loadProject({
  id: "prod",
  name: "Production",
  source: { type: "indexeddb", config: { dbName: "prod-db" } },
});

await workspace.setActiveProject("dev");

Provider Configuration

The IndexedDB provider accepts all standard IndexedDBFileSystem configuration options:

interface IndexedDBConfig {
  // Database name (required for persistence)
  dbName?: string;

  // Database version (default: 1)
  version?: number;

  // Object store name (default: "files")
  storeName?: string;

  // Enable cross-tab sync (default: true)
  enableExternalSync?: boolean;

  // Deep change detection (default: false)
  deepChangeDetection?: boolean;
}

External Change Detection

IndexedDBFileSystem can detect changes made to the database from external sources (other tabs, windows, or direct IndexedDB access):

Cross-Tab Synchronization

Changes made in one tab are automatically synchronized to other tabs using BroadcastChannel:

// Tab 1
const fs1 = new IndexedDBFileSystem({ dbName: "my-app" });
fs1.watch("**/*", (event) => {
  console.log(`Change detected: ${event.type} ${event.path}`);
});

// Tab 2
const fs2 = new IndexedDBFileSystem({ dbName: "my-app" });
await fs2.writeFile("/data.json", { updated: true });
// Tab 1 will receive the change notification automatically

Polling for External Changes

To detect changes made directly to IndexedDB (outside of FileSystem API):

const fs = new IndexedDBFileSystem({ dbName: "my-app" });

// Start watching for external changes (default: 250ms polling)
await fs.startWatching(); // Poll every 250ms
// or with custom interval
await fs.startWatching(1000); // Poll every 1 second

// Your watch handlers will now receive notifications for external changes
fs.watch("**/*", (event) => {
  console.log(`External change: ${event.type} ${event.path}`);
});

// Stop watching when done
await fs.stopWatching();

Deep Change Detection

By default, changes are detected by file modification date and size. To detect content changes even when date/size remain the same:

const fs = new IndexedDBFileSystem({
  dbName: "my-app",
  deepChangeDetection: true, // Enable content hash comparison
});

// Now changes will be detected even if:
// - File content changes but date doesn't update
// - Metadata changes but file size remains same
await fs.startWatching();

Note: Deep change detection has performance implications as it requires calculating hashes for all file contents. Use it only when necessary.

Disable External Sync

If you don't need external change detection:

const fs = new IndexedDBFileSystem({
  dbName: "my-app",
  enableExternalSync: false, // Disable all external sync features
});

Browser Storage Limits

IndexedDB storage limits vary by browser:

• Chrome/Edge: Up to 60% of total disk space
• Firefox: Up to 50% of free disk space
• Safari: Up to 1GB initially, can request more

Check available storage:

if ("storage" in navigator && "estimate" in navigator.storage) {
  const estimate = await navigator.storage.estimate();
  console.log(`Using ${estimate.usage} of ${estimate.quota} bytes`);
}

Use Cases

1. Offline Web Apps

const fs = new IndexedDBFileSystem({ dbName: "offline-app" });

// Save user documents locally
await fs.writeFile("/documents/report.pdf", pdfBlob);
await fs.writeFile("/documents/data.xlsx", excelBlob);

// Access offline
const files = await fs.readDir("/documents");

2. Browser-Based IDEs

const fs = new IndexedDBFileSystem({ dbName: "web-ide" });

// Save project files
await fs.mkdir("/src");
await fs.writeFile("/src/index.js", sourceCode);
await fs.writeFile("/package.json", packageJson);

// Watch for changes
fs.watch("**/*.js", (event) => {
  console.log(`File ${event.type}: ${event.path}`);
});

3. Client-Side Caching

const fs = new IndexedDBFileSystem({ dbName: "app-cache" });

// Cache API responses
await fs.writeFile("/cache/users.json", await fetchUsers());

// Read from cache
try {
  const cached = await fs.readFile("/cache/users.json");
  return cached.content;
} catch {
  // Cache miss, fetch fresh data
  const fresh = await fetchUsers();
  await fs.writeFile("/cache/users.json", fresh);
  return fresh;
}

Error Handling

try {
  await fs.writeFile("/data.json", data);
} catch (error) {
  if (error.name === "QuotaExceededError") {
    console.error("Storage quota exceeded");
    // Handle cleanup or request more storage
  }
}

Performance Tips

1. Batch Operations: Group multiple operations when possible
2. Large Files: Consider chunking very large files
3. Indexing: Use meaningful paths for better organization
4. Event Listeners: Remove unused listeners to prevent memory leaks
5. Initialization: Call initialize() once to load all files efficiently

Event Performance

// Dispose of event listeners when done
const disposable = fs.events.on(FileSystemEvents.FILE_WRITTEN, handler);

// Later...
disposable.dispose(); // Remove the listener

// Or remove all listeners for an event
fs.events.removeAllListeners(FileSystemEvents.FILE_WRITTEN);

Cleanup

Always dispose of the file system instance when done to clean up resources:

const fs = new IndexedDBFileSystem({ dbName: "my-app" });

// Use the file system...

// Clean up when done
fs.dispose(); // Closes DB connection, stops watching, cleans up listeners

Testing

When testing code that uses IndexedDBFileSystem in Node.js environments, you have two options:

Option 1: Using the testing module (Recommended)

// Import from the testing module which handles the polyfill automatically
import { IndexedDBFileSystem } from "@firesystem/indexeddb/testing";

describe("My app", () => {
  it("should work with IndexedDB", async () => {
    const fs = new IndexedDBFileSystem({ dbName: "test" });
    await fs.writeFile("/test.txt", "Hello");
    // ...
  });
});

Note: You still need to install fake-indexeddb as a dev dependency:

npm install --save-dev fake-indexeddb

Option 2: Manual setup

npm install --save-dev fake-indexeddb
# or
yarn add -D fake-indexeddb
# or
pnpm add -D fake-indexeddb

Then import it before your tests:

// In your test setup file or at the top of test files
import "fake-indexeddb/auto";

// Now you can test IndexedDBFileSystem
import { IndexedDBFileSystem } from "@firesystem/indexeddb";

describe("My app", () => {
  it("should work with IndexedDB", async () => {
    const fs = new IndexedDBFileSystem({ dbName: "test" });
    await fs.writeFile("/test.txt", "Hello");
    // ...
  });
});

API Reference

See @firesystem/core for the complete VFS API documentation.

License

MIT © Anderson D. Rosa