@firesystem/core

Virtual File System (VFS) core interfaces and types for JavaScript/TypeScript applications. This package provides a consistent API for implementing file system operations across different storage backends.

Features

• 🎯 Unified API - Same interface for IndexedDB, Memory, S3, or any storage backend
• 📁 Full FS Operations - Read, write, mkdir, rmdir, rename, move, copy, and more
• 👀 Watch System - Real-time notifications with glob pattern support
• 🔍 Glob Patterns - Search files using familiar glob syntax
• 🚀 Zero Dependencies - Core package has no runtime dependencies
• 📝 TypeScript First - Written in TypeScript with complete type definitions
• ✅ Battle Tested - Comprehensive test suite ensuring reliability

Installation

npm install @firesystem/core
# or
yarn add @firesystem/core
# or
pnpm add @firesystem/core

Available Implementations

• @firesystem/indexeddb - Browser storage using IndexedDB
• @firesystem/memory - In-memory storage for testing
• @firesystem/s3 - AWS S3 storage backend
• @firesystem/workspace - Multi-source workspace manager
• More coming soon (Node.js fs, etc.)

Creating Custom Implementations

VFS provides a BaseFileSystem abstract class that implements common functionality:

import { BaseFileSystem } from "@firesystem/core";

export class MyCustomFileSystem extends BaseFileSystem {
  // Define capabilities
  readonly capabilities = {
    readonly: false,
    caseSensitive: true,
    atomicRename: true,
    supportsWatch: true,
    supportsMetadata: true,
    maxFileSize: 10 * 1024 * 1024, // 10MB
    maxPathLength: 255,
  };

  // Implement required methods
  async readFile(path: string): Promise<FileEntry> {
    // Your implementation
  }

  async writeFile(
    path: string,
    content: any,
    metadata?: FileMetadata,
  ): Promise<FileEntry> {
    // Your implementation
  }

  // ... implement other required methods

  // Override permission methods if needed
  async canModify(path: string): Promise<boolean> {
    // Custom permission logic
    return super.canModify(path); // or your own logic
  }
}

API Overview

IFileSystem Interface

The main interface that all VFS implementations must follow:

interface IFileSystem {
  // File operations
  readFile(path: string): Promise<FileEntry>;
  writeFile(
    path: string,
    content: any,
    metadata?: FileMetadata,
  ): Promise<FileEntry>;
  deleteFile(path: string): Promise<void>;
  exists(path: string): Promise<boolean>;

  // Directory operations
  readDir(path: string): Promise<FileEntry[]>;
  mkdir(path: string, recursive?: boolean): Promise<FileEntry>;
  rmdir(path: string, recursive?: boolean): Promise<void>;

  // File system operations
  rename(oldPath: string, newPath: string): Promise<FileEntry>;
  move(sourcePaths: string[], targetPath: string): Promise<void>;
  copy(sourcePath: string, targetPath: string): Promise<FileEntry>;

  // Watch operations
  watch(pattern: string, callback: (event: FSEvent) => void): Disposable;

  // Utility operations
  stat(path: string): Promise<FileStat>;
  glob(pattern: string): Promise<string[]>;

  // Storage management
  clear(): Promise<void>;
  size(): Promise<number>;

  // Permission checking
  canModify(path: string): Promise<boolean>;
  canCreateIn(parentPath: string): Promise<boolean>;

  // Atomic operations (optional)
  writeFileAtomic?(
    path: string,
    content: any,
    metadata?: FileMetadata,
  ): Promise<FileEntry>;

  // Capabilities (optional)
  readonly capabilities?: IFileSystemCapabilities;
}

Usage Examples

Basic File Operations

import { IndexedDBFileSystem } from "@firesystem/indexeddb";

const fs = new IndexedDBFileSystem({ dbName: "my-app" });

// Write a file
await fs.writeFile("/hello.txt", "Hello, World!");

// Read a file
const file = await fs.readFile("/hello.txt");
console.log(file.content); // "Hello, World!"

// Check if file exists
const exists = await fs.exists("/hello.txt"); // true

// Delete a file
await fs.deleteFile("/hello.txt");

Working with Directories

// Create a directory
await fs.mkdir("/src");

// Create nested directories
await fs.mkdir("/src/components/Button", true);

// List directory contents
const files = await fs.readDir("/src");
console.log(files); // [{ name: "components", type: "directory", ... }]

// Remove empty directory
await fs.rmdir("/src/components/Button");

// Remove directory and all contents
await fs.rmdir("/src", true);

File Metadata

// Write file with metadata
await fs.writeFile(
  "/document.json",
  { title: "My Document" },
  {
    tags: ["important", "work"],
    description: "Quarterly report",
  },
);

// Read file with metadata
const doc = await fs.readFile("/document.json");
console.log(doc.metadata?.tags); // ["important", "work"]

// Get file statistics
const stats = await fs.stat("/document.json");
console.log(stats.size); // File size in bytes
console.log(stats.created); // Date created
console.log(stats.modified); // Date last modified

Moving and Copying Files

// Rename a file
await fs.rename("/old-name.txt", "/new-name.txt");

// Move files to a directory
await fs.mkdir("/archive");
await fs.move(["/file1.txt", "/file2.txt"], "/archive");

// Copy a file
await fs.copy("/template.html", "/index.html");

Glob Patterns

// Find all JavaScript files
const jsFiles = await fs.glob("**/*.js");

// Find all files in src directory
const srcFiles = await fs.glob("/src/**/*");

// Find all test files
const testFiles = await fs.glob("**/*.test.{js,ts}");

// Find all files in root directory only
const rootFiles = await fs.glob("*");

Watch System

// Watch all files
const watcher = fs.watch("**", (event) => {
  console.log(`${event.type}: ${event.path}`);
});

// Watch specific file types
fs.watch("**/*.json", (event) => {
  if (event.type === "updated") {
    console.log(`JSON file updated: ${event.path}`);
  }
});

// Watch specific directory
fs.watch("/src/**/*", (event) => {
  console.log(`Change in src: ${event.type} ${event.path}`);
});

// Stop watching
watcher.dispose();

Watch Event Types

type FSEventType =
  | "created" // File or directory was created
  | "updated" // File content was modified
  | "deleted" // File or directory was deleted
  | "moved" // File or directory was moved (event.oldPath available)
  | "renamed"; // File or directory was renamed (event.oldPath available)

Storage Management

// Get total storage size
const totalSize = await fs.size();
console.log(`Total storage: ${totalSize} bytes`);

// Clear all files (except root directory)
await fs.clear();

Permission Checking

VFS now supports permission checking before operations:

// Check if can modify a file
if (await fs.canModify("/protected.txt")) {
  await fs.deleteFile("/protected.txt");
} else {
  console.log("File is read-only or protected");
}

// Check if can create in directory
if (await fs.canCreateIn("/restricted")) {
  await fs.writeFile("/restricted/new.txt", "content");
} else {
  console.log("Cannot create files in this directory");
}

Atomic Writing

For critical operations, use atomic writing when available:

// Writes to temp file then renames (atomic)
// Falls back to regular write if not supported
if (fs.writeFileAtomic) {
  await fs.writeFileAtomic("/important.json", {
    data: "critical data",
  });
} else {
  await fs.writeFile("/important.json", {
    data: "critical data",
  });
}

File System Capabilities

Check what a file system supports:

// Check capabilities
if (fs.capabilities) {
  console.log(`Read-only: ${fs.capabilities.readonly}`);
  console.log(`Case sensitive: ${fs.capabilities.caseSensitive}`);
  console.log(`Atomic rename: ${fs.capabilities.atomicRename}`);
  console.log(`Supports watch: ${fs.capabilities.supportsWatch}`);

  if (fs.capabilities.maxFileSize) {
    console.log(`Max file size: ${fs.capabilities.maxFileSize} bytes`);
  }
}

// Example: Check before large file operations
const largeData = new ArrayBuffer(100 * 1024 * 1024); // 100MB
if (
  fs.capabilities?.maxFileSize &&
  largeData.byteLength > fs.capabilities.maxFileSize
) {
  throw new Error(
    `File too large. Max size: ${fs.capabilities.maxFileSize} bytes`,
  );
}

Path Handling

All paths in VFS are normalized automatically:

// These all resolve to the same path
await fs.writeFile("file.txt", "content"); // -> /file.txt
await fs.writeFile("/file.txt", "content"); // -> /file.txt
await fs.writeFile("//file.txt", "content"); // -> /file.txt
await fs.writeFile("/file.txt/", "content"); // -> /file.txt

Error Handling

VFS follows POSIX error conventions:

try {
  await fs.readFile("/missing.txt");
} catch (error) {
  // Error: ENOENT: no such file or directory, open '/missing.txt'
}

try {
  await fs.rmdir("/has-files");
} catch (error) {
  // Error: ENOTEMPTY: directory not empty, rmdir '/has-files'
}

try {
  await fs.mkdir("/already/exists");
} catch (error) {
  // Error: EEXIST: file already exists, mkdir '/already/exists'
}

Type Definitions

FileEntry

interface FileEntry {
  path: string;
  name: string;
  type: "file" | "directory";
  size?: number;
  created?: Date;
  modified?: Date;
  metadata?: FileMetadata;
  content?: any; // Only present when reading files
}

FileStat

interface FileStat {
  path: string;
  size: number;
  type: "file" | "directory";
  created: Date;
  modified: Date;
  accessed?: Date;
  readonly?: boolean; // Indicates if file/directory is read-only
}

FileMetadata

interface FileMetadata {
  tags?: string[];
  description?: string;
  [key: string]: any; // Custom metadata
}

FSEvent

interface FSEvent {
  type: FSEventType;
  path: string;
  oldPath?: string; // Present for move/rename events
  timestamp: Date;
  metadata?: Record<string, any>;
}

Disposable

interface Disposable {
  dispose(): void;
}

IFileSystemCapabilities

interface IFileSystemCapabilities {
  // Operation support
  readonly: boolean; // Is the file system read-only?
  caseSensitive: boolean; // Does it differentiate case?
  atomicRename: boolean; // Is rename operation atomic?

  // Limits
  maxFileSize?: number; // Maximum file size in bytes
  maxPathLength?: number; // Maximum path length

  // Features
  supportsWatch: boolean; // Does it support file watching?
  supportsMetadata: boolean; // Does it support custom metadata?
}

Glob Pattern Support

VFS supports standard glob patterns:

• * - Matches any characters except /
• ** - Matches any number of directories
• ? - Matches single character except /
• {a,b} - Matches either a or b
• [abc] - Matches any character in brackets
• [!abc] - Matches any character not in brackets

Examples

"*.js"; // All .js files in root
"**/*.js"; // All .js files recursively
"src/**/*.ts"; // All .ts files under src
"*.{js,ts}"; // All .js and .ts files in root
"test/**"; // Everything under test directory
"**/test/*"; // All files directly under any test directory

Best Practices

1. Always use absolute paths internally

// Good
await fs.writeFile("/config/app.json", config);

// Okay (will be normalized to /config/app.json)
await fs.writeFile("config/app.json", config);

2. Check existence before operations

if (await fs.exists("/old-file.txt")) {
  await fs.deleteFile("/old-file.txt");
}

3. Use recursive flag for nested directories

// This will fail if parent doesn't exist
await fs.mkdir("/deep/nested/dir");

// This will create all parent directories
await fs.mkdir("/deep/nested/dir", true);

4. Dispose watchers when done

const watcher = fs.watch("**/*.log", handler);

// Later...
watcher.dispose(); // Stop watching to free resources

5. Handle errors appropriately

async function safeReadFile(path: string): Promise<string | null> {
  try {
    const file = await fs.readFile(path);
    return file.content;
  } catch (error) {
    if (error.message.includes("ENOENT")) {
      return null; // File doesn't exist
    }
    throw error; // Re-throw other errors
  }
}

Testing

For testing, use the in-memory implementation:

import { MemoryFileSystem } from "@firesystem/memory";

describe("My App", () => {
  let fs: IFileSystem;

  beforeEach(() => {
    fs = new MemoryFileSystem();
  });

  it("should save user data", async () => {
    await saveUserData(fs, { name: "John" });

    const file = await fs.readFile("/users/john.json");
    expect(file.content.name).toBe("John");
  });
});

Contributing

Contributions are welcome! Please read our Contributing Guide for details.

License

MIT © Anderson D. Rosa