@kadi.build/file-manager

Foundational file management package providing local and remote file operations, file watching, compression utilities, and file streaming.

Features

• Local File Operations — Upload, download, copy, move, rename, delete files and folders
• Remote File Operations — SSH/SFTP-based remote operations with bastion/jump host support
• File Watching — Real-time file system monitoring with configurable debouncing and filtering
• Compression — ZIP and TAR.GZ compression/decompression with progress tracking and batch operations
• File Streaming — Memory-efficient streaming with range request support, ETag generation, and download tracking

Installation

npm install @kadi.build/file-manager

Quick Start

import { createFileManager } from '@kadi.build/file-manager';

// Create a file manager instance
const fm = await createFileManager();

// Local file operations
await fm.uploadFile('./source.txt', './destination.txt');
const info = await fm.getFileInfo('./myfile.txt');
const files = await fm.listFiles('./my-directory');

// Compression
await fm.compressFile('./folder', './backup.zip', { format: 'zip' });
await fm.decompressFile('./backup.zip', './extracted/');

// File watching
const watcher = await fm.startWatching('./watched-dir');
fm.on('fileEvent', (event) => {
  console.log(`${event.type}: ${event.path}`);
});

API Reference

FileManager

The main orchestrator that provides a unified interface to all providers.

import { FileManager, ConfigManager } from '@kadi.build/file-manager';

const config = new ConfigManager();
await config.load();
const fm = new FileManager(config);

Connection & Validation

| Method | Description | |--------|-------------| | testConnection(providerName) | Test connectivity for a specific provider ('local', 'watch', 'compression', 'remote') | | validateProvider(providerName) | Validate provider configuration | | getSystemInfo() | Get system information (platform, memory, disk space) |

File Operations

| Method | Description | |--------|-------------| | uploadFile(source, dest, providerName?) | Copy/upload a file | | downloadFile(source, dest, providerName?) | Copy/download a file | | getFileInfo(filePath, providerName?) | Get file metadata | | listFiles(directory, providerName?, options?) | List files in a directory | | deleteFile(filePath, providerName?) | Delete a file | | renameFile(oldPath, newPath, providerName?) | Rename a file | | copyFile(source, dest, providerName?) | Copy a file | | moveFile(source, dest, providerName?) | Move a file | | searchFiles(directory, pattern, options?) | Search for files matching a pattern |

Folder Operations

| Method | Description | |--------|-------------| | createFolder(folderPath, providerName?) | Create a directory (recursive) | | listFolders(directory, providerName?) | List subdirectories | | deleteFolder(folderPath, options?) | Delete a directory | | renameFolder(oldPath, newPath) | Rename a directory | | copyFolder(source, dest) | Copy a directory recursively | | moveFolder(source, dest) | Move a directory | | getFolderInfo(folderPath) | Get directory metadata |

Compression

| Method | Description | |--------|-------------| | compressFile(input, output, options?) | Compress a file or directory | | decompressFile(archive, outputDir, options?) | Decompress an archive | | compress(input, output, options?) | Alias for compressFile | | decompress(archive, outputDir, options?) | Alias for decompressFile | | compressMultipleFiles(files, outputDir, options?) | Batch compress multiple files | | decompressMultipleFiles(archives, outputDir, options?) | Batch decompress multiple archives | | getCompressionStatus() | Get compression provider status | | getCompressionProvider() | Get the underlying compression provider |

Compression Options:

{
  format: 'zip' | 'tar.gz',  // Archive format (default: 'zip')
  level: 1-9,                  // Compression level (default: 6)
  includeRoot: true,           // Include root directory in archive
  overwrite: true              // Overwrite existing files during extraction
}

File Watching

| Method | Description | |--------|-------------| | startWatching(path, options?) | Start watching a path for changes | | stopWatching(watchIdOrPath) | Stop a specific watcher | | stopAllWatching() | Stop all active watchers | | listActiveWatchers() | List all active watchers | | getWatcherInfo(watchId) | Get details about a watcher | | getWatchingStatus() | Get overall watching status |

Watch Options:

{
  recursive: true,         // Watch subdirectories
  events: ['add', 'change', 'unlink'],  // Event types to listen for
  ignoreDotfiles: false,   // Ignore dotfiles
  debounceMs: 100          // Debounce interval in ms
}

Events:

fm.on('fileEvent', (event) => {
  // event.type: 'add' | 'change' | 'unlink' | 'addDir' | 'unlinkDir'
  // event.path: string
  // event.watchId: string
});

Remote Operations

| Method | Description | |--------|-------------| | connectRemote() | Connect to the remote server | | disconnectRemote() | Disconnect from the remote server | | syncToRemote(localPath, remotePath) | Upload files to remote | | syncFromRemote(remotePath, localPath) | Download files from remote |

Remote Configuration (via environment variables):

KADI_REMOTE_HOST=example.com
KADI_REMOTE_PORT=22
KADI_REMOTE_USERNAME=user
KADI_REMOTE_PASSWORD=pass
KADI_REMOTE_PRIVATE_KEY_PATH=~/.ssh/id_rsa

Individual Providers

For advanced use cases, you can import providers directly:

import {
  LocalProvider,
  WatchProvider,
  CompressionProvider,
  RemoteProvider,
  FileStreamingUtils,
  FileStreamer,
  DownloadTracker,
  PathUtils
} from '@kadi.build/file-manager';

Factory Functions

import { createFileManager, compressFile, decompressFile, watchDirectory } from '@kadi.build/file-manager';

// Quick file manager creation
const fm = await createFileManager(options);

// One-shot compression
await compressFile('./input', './output.zip', { format: 'zip' });

// One-shot decompression
await decompressFile('./archive.zip', './output/');

// Quick directory watching
const watcher = await watchDirectory('./path', (event) => {
  console.log(event);
});

Configuration

Configuration is loaded from multiple sources (in priority order):

1. Constructor options
2. Environment variables (prefixed with KADI_)
3. config.yml file (walks up from CWD — looks for file-manager: section)
4. .env file (walks up from CWD — legacy fallback)

config.yml

Place a config.yml in your project root (or any parent directory):

file-manager:
  local_root: ./
  default_upload_directory: ./uploads
  default_download_directory: ./downloads
  default_temp_directory: ./temp
  max_file_size: 1073741824
  watch_debounce_ms: 100
  watch_max_watchers: 10
  compression_format: zip
  compression_level: 6

Environment Variables (overrides)

| Variable | Description | Default | |----------|-------------|---------| | KADI_LOCAL_ROOT | Root directory for local operations | . | | KADI_WATCH_DEBOUNCE_MS | Debounce interval for file watching | 100 | | KADI_WATCH_MAX_WATCHERS | Maximum concurrent watchers | 10 | | KADI_COMPRESSION_FORMAT | Default compression format | zip | | KADI_COMPRESSION_LEVEL | Default compression level | 6 | | KADI_REMOTE_HOST | Remote SSH host | — | | KADI_REMOTE_PORT | Remote SSH port | 22 | | KADI_REMOTE_USERNAME | Remote SSH username | — | | KADI_REMOTE_PASSWORD | Remote SSH password | — | | KADI_REMOTE_PRIVATE_KEY_PATH | Path to SSH private key | — |

Testing

# Run all tests
npm test

# Run individual test suites
npm run test:local        # Local file operations (33 tests)
npm run test:watch        # File watching (30 tests)
npm run test:compression  # Compression (42 tests)
npm run test:streaming    # File streaming (12 tests)
npm run test:remote       # Remote operations (18 tests)

Total: 135 tests across 5 suites

Project Structure

packages/file-manager/
├── package.json
├── README.md
├── src/
│   ├── index.js              # Main entry point & exports
│   ├── FileManager.js        # Main orchestrator
│   ├── ConfigManager.js      # Configuration management
│   ├── providers/
│   │   ├── LocalProvider.js       # Local file operations
│   │   ├── WatchProvider.js       # File watching (chokidar)
│   │   ├── CompressionProvider.js # ZIP & TAR.GZ compression
│   │   └── RemoteProvider.js      # SSH/SFTP remote operations
│   └── utils/
│       ├── FileStreamingUtils.js  # Streaming, MIME detection, ETags
│       └── PathUtils.js           # Path validation & normalization
└── tests/
    ├── run-all-tests.js
    ├── test-utils.js
    ├── test-helpers/
    │   └── index.js
    ├── test-local-operations.js
    ├── test-file-watching.js
    ├── test-compression.js
    ├── test-file-streaming.js
    └── test-remote-operations.js

License

MIT