@avdl/folder-walker

A powerful, fluent, and configurable directory traversal utility for Node.js with TypeScript support.

Features

• 🚀 Fluent API - Chain methods for intuitive configuration
• 📁 Flexible traversal - Files, folders, or both
• 🔍 Smart filtering - By extension, pattern, size, and more
• 📖 Content reading - Optional file content loading
• ⚡ High performance - Async iterators and parallel processing
• 🛡️ Type-safe - Full TypeScript support with great inference
• 0️⃣ Zero dependencies - Lightweight and secure

Installation

npm install @avdl/folder-walker
# or
yarn add @avdl/folder-walker

Quick Start

import { FolderWalker } from '@avdl/folder-walker';

// Basic usage - iterate over all files
const walker = new FolderWalker();
for await (const file of walker.files('.')) {
  console.log(file.path);
}

// Advanced configuration with async operations
const configuredWalker = new FolderWalker()
  .withDepth(3)
  .withExtensions(['.ts', '.js'])
  .withFileContents('utf-8')
  .exclude(['node_modules', '.git'])
  .onFile(async ({ path, content }) => {
    // Async operations like saving to database, uploading, etc.
    await processFile(path, content);
    console.log(`Processed ${path}`);
  });

await configuredWalker.execute('./src');

API Reference

Configuration Methods

• withDepth(n) - Set maximum traversal depth
• withFilesOnly() - Only traverse files
• withFoldersOnly() - Only traverse folders
• withFilesAndFolders() - Traverse both (default)
• withFileContents(encoding) - Read file contents
• withExtensions(extensions) - Filter by file extensions
• withPattern(regex) - Filter by filename pattern
• exclude(patterns) - Exclude paths
• withHidden() - Include hidden files
• withSymlinks() - Follow symbolic links
• withMaxFileSize(bytes) - Skip large files
• withFilter(asyncFilter) - Add custom async filter function

Callbacks (Async Support)

• onFile(handler) - Process each file (supports async functions)
• onFolder(handler) - Process each folder (supports async functions)
• onError(handler) - Handle errors

Result Collection

• withFileCollector() - Collect file handler results into array
• withFileCollector(collectorFn) - Transform collected results with custom function
• withFolderCollector() - Collect folder handler results into array
• withFolderCollector(collectorFn) - Transform collected results with custom function

Execution

• execute(path) - Run with callbacks, returns void (streaming) or collected results
• walker.files(path) - Async iterator for files
• walker.folders(path) - Async iterator for folders
• walker.entries(path) - Async iterator for all entries

Running Examples

Try the included examples to see folder-walker in action:

# List all available examples
npm run examples:list

# Run individual examples
npm run example:basic      # Basic usage patterns
npm run example:async      # Async operations  
npm run example:filters    # Custom filtering
npm run example:collectors # Type-safe collection

# Run all examples
npm run examples

Examples

Find all TypeScript files

const walker = new FolderWalker()
  .withExtensions(['.ts', '.tsx'])
  .exclude(['node_modules', 'dist']);

for await (const file of walker.files('.')) {
  console.log(file.path);
}

Process markdown files with async operations

await new FolderWalker()
  .withExtensions(['.md'])
  .withFileContents('utf-8')
  .onFile(async ({ path, content }) => {
    const wordCount = content.split(/\s+/).length;
    await saveToDatabase(path, wordCount); // Async operation
    console.log(`${path}: ${wordCount} words`);
  })
  .execute('./docs');

Type-safe result collection

// Streaming mode (no collection) - returns void
await new FolderWalker()
  .onFile(({ path }) => {
    console.log(`Processing: ${path}`); // void return
  })
  .execute('./src');

// Collect results - returns ProcessResult[]
const results = await new FolderWalker()
  .onFile(({ path, stats }) => ({ path, size: stats.size })) // ProcessResult return
  .withFileCollector() // Enable collection
  .execute('./src');

// Custom collector transformation - returns ProjectSummary
const summary = await new FolderWalker()
  .onFile(({ path, stats }) => ({ path, size: stats.size }))
  .withFileCollector((files) => ({
    totalFiles: files.length,
    totalSize: files.reduce((sum, f) => sum + f.size, 0)
  })) // Transform to summary
  .execute('./src');

Custom async filters

await new FolderWalker()
  .withFileContents('utf-8')
  .withFilter(async (entry) => {
    // Custom filter: files containing specific content
    if (entry.isFile && entry.content) {
      const hasKeyword = (entry.content as string).includes('TODO');
      await logAnalysis(entry.path, hasKeyword); // Async operation
      return hasKeyword;
    }
    return false;
  })
  .withFilter((entry) => {
    // Multiple filters: also check file size
    return entry.stats.size < 10000;
  })
  .onFile(({ path }) => console.log(`TODO found in: ${path}`))
  .execute('./src');

Concurrent file processing

const tasks: Promise<void>[] = [];

await new FolderWalker()
  .onFile(async ({ path, stats }) => {
    if (stats.size > 1024 * 1024) { // 1MB
      // Process large files concurrently
      const task = processLargeFile(path);
      tasks.push(task);
    }
  })
  .execute('.');

// Wait for all concurrent operations
await Promise.all(tasks);

Filter Order and Performance

Filters are applied in the following order for optimal performance:

1. Built-in filters (fastest, applied first):

   • Hidden files (withHidden())
   • Include/exclude patterns
   • File extensions (withExtensions())
   • Filename patterns (withPattern())
   • File size limits (withMaxFileSize())

2. Custom filters (withFilter()):

   • Applied in the order they were added
   • Support both sync and async operations
   • Can access full file metadata and content

Performance Tips:

• Place expensive filters (file content analysis) last
• Use built-in filters when possible (they're optimized)
• Custom filters receive fully-populated entry objects

License

MIT