Walk

Directory and file walking utility for node apps

Features:

• Easy to use
• Simple, lightweight, and fast
• Async, sync, streams, and promise api
• Simple filtering with glob pattern matching
• Require or import file contents, or read as stream, buffer, or string
• Resolves require-style path strings
• Normalized file objects with helper methods
• Fast pattern matching via micromatch
• Include and exclude pattern matching options
• Only 2 dependencies: micromatch, lodash

About

I needed a better way to walk directories and read files during build and/or run time. I wanted an api that was simple, supported glob pattern matching like gulp, and returned objects with a similar format as vinyl. This package allows you to simply read any directory (or file), filter results with glob pattern matching, and return an array of file objects. It can also require or import file contents, read them as streams, buffers, or strings, and resolve require-style path strings.

Usage

Add walk as a dependency for your app and install via npm

npm install @danmasta/walk --save

Require the package in your app

const walk = require('@danmasta/walk');

By default walk returns a readable stream interface. You can use the methods: map(), tap(), each(), promise(), then(), and catch() to iterate file objects and return promises

Options

name | type | description -----|----- | ----------- cwd | string | Base directory to start walk from. Default is process.cwd root | string | Directory or file path as root to walk from. This affects the relative paths used in file objects and matching. Default is ./ src | array\|string\|regexp | Micromatch pattern for result filtering by including any matches. Can be a path string, glob pattern string, regular expression, or an array of strings. Defaults to *(../)*(**/)* dot | boolean | Whether or not to include dot files when matching. Default is true ignore | array\|string\|regexp | Micromatch pattern for result filtering by ignoring any matches. Can be a path string, glob pattern string, regular expression, or an array of strings. Defaults to *(../)*(**/)(.git\|node_modules) encoding | string | Which encoding to use when reading or writing file contents. Default is utf8 resolve | boolean | Whether or not to attempt to resolve file paths if not found. Default is true paths | array\|string | Which paths to walk for files. Default is undefined

Methods

Name | Description -----|------------ map(fn) | Runs an iterator function over each file. Returns a promise that resolves with a new array of return values tap(fn) | Runs an iterator function over each file. Returns a promise that resolves with an array of file objects each(fn) | Runs an iterator function over each file. Returns a promise that resolves with undefined promise | Returns a promise that resolves with an array of file objects then(fn) | Run a callback when the promise is fulfilled. Returns a promise that resolves with an array of file objects catch(fn) | Run a callback when the promise is rejected. Returns a promise that resolves with an array of file objects

File Objects

Each file object returned from walk has the following signature:

Properties

name | type | description -----|----- | ----------- cwd | string | Current working directory. Defaults to process.cwd root | string | Base directory to use for relative pathing. Defaults to cwd path | string | Absolute path of the file on disk relative | string | Relative path of file based from normalized root relativeFromCwd | string | Relative path of file based from normalized cwd dir | string | Parent directory where file is located base | string | File name with extension name | string | File name without extension ext | string | File extension stat | object | The fs.stat object for the file contents | string\|object | Contents of the file. Default is undefined encoding | string | Default encoding to use when reading or writing file contents. Default is utf8

Methods

name | description -----| ----------- createReadStream(opts) | Returns a readable stream for the file createWriteStream(opts) | Returns a writable stream for the file append(data, opts) | Appends data to the file read(opts) | Reads the file contents. Returns string or buffer based on encoding readAsString(opts) | Shortcut for read() that ensures encoding is set or throws an error readAsBuffer(opts) | Shortcut for read() that sets the encoding to null readStr | Alias for readAsString readBuf | Alias for readAsBuffer write(data, opts) | Writes data to the file require | Reads the file contents using require import | Reads the file contents using import. Returns a promise requireOrImport | Reads the file contents using import or require based on esm-ness. Returns a promise requireImportOrRead | Reads the file contents using import or require if able, otherwise reads as string or buffer based on encoding. Returns a promise isModule | Returns true if file.contents is a module isBuffer | Returns true if file.contents is a buffer isStream | Returns true if file.contents is a stream isNull | Returns true if file.contents is null isNil | Returns true if file.contents is null or undefined isString | Returns true if file.contents is a string isDirectory | Returns true if the file is a directory isSymbolicLink | Returns true if the file is a symbolic link isBlockDevice | Returns true if the file is a block device isCharacterDevice | Returns true if the file is a character device isFIFO | Returns true if the file is a first-in-first-out (FIFO) pipe isFile | Returns true if the file is a file isSocket | Returns true if the file is a socket isEmpty | Returns true if the file is empty (zero bytes) getEncodingFromBom | Returns the encoding from the file byte order mark if set, otherwise undefined

Sync

This package also supports a fully synchronous api. Instead of requiring the default package just require @danmasta/walk/sync. The api is exactly the same for walking and file objects

Examples

const walk = require('@danmasta/walk');

Walk the current working directory and pipe all files to a destination stream

walk('./').pipe(writeStream());

Walk the current working directory, exclude all .json files

walk('./', { src: '**/*.!(json)' }).then(res => {
    console.log('files:', res);
});

Walk a child directory, include only .json files

walk('./config', { src: '**/*.json' }).then(res => {
    console.log('files:', res);
});

Walk a directory using an absolute path

walk('/usr/local').then(res => {
    console.log('files:', res);
});

Read the contents of all pug files in ./views as a string

walk('./views', { src: '**/*.pug' }).map(file => {
    file.readStr().then(res => {
        console.log('template:', file.path, res);
    });
});

Read the contents of all pug files in ./views as a buffer

walk('./views', { src: '**/*.pug' }).map(file => {
    file.readBuf().then(res => {
        console.log('template:', file.path, res);
    });
});

Require all js files in the ./routes directory and run a callback for each one

walk('./routes', { src: '**/*.js' }).each(route => {
    app.use(route.require());
}).then(() => {
    console.log('all routes loaded');
});

Load all templates from the ./views directory synchronously

const walk = require('@danmasta/walk/sync');

let templates = walk('./views', { src: '**/*.pug' });

console.log('templates:', templates);

Testing

Tests are currently run using mocha and chai. To execute tests run npm run test. To generate unit test coverage reports run npm run coverage

Contact

If you have any questions feel free to get in touch