Forked from https://github.com/hilja/file-batcher All issues should be reported there.

File Batcher

A tool to batch process large quantities of files asynchronously. It provides you some convenient tools to edit Front Matter fueled Markdown files (the stuff that most static site generators use). But is not limited to any specific filetype or format.

It could be used for for example to:

• Search and replace text in multiple files.
• To rename large quantities of files.
• Find unused packages by scanning source files.
• Reformatting chunks of text.
• Analyzing the readability of text in blog posts.
• Minifying files attached to blog posts.

Features

• Is asynchronous.
• Can glob.
• Can do bulk edits without you writing any iterators.
• Provides a sane API to edit complex shapes with (immutability-helper)
• The remove method doesn’t delete the files, but chugs them into the trash.
• Exposes read, write, and remove methods to edit individual posts.
• Is probably fast.

Install

npm i frontmatter-file-batcher

Usage

Imagine having thousands upon thousands of files like this:

---
title: Foo
description: Very Foo
date: "2019-07-12 07:28"
categories:
  - images
  - trains
---
Hello!

We can edit them like so:

import { batch } from "file-batcher";

// A sane limit of 100 concurrent operations.
batch("fixtures/test-content/**", 100, async ({ goods, actions }) => {
  const { author } = goods.data;
  const { update, save } = actions;

  if (!author) return;

  // Update the author’s name with the provided immutability-helper. The update
  // function is prepopulated with the data (`goods`) from the post.
  const newData = update({
    data: { author: { $set: "Slartibartfast" } },
  });

  // At the end you can save your post with the new data.
  await save(newData);

  console.log("Just saved:", goods.path);
});

See examples for more examples.

API

You've got 4 public methods at your disposal: batch, read, remove, and write.

batch(input[, limit, onEach])

Return a promise that resolves to an array with the parsed Front Matter.

input

Type: string|array

If string, it uses uses glob. Or an array of file paths, relative to the current working directory.

limit

Type: string|array Default: Infinite

The concurrency.

onEach(args)

The function that runs on every iteration. It provides you a set of handy tools in the args. See below.

args

Type: object

args.goods

Type: object|any

If the target file is Front Matter, then goods will be an object provided by gray-matter, with a following shape:

{
  content: 'Hello\n',
  data: {
    title: 'foo',
    description: 'bar',
    categories: ['images']
  },
  isEmpty: false,
  excerpt: '',
  path: '/path/to/the/articles/bar.md'
}

If the file is not Front Matter, then goods will be the file’s contents verbatim.

args.actions

Type: object

Actions has all the tools you need to edit/save the files.

args.actions.update(pattern)

Type: function

This is a prepopulated immutability-helper. It's just a tool that provides a syntax for editing complex shapes. Using it is completely optional.

The syntax:

const newData = update({
  data: { author: { $set: "Slartibartfast" } },
});

See more advanced examples in the immutability-helper docs.

args.actions.save(data[, path])

Type: function

Saves the currently iterated file with the data passed to it. See the write method for more info.

data

Type: object

The data to save, in the upper-mentioned shape.

Example:

// Inside the `onEach`.
const capitalize = (string) => string[0].toUpperCase() + string.substring(1);

goods.data.name = capitalize(goods.data.name);

await save(goods);

path?

Type: string

This helper function is prepopulated with with the current file, so, if you’re operating on that file, you don’t need to pass in a path. You can, tho, if you want to save it to a new location.

args.actions.remove(path?)

Type: function Returns: Promise

This one doesn't actually delete anything, but moves it to your computers Trash.

path?

Type: string

It’s prepopulated with the current file, so use this param only if you want to delete another file, that isn’t the one in the iteration.

args.index

Type: number

args.files

Type: array

The original array of files we’re looping over.

delay(milliseconds[, options])

Type: function Return: Promise

See delay.

Throttles the current iteration, good if you’re using somehow constricted API.

Example:

const onEach = ({ throttle }) => {
  // Make only 20 calls per minute.
  await delay(3000)
  // Call your API or whatever.
  const data = fetch('https://example.com/rpm-limited-api')
}

read(file)

Returns: Promise<object>

Reads a file asynchronously and returns its contents. If it’s a markdown file, the Frontmatter will be parsed and it will return the the following shape:

{
  content: '',
  data: {
    description: 'You literally took a shit in the blender',
    title: 'Kitchen nightmares'
  },
  isEmpty: false,
  excerpt: '',
  path: '/path/to/kitchen-nightmares.md'
}

file

Type: string

A path to a file. Relative to the current working directory.

remove(path[, options])

Returns: Promise

Moves a given file into your computers trash, where you can then recover it, if you so like.

path

Type: string

A path to a file. Relative to the current working directory.

options?

Type: object

Options to pass to the underlying library trash.

write(path, data[, options])

An asynchronous function that takes some data and writes it into a file. If the data looks like it's meant to be parsed into a Front Matter, then it will be parsed into Front Matter. E.g. it has a shape: { content: '', data: {} }.

path

Type: string

Where to write the data. Relative to the current working directory.

data

Type: object|any

If it's an object that has data and content in it, then it's parsed into Front Matter. Otherwise it's just written into the file as is.

options.writeFile?

Type: object

Options to pass to Node's fs.readFile.

options.stringify?

Options to pass to gray-matter's stringify method.

Similar packages

If you need something more generic and with more API, then Gulp might be your thing.