Directory As Data

NOTE: This is experimental software.

Take a directory of files and turn it into a collection of data.

Getting Started

Given a directory structure as follows:

~/example-directory
 ├── bar.json
 ├── foo-and-then-some.json
 ├─┬ baz-and-stuff\
 │ ├── baz-and-bar.json
 │ └── baz-and-foo.json
 ├─┬ dynamic\
 │ ├── dynamic-js.js
 │ └── dynamic-ts.ts
 └─┬ posts\
   └── hello-world.md

We may build a collection with the following example setup:

// index.ts
import directoryAsData from "@lazylife.ninja/directory-as-data"
import MdToJs from "./utils/md-to-js"


const dataPromise = directoryAsData({
    cwd: __dirname, // optional, default: process.cwd()
    src: "./example-directory",
    resolvers: [
      {
        // Read and parse JSON files
        test: /\.json$/,
        // `file` is a VFile (<https://github.com/vfile/vfile) with empty `contents`
        // options.read is a promisified version of toVFile.read (https://github.com/vfile/to-vfile#tovfilereadoptions-encoding-callback)
        async resolve(file, { read }) {
          await read(file)
          const data = JSON.parse(file.toString())

          return data
        }
      },
      {
        // Dynamically import JS and TS files
        // In this example scripts should export a function that returns a JSON object
        // wrapped in a Promise
        test: /\.[jt]s$/,
        async resolve(file) {
          const { default: run } = await import(file.path)
          const data = await run()

          return data
        }
      },
      {
        // Read and parse markdown files with front matter
        // Create a custom util for liberal choice of markdown engine
        test: /\.md$/,
        async resolve(file, { read }) {
          await read(file)
          const markdown = file.toString()
          const data = MdToJs(markdown)

          return data
        }
      }
    ]

The resulting data looks something like this:

Object {

  /* All files with their resolved data are kept at `result._.files` */

  "_": Object {
    "files": Array [
      VFile {
        "cwd": "~/",
        "data": Object {
          "title": "bar",
        },
        "history": Array [
          "~/bar.json",
        ],
        "messages": Array [],
      },
      VFile {
        "cwd": "~/",
        "data": Object {
          "title": "foo",
        },
        "history": Array [
          "~/foo-and-then-some.json",
        ],
        "messages": Array [],
      },

      /* ... */

    ],
  },

  /* files at the root of the source directory create an entry with the camelcase filename as property name */

  "bar": Object {
    "title": "bar",
  },
  "fooAndThenSome": Object {
    "title": "foo",
  },

  /* first level sub-directories create collection entries */

  "collections": Object {
    "bazPlusMore": Array [
      Object {
        "title": "baz bar",
      },
      Object {
        "title": "baz foo",
      },
    ],
    "dynamic": Array [
      Object {
        "title": "Dynamic Typescipt",
      },
      Object {
        "title": "Dynamic",
      },
    ],
    "posts": Array [
      Object {
        "html": "<p>This is just to say: \\"Hello\\".</p>
",
        "title": "Hello World",
      },
    ],
  }
}

The full example can be found at ./test.

Run the example by cloning this repository, installing dependencies and executing npm test.

API

For lack of documentation please have a look at the Typescript declaration file ./index.d.ts.

Also check out the source located at ./src/index.ts.