librarian

In-memory dependency installer.

How does it work?

Given a package folder (anything containing a package.json file), Librarian will create a virtual fs-like object representing the node_modules folder of the package:

const librarian = require('librarian')
const folderPath = '/path/to/my/folder'
const vfs = await librarian.createVfs('/path/to/my/folder')
vfs.readFileSync(`${folderPath}/node_modules/lodash/index.js`) // Buffer < .. >
vfs.writeFileSync(
  `${folderPath}/node_modules/lodash/index.js`,
  'console.log("foo")' // change virtual contents
)
vfs.copySync(
  '/home/me/my-code/some-lodash-replacement',
  `${folderPath}/node_modules/lodash` // replace virtual contents with a different implementation
)
vfs.copySync(
  `${folderPath}/node_modules`,
  '/some/path/on/my/hd' // dump virtual contents to disk
)

Librarian caches packages on disk, recreating the node_modules folder in memory on runtime. This saves not only disk space, but also installation time compared to traditional package managers, because files do not need to be copied or linked to the hard-drive for each project.

Librarian has a built-in adapter that patches the fs module in order to provide the vfs to a node executable without writing anything to the hd. This is inspired by tink:

const librarian = require('librarian')
const folderPath = '/path/to/my/package'
const testerCode = `
  const leftPad = require('left-pad')
  console.log(leftPad('foo', 5, 0))
`
fs.writeFileSync(`${folderPath}/index.js`, testerCode)
await librarian.runModule(`${folderPath}/index.js`) // "00foo"

In production

Librarian powers the component playground at bit.dev It was developed to provide a fast and smooth installation experience so that developers can create, change and maintain their components.

API

librarian.createVfs(<path>)

Example: vfs = await librarian.createVfs('/path/to/my/module') Returns a Promise that resolves into a virtual filesystem containing the node_modules of the package. path should be a folder containing at least a package.json file. The returned vfs is a memfs instance. The node_modules folder inside vfs will be contained inside the given path, eg.

vfs.readdirSync('/path/to/my/module/node_modules') // [ /* contents of node_modules */ ]
vfs.readdirSync('/node_modules')` // ENOENT

librarian.runModule(<path-to-executable>)

Runs executable in a child process with its fs module with a virtual file system to provide it with its node_modules. Note that the executable should be in a folder containing at least a package.json with the appropriate dependencies.

Returns a node ChildProcess instance.

librarian.runMultipleInstalls([<path-to-folder1>, <path-to-folder2>])

Runs a librarian installation in all folders passed to it concurrently. This means populating the cache with all their dependencies and transitive dependencies, as well as creating a lockfile librarian-manifests.json for each one of them.

Returns a Promise that resolves once all installations are complete.

Testing

npm test

Roadmap

At the moment, Librarian is not a fully-fledged package manager. We believe it is stable enough to be an infrastructure for one, and are now working on adding some missing features. Notably:

• The ability to add packages to a running librarian instance (started with the runModule method). This would be similar to new packages added to the node_modules folder.
• Lazily place files in memory rather than preloading everything for less resource utilization.
• Interface with FUSE for real-time browsing of a virtual node_modules folder.
• The ability to run in the browser

Contributing

We enthusiastically welcome contributions. There is a lot to do, and we have big plans for Librarian. Check out our open issues or start a conversation by opening a new one, or a pull request.

Acknowledgements

Thanks to Josh Vanderwillik for contributing the package name on npm.