local-data

Motivation

I want to use my local file system as a data lake for all my local data of interest.

Usage

Set the LOCAL_DATA_DIR environment variable to specify where your data repositories will be stored:

export LOCAL_DATA_DIR=/path/to/your/data/directory

CLI Commands

# List all objects in a repository
local-data list <repo>

# Add a new object to the repository
local-data add <repo> <objectName> <objectData>

# Check if an object exists
local-data exists <repo> <objectName>

# Get an object from the repository
local-data get <repo> <objectName> [--encoding <encoding>]

# Link a source file to an object in the repository
local-data link <repo> <objectName> <sourceFilePath>

# Synchronize all linked files in the repository
local-data sync <repo>

Programmatic API

import { LocalData } from '@sebastianappelberg/local-data';

const repo = new LocalData('my-repo');

// List objects
const objects = await repo.list();

// Add an object
await repo.add('my-object', 'some data');

// Check if object exists
const exists = await repo.exists('my-object');

// Get an object
const data = await repo.get('my-object');

// Link a file
await repo.link('my-object', '/path/to/source/file');

// Sync all linked files
await repo.sync();

Installation

npm install @sebastianappelberg/local-data

or

yarn add @sebastianappelberg/local-data

or

pnpm add @sebastianappelberg/local-data

License

This project is licensed under the MIT License. See the LICENSE file for details.