client-store

A simple data storage library primarily designed to work in web based application environments that use a modern web browser. It uses localStorage or a compatible ClientStorage implementation. It provides a set of functions to store structured data like a database containing tables and supporting queries and standard CRUD operations for data. It provides basic insert/update/delete/query capabilities similar to a database and extends what is available in localStorage. The structured data is stored as serialized JSON in the selected storage engine.

License

• Licensed: MIT license

Credits and Inspiration

• localStorageDB This package was forked from localStorageDB which was created by Kailash Nadh (https://github.com/knadh/localStorageDB) and the code was updated to modern JavaScript standards and TypeScript support added and naming changed except for the public methods. Many thanks to Kailash Nadh for the original implementation and inspiration. The package appears to no longer be maintained and projects that depended on the package needed to evolve in breaking fashion with new features. The original forked code that was the basis for this package is available in this repo under "archive" folder in the localStorageDB.ts_ file and this code will no longer be used as a reference for future development and will NOT be altered. - Initial changes (made in new code forked from localStorageDB.ts into clientStore.ts) include: - Updated to modern JavaScript standards (ES6+, parameters pascalCase etc) - Added JSDoc comments - Renamed ID record identifier from "ID" to "ROW_IDENTIFIER" to be more consistent with SQL naming conventions and change datatype from integer to uuid to help with data management and synchronization so that ID values are globally unique and can be used to synchronize data across multiple clients

Other Storage Options

• IndexedDB
• https://github.com/dreamsavior/Better-localStorage
• lowdb
• local-storage-db
• use-local-storage
• convex-backend
• clientdb
• tiny-localstorage-db
• sqlite3
• localForage - https://github.com/localForage/localForage/blob/master/src/localforage.js

Dependencies

The package has no dependencies other than the browser's localStorage APIs. A compatible ClientStorage implementation can be provided as the storage engine.

NOTE: The maximum storage capacity for localStorage varies across browsers, but a common limit is around 5-10 MB per origin. Some browsers, like Safari, may prompt the user to increase the limit if the initial quota is exceeded, while others may silently fail to store data if the limit is reached.

• localStorage

Features

• Provides ability to store (on the client-side withing web based applications) structured data like a database containing tables and rows of data in a tabular format.
• Supports query operations against the stored data and standard CRUD operations (basic create/read(query)/update/delete capabilities).
• The structured data is stored as serialized JSON in the selected storage engine (localStorage or a custom storage engine).

Feature Roadmap

Near Term

1. Implement support to store JSON objects and arrays as column values.
2. Create a GitHub Actions workflow to automatically run these tests on pull requests
3. Add GitHub action to automatically build and publish the package to npm on push to main
4. Add GitHub Pages to host the documentation

Long Term (Optional)

1. Add support for IndexedDB - see https://github.com/localForage/localForage/blob/master/src/drivers/indexeddb.js
2. Add support for SQLite
3. Add support for remote storage engines and support for sync to remote storage engines

Installation

NPM

npm install @spinningideas/client-store

Run Tests

See testing section below in the README for more information on running tests.

npm run test

Supported Environments

Browsers

Browsers need to support "Local Storage" and "Session Storage" in order for clientStore to function in browser environments.

Usage / Examples

Browser Environment

Creating a database, table, and populating the table

// Initialize. If the storage doesn't exist, it is created
const moviesStore = clientStore("movies", localStorage);

// Or Check if the table exists and if not create the table then setup data. Useful for initial storage setup
if (moviesStore.tableExists("movies") === false) {
  // create the "movies" table
  moviesStore.createTable("movies", [
    "episodeId",
    "title",
    "releaseYear",
    "boxOffice",
    "isBest",
  ]);

  // insert some data
  moviesStore.insert("movies", {
    episodeId: "IV",
    title: "Star Wars: A New Hope",
    releaseYear: 1977,
    boxOffice: 775.4, // box office in millions of dollars
    isBest: false,
  });
  moviesStore.insert("movies", {
    episodeId: "V",
    title: "Star Wars: The Empire Strikes Back",
    releaseYear: 1980,
    boxOffice: 538.4, // box office in millions of dollars
    isBest: true, // The Empire Strikes Back is considered the best
  });
  moviesStore.insert("movies", {
    episodeId: "VI",
    title: "Star Wars: Return of the Jedi",
    releaseYear: 1983,
    boxOffice: 475.1, // box office in millions of dollars
    isBest: false,
  });

  // save the data to localStorage
  // all create/drop/insert/update/delete operations should be committed
  moviesStore.commit();
}

Create and seed sata into Table in one process

// Seed rows of data for pre-population and setup of storage. Useful for initial storage setup
const rows = [
  {
    episodeId: "IV",
    title: "Star Wars: A New Hope",
    releaseYear: 1977,
    boxOffice: 775.4, // box office in millions of dollars
    isBest: false,
  },
  {
    episodeId: "V",
    title: "Star Wars: The Empire Strikes Back",
    releaseYear: 1980,
    boxOffice: 538.4, // box office in millions of dollars
    isBest: true, // The Empire Strikes Back is considered the best
  },
  {
    episodeId: "VI",
    title: "Star Wars: Return of the Jedi",
    releaseYear: 1983,
    boxOffice: 475.1, // box office in millions of dollars
    isBest: false,
  },
];

// create the table and insert records in one go
moviesStore.createTableWithData("movies", rows);

moviesStore.commit();

Alter existing Table to add two new Columns

// If database already exists, and want to alter existing tables
if (!moviesStore.columnExists("movies", "runTime")) {
  moviesStore.alterTable("movies", "runTime", 121);
  moviesStore.commit(); // commit the new columns to storage
}

// Multiple columns can also added at once
if (
  !(
    moviesStore.columnExists("movies", "runTime") &&
    moviesStore.columnExists("movies", "rating")
  )
) {
  moviesStore.alterTable("movies", ["runTime", "rating"], {
    runTime: 121,
    rating: "PG",
  });
  moviesStore.commit(); // commit the new columns to storage
}

Querying Data

// Define query parameters
const queryParams = { releaseYear: 1980 };

// Simple select queries
const movies1980 = moviesStore.query("movies", queryParams);

// Query with multiple conditions
const specificMovie = moviesStore.query("movies", {
  releaseYear: 1977,
  boxOffice: 775.4,
});

// Select all movies (no query parameters)
const allMovies = moviesStore.query("movies");

// Select all movies released after 1979 using a filter function
const newerMovies = moviesStore.query("movies", (row) => {
  // The callback function is applied to every row in the table
  if (row.releaseYear > 1979) {
    // If it returns true, the row is selected
    return true;
  } else {
    return false;
  }
});

// Or with a more concise arrow function
const newerMoviesAlt = moviesStore.query(
  "movies",
  (row) => row.releaseYear > 1979,
);

// Select movies with box office over 500 million, limited to 2 results (start: 0, limit: 2)
const highGrossing = moviesStore.query(
  "movies",
  (row) => row.boxOffice > 500,
  0,
  2,
);

// Select the best movie (using the boolean field)
const bestMovie = moviesStore.query("movies", { isBest: true });

Sorting Data

// Select 2 rows sorted in ascending order by boxOffice
const sortedMovies = moviesStore.query("movies", null, 0, 2, [
  ["boxOffice", "ASC"],
]);

// Select all rows first sorted in ascending order by boxOffice, and then, in descending, by releaseYear
const multiSortedMovies = moviesStore.query("movies", null, 0, null, [
  ["boxOffice", "ASC"],
  ["releaseYear", "DESC"],
]);

// Combine query, limit (start: 0, limit: 1) and sort
const filteredSortedMovies = moviesStore.query(
  "movies",
  { releaseYear: 1980 },
  0,
  1,
  [["boxOffice", "ASC"]],
);

Getting Distinct rows of data

// Get records with distinct releaseYear and boxOffice values
const distinctMovies = moviesStore.query("movies", null, 0, null, null, [
  "releaseYear",
  "boxOffice",
]);

Example Query Results

// Query results are returned as arrays of object literals
// A "ROW_IDENTIFIER" field with the internal auto-incremented identifier of the row is also included
// Thus, ROW_IDENTIFIER is a reserved field name

const bestMovie = moviesStore.query("movies", { isBest: true });
console.log(bestMovie);

/* Results:
[
  {
    ROW_IDENTIFIER: "9c96a0e5-c372-45b6-8456-77e24720fa56",
    episodeId: "V",
    title: "Star Wars: The Empire Strikes Back",
    releaseYear: 1980,
    boxOffice: 538.4,
    isBest: true
  }
]
*/

Updating Data

// Update all movies from 1977 to $800M box office
const updatedCount1 = moviesStore.update(
  "movies",
  { releaseYear: 1977 },
  (row) => {
    return { boxOffice: 800.0 };
  },
);
console.log(`Updated ${updatedCount1} records`);

// Or update all movies released before 1980 to $800M box office
const updatedCount2 = moviesStore.update(
  "movies",
  (row) => row.releaseYear < 1980, // Simplified arrow function with implicit return
  (row) => ({ boxOffice: 800.0 }), // Arrow function with implicit return of object
);
console.log(`Updated ${updatedCount2} records`);

// Don't forget to commit changes
moviesStore.commit();

Upsert - Insert or Update conditionally

// If there's a movie with episodeId VI, update it, or insert it as a new row
const result = moviesStore.upsert(
  "movies",
  { episodeId: "VI" },
  {
    episodeId: "VI",
    title: "Star Wars: Return of the Jedi",
    releaseYear: 1983,
    boxOffice: 500.5, // box office in millions of dollars
    isBest: false,
  },
);

// You can also use upsertOrUpdate which is an alias for upsert
const result2 = moviesStore.upsertOrUpdate(
  "movies",
  { episodeId: "VII" },
  {
    episodeId: "VII",
    title: "Star Wars: The Force Awakens",
    releaseYear: 2015,
    boxOffice: 2068.0,
    isBest: false,
  },
);

// If result is null, insertion failed
// If result is an array, it contains the ROW_IDENTIFIERs of updated rows
console.log(result ? `Updated ${result.length} rows` : "Inserted new row");
console.log(result2 ? `Updated ${result2.length} rows` : "Inserted new row");

moviesStore.commit();

Available Methods

Storing complex objects

While the library is meant for storing fundamental types (strings, numbers, bools), it is possible to store object literals and arrays as column values, with certain caveats. Some comparison queries, distinct etc. may not work. In addition, if you retrieve a stored array in a query result and modify its values in place, these changes will persist throughout further queries until the page is refreshed. This is because clientStore loads and unserializes data and keeps it in memory in a global pool until the page is refreshed, and arrays and objects returned in results are passed by reference.

If you really need to store arrays and objects, you should implement a deep-copy function through which you pass the results before manipulation.

Package Publishing

This package is set up to be published to npm as an ES Module. The package includes the following features:

• ES Modules build for modern bundlers and environments
• TypeScript declaration files
• Tree-shakable exports

Using the Package

ES Modules

import clientStore from "@spinningideas/client-store";

const store = clientStore("myDatabase");

Publishing to npm

To log in to npm and publish your package, you can run the following command directly in your terminal:

npm login

To publish a new version of the package to npm:

1. Update the version in package.json
2. Run tests to ensure everything is working correctly: npm test
3. Build the package: npm run build
4. Publish to npm: npm publish

Alternatively, you can use npm version commands which will handle versioning and tagging:

npm version patch  # for bug fixes
npm version minor  # for new features
npm version major  # for breaking changes
npm publish

Testing

Testing is done via Vitest.

Running Tests

The client-store library uses Vitest for testing. To run the tests, use the following command:

npm run test

Test Coverage

To run tests with coverage reporting, use:

npm run test:coverage

This will run the tests and display a coverage summary in the terminal, showing:

• Percentage of statements covered
• Percentage of branches covered
• Percentage of functions covered
• Percentage of lines covered

Coverage Report

To view a detailed HTML coverage report in your browser, run:

npm run coverage:report

The HTML report provides a detailed view of which parts of your code are covered by tests and which aren't, with color-coded highlighting:

• Green: Code that is covered by tests
• Red: Code that is not covered by tests
• Yellow: Branches that are partially covered

Test Structure

The tests are organized into several sections:

1. Basic Operations: Tests for core functionality like creating stores and tables
2. Storage Operations: Tests for storage-related methods like import/export and storage management
3. CRUD Operations: Tests for data manipulation methods (Create, Read, Update, Delete)
4. Additional Table Operations: Tests for table-specific operations
5. Error Handling: Tests for proper error handling in various scenarios

Adding New Tests

When adding new functionality to the library, please ensure you add corresponding tests to maintain good test coverage. Tests should be added to the appropriate section in tests/client-store.test.ts.

npm package publishing

Pre-requisites

npm pack npm version minor npm publish