bmcrusher-node

For transferring crushed data between app script and node using various plugins.

Installation

yarn add bmcrusher-node

the crusher

This is a key value store client, with various back ends as plugins. Values are compressed, and if necessary spread over multiple physical records in the backend store depending on its capabilities. Different backends are supported by plugins.

This is the Node client. Alternative clients allow you to share data across multiple platforms. In addition to the Node client, there is also an Apps Script client which has a set of plugins for multilple backends. So sharing data between Apps Script and Node is just a matter of using the appropriate plugin.

There are only 3 methods in a client once intialized to a back end plugin.

put

Put a value. If it's too big it'll compress it and then split it into pieces. An optional expiry time is available to limit the lifetime of a kv pair. If the back end supports automatic lifecycle housekeeping (for example redis, Apps Script cache service and google cloud storage) expired items will be automatically removed sometime after expiry. Irrespective of the back end capabilities, items are never returned if they have expired.

The key should be a string, and the value can be either a string or a stringifiable object.

crusher.put(key, someValue  [,expiryTimeInSeconds])

get

Get a value. If it's in pieces it will reconstitute it to the original. If it's expired or doesn't exist, it'll return null. If the original input was an object, the object will be recreated.

crusher.get(key)

remove

Remove a value

crusher.remove(key)

Coming soon

Other values such as blobs etc will be supported shortly too. That will allow entire files such as images to be cached and reconstitued between platforms, as will various file conversions (for example between Google Sheets and Excel, or Google Docs and PDF). Watch this space for additional plugins to support these capabilities.

Plugins

Google Apps Script has a selection of supported plugins

some of which are specific to Apps Script platform

• CacheService
• PropertyService

and others which are generic and are/will be implemented for Node as well

• Upstash (redis/graphql)
• Github
• Drive
• Google cloud storage
• One Drive

For details on Apps Script implementations, see https://ramblings.mcpher.com/apps-script/apps-script-cache-crusher/

Some of the Node plugins will be implemented and built in to this module, but you can easily build your own for alternative backends.

CrusherPluginUpstashService

Uses Upstash as a redis backend.

For setting up Upstash and Apps script see https://ramblings.mcpher.com/apps-script/apps-script-cache-crusher/upstash/

You'll need an Upstash account and credentials.

Node usage

First get your upstash credentials. Choose the appropriate credential depending on whether you are reading or read/writing.

const upstashrw = "xxx";
const upstashr = "xxx";

Initialize the crusher

This is a similar pattern and options as described in the Apps Script writeup in https://ramblings.mcpher.com/apps-script/apps-script-cache-crusher/upstash/.

At a minumum you should provide a token service function that returns your upstash key. I also recommend a prefix to be applied to cache keys in case you want to use the same Upstash store for something else at some point.

const { CrusherPluginUpstashService } = require("bmcrusher-node");
const { upstashrw } = require("./private/secrets");

const crusher = new CrusherPluginUpstashService().init({
  tokenService: () => upstashrw,
  prefix: "/crusher/store"
});

Now you can use the the standard crusher.get, crusher.put and crusher.remove methods.

CrusherPluginGitService

Uses Github as a backend.

For setting up Github and Apps script see https://ramblings.mcpher.com/apps-script/apps-script-cache-crusher/github/

You'll need a github token. The simplest is to create a personal token, since this is to be server based. For a client version, you'd want to create an oauth flow. Here's the docs for how to create a personal access token https://docs.github.com/en/github/authenticating-to-github/creating-a-personal-access-token

Node usage

First get your token, created with repo scope.

const crusherGit = "gxxxxxxxx";

Initialize the crusher

This is a similar pattern and options as described in the Apps Script writeup in https://ramblings.mcpher.com/apps-script/apps-script-cache-crusher/github/

At a minumum you should provide a token service function that returns your github token. I also recommend a prefix to be applied to cache keys in case you want to use the same repo for something else at some point.

const { CrusherPluginGitService } = require("bmcrusher-node");
const { crusherGit } = require("./private/secrets");

const crusher = new CrusherPluginGitService().init({
  tokenService: () => crusherGit,
  prefix: "store",
  repo: "-crusher-store",
  owner: "brucemcpherson"
});

Now you can use the the standard crusher.get, crusher.put and crusher.remove methods.

CrusherPluginGcsService

Uses Google Cloud Storage as a backend.

For setting up Github and Apps script see https://ramblings.mcpher.com/cacheplugins/apps-script-and-node-gcs/

You'll need a service account .json file with enough scope to write to the cloud storage bucket you're using for this. See https://ramblings.mcpher.com/apps-script/apps-script-cache-crusher/gcs/

Node usage

First get your service account credentials using whichevery method you prefer. I usually do something like this

const _credentials = require("./bmcrusher-test-xxxxxxx.json");
const getGcpCreds = () => _credentials;

Initialize the crusher

This is a similar pattern and options as described in the Apps Script writeup in https://ramblings.mcpher.com/apps-script/apps-script-cache-crusher/gcs/

At a minumum you should provide a token service function that returns your service account credentials. I also recommend a prefix to be applied to cache keys in case you want to use the same bucket for something else at some point.

const { CrusherPluginGcsService } = require("bmcrusher-node");
const { getGcpCreds } = require("./private/secrets");

const gcsCrusher = new CrusherPluginGcsService().init({
  tokenService: () => getGcpCreds(),
  prefix: "/crusher/store",
  bucketName: "bmcrusher-test-bucket-store",
});

Now you can use the the standard crusher.get, crusher.put and crusher.remove methods.