@backblaze-labs/b2-action-toolkit — Backblaze B2 GitHub Actions TypeScript Toolkit

Shared TypeScript library for building Backblaze B2 GitHub Actions — providing integrity-correct upload/download, tar+zstd archiving, input parsing with secret masking, and a B2Simulator test harness. Built on @backblaze-labs/b2-sdk. Every action in the Backblaze B2 GitHub Actions suite is a thin workflow-semantics layer on top of this toolkit.

First 10 GB of Backblaze B2 storage is always free.

What it provides

• Integrity-correct transfer — closes B2's multipart SHA-1 gap: uploadFile stream-computes a whole-file SHA-1 and stamps it into fileInfo.large_file_sha1; downloadFile recomputes and rejects corrupt or truncated objects.
• tar + zstd archiving with the base-dir glob model — createArchive / extractArchive backed by streaming tar + native Node node:zlib Zstd; computeBaseDir + resolveFiles give a filesystem-free base-dir contract so archives round-trip to the same absolute path on any runner.
• B2 input parsing + secret masking — getB2Inputs, connectBucket, connectB2; shared helpers multiline, boolInput, intInput, resolvePrefix so every action parses credentials the same way.
• Lifecycle eviction — ensureLifecycleRule (set-once) keeps buckets from billing forever; scoped to the action's prefix, never the whole bucket.
• JSON sidecars + file hashing — putJsonObject / getJsonObject for manifests and latest pointers; hashFile for sha1/sha256 checksums.
• Actionable error helpers — describeB2Error maps TooManyRequestsError, CapExceededError, ChecksumMismatchError to one clear line.
• B2Simulator test harness — makeTestB2 / makeTempDir under the ./test-helpers subpath; in-memory B2, no network, no real bucket.

Install

npm install @backblaze-labs/b2-action-toolkit

Peer / runtime dependencies (declared as dependencies, installed automatically):

| Package | Purpose | |---|---| | @backblaze-labs/b2-sdk | B2 API client + simulator | | @actions/core | Input parsing, secret masking, logging | | tar | Streaming tar v7 |

Node.js >= 22 required (uses native node:zlib Zstd compression and fs.openAsBlob).

Quick start

import {
  getB2Inputs,
  connectBucket,
  uploadFile,
  downloadFile,
} from '@backblaze-labs/b2-action-toolkit';

// Parse + mask credentials from action inputs
const inputs = getB2Inputs();
const bucket = await connectBucket(inputs, 'my-action/1.0.0');

// Integrity-correct upload — stamps large_file_sha1 for multipart objects
await uploadFile(bucket, 'cache/my-key.tar.zst', '/tmp/archive.tar.zst');

// Integrity-correct download — verifies SHA-1 on the fly, rejects corruption
await downloadFile(bucket, 'cache/my-key.tar.zst', '/tmp/restored.tar.zst');

For archiving a glob of paths:

import {
  computeBaseDir,
  resolveFiles,
  createArchive,
  extractArchive,
} from '@backblaze-labs/b2-action-toolkit';

const patterns = ['~/.cache/huggingface/**'];
const base = computeBaseDir(patterns);   // filesystem-free; identical on save + restore

// Save
const files = await resolveFiles(patterns, base);
await createArchive(files, '/tmp/hf.tar.zst', { cwd: base });

// Restore — tree lands back at the original absolute path on any runner
await extractArchive('/tmp/hf.tar.zst', base);

API

All names below are exported from the package root (@backblaze-labs/b2-action-toolkit).

Inputs

| Export | Kind | Purpose | |---|---|---| | getB2Inputs() | function | Parse key-id, application-key, bucket, realm from action inputs; masks the key via core.setSecret | | connectBucket(inputs, ua) | function | Authorize and return a Bucket instance; ua sets the User-Agent | | connectB2(inputs, ua) | function | Same as connectBucket but also returns downloadHost; use when building download URLs | | multiline(name) | function | Newline-split, trimmed, empties dropped — canonical list-of-paths input parser | | boolInput(name, fallback?) | function | Strict true/false (case-insensitive); blank → fallback; throws otherwise | | intInput(name, def, min, max) | function | Bounded integer input; throws on non-integer or out-of-range | | resolvePrefix(opts?) | function | Explicit prefix input wins, else GITHUB_REPOSITORY, else ''; opts.repoFallback controls the fallback | | B2Inputs | type | Parsed credential + bucket config | | B2Connection | type | { bucket, downloadHost } returned by connectB2 | | ResolvePrefixOptions | type | Options for resolvePrefix |

Paths

| Export | Kind | Purpose | |---|---|---| | computeBaseDir(patterns) | function | Derives the absolute common ancestor of all glob patterns — pure, filesystem-free, identical on save and restore | | resolveFiles(patterns, baseDir) | function | Expands ~, globs, and plain paths into a sorted de-duped list of existing paths relative to baseDir; throws on empty result or missing literal path |

Archive

| Export | Kind | Purpose | |---|---|---| | createArchive(paths, outFile, opts?) | function | Streaming tar + zstd compress; opts.level sets compression level, opts.cwd sets the working directory | | extractArchive(file, destDir) | function | Streaming zstd decompress + tar extract into destDir | | ArchiveResult | type | Return value of createArchive | | CreateArchiveOptions | type | Options for createArchive (level, cwd) |

Storage

| Export | Kind | Purpose | |---|---|---| | uploadFile(bucket, key, path, opts?) | function | Integrity-correct upload — stamps large_file_sha1 into fileInfo for multipart objects | | downloadFile(bucket, key, destPath, opts?) | function | Integrity-correct download — stream-verifies SHA-1, rejects corrupt/truncated objects | | objectExists(bucket, key) | function | Returns true if the object key exists | | findLatestByPrefix(bucket, prefix) | function | Returns the most recent FileVersion matching the prefix, or null | | assertSha1Match(actual, stored) | function | Throws ChecksumMismatchError when hashes differ; no-ops when stored value is null/'none' | | putJsonObject(bucket, key, value) | function | Upload a small JSON payload as application/json from a BufferSource | | getJsonObject<T>(bucket, key) | function | Download and parse a JSON sidecar; returns null on genuine miss, propagates parse errors | | hashFile(path, algo) | function | Stream-hash a file with 'sha1' or 'sha256'; returns hex digest | | LARGE_FILE_SHA1_KEY | const | fileInfo key used for whole-file SHA-1 ("large_file_sha1") | | SRC_LAST_MODIFIED_KEY | const | fileInfo key used for source modification time | | UploadFileOptions | type | Options for uploadFile | | DownloadFileOptions | type | Options for downloadFile |

Lifecycle

| Export | Kind | Purpose | |---|---|---| | ensureLifecycleRule(bucket, prefix, days?) | function | Set-once eviction rule scoped to prefix; writes only when absent or changed (cheap read-first) | | DEFAULT_RETENTION_DAYS | const | Default eviction window (7 days) |

Errors

| Export | Kind | Purpose | |---|---|---| | describeB2Error(err) | function | One actionable message per common B2 failure: rate limit, cap exceeded, checksum mismatch, generic fallback |

SDK re-exports

These are re-exported so action repos don't need a direct @backblaze-labs/b2-sdk dependency.

| Export | Kind | Purpose | |---|---|---| | Bucket | type | B2 bucket handle | | FileVersion | type | B2 file version metadata | | LifecycleRule | type | B2 lifecycle rule shape | | B2Error | class | Base B2 error class | | TooManyRequestsError | class | Rate-limit error (check retryAfter) | | CapExceededError | class | Account storage/download cap error | | ChecksumMismatchError | class | In-transit corruption error | | classifyError(err) | function | Classify an unknown error into a known B2 error type |

Testing with the B2Simulator

Import the test harness from the published ./test-helpers subpath — all action repos share this instead of duplicating the wiring:

import { makeTestB2, makeTempDir } from '@backblaze-labs/b2-action-toolkit/test-helpers';
import { describe, it, expect } from 'vitest';

describe('my action storage logic', () => {
  it('round-trips a file with integrity', async () => {
    const { bucket } = await makeTestB2('my-bucket');
    const tmp = await makeTempDir();

    await uploadFile(bucket, 'test/key', `${tmp}/input.bin`);
    await downloadFile(bucket, 'test/key', `${tmp}/output.bin`);
    // assert...
  });
});

makeTestB2 wires a B2Simulator with lowered recommendedPartSize/minimumPartSize so multipart paths are exercised cheaply. Use sim.injectFailure(...) for retry/error-path coverage — see ARCHITECTURE.md.

Used by

These GitHub Actions are all thin layers on this toolkit:

| Action | Description | |---|---| | b2-cache | Backblaze B2 cache action for GitHub Actions — save and restore build caches | | b2-artifact | Backblaze B2 artifact upload/download for GitHub Actions | | b2-eval-report | Backblaze B2 evaluation report storage for GitHub Actions ML pipelines | | b2-dvc-remote | Backblaze B2 DVC remote setup action for GitHub Actions | | b2-hf-cache | Backblaze B2 Hugging Face model cache action for GitHub Actions | | b2-model-publish | Backblaze B2 model publishing action for GitHub Actions | | b2-release-assets | Backblaze B2 release asset upload action for GitHub Actions |

Conventions

Every action in the suite follows the layout, build, test, and naming rules documented in ARCHITECTURE.md — the single source of truth for how the toolkit and each action are structured, how the base-dir contract works, and how the integrity rule is applied.

License

MIT © Backblaze, Inc.