@sequenceholdings/storage
v0.1.0
Published
Atlas workspace storage contracts and streaming client
Readme
Atlas storage client
Typed Workspaces V2 metadata and direct-to-GCS sequential resumable transfers. Initial release is pending publication and public-contract review.
import { createStorageClient } from '@sequenceholdings/storage'
const storage = createStorageClient({ baseUrl: atlasUrl, getToken: authenticatedToken })
const workspace = storage.workspace(workspaceId)
const result = await workspace.uploads.put({
path: 'reports/quarterly.pdf', parentId: null,
contentType: 'application/pdf',
}, fileBlob, { onProgress: uploadedBytes => console.log(uploadedBytes) })
// Store result.fileId (and optionally result.versionId) in your application's data model.uploads.start returns an upload ID before transfer. Keep that ID for
uploads.resume(id, originalSource). Source can be a Blob, Web ReadableStream or
Node async iterable. The client retains fixed-size chunks, probes GCS offsets and
retries transient failures. Supply size when known (up to 50 GiB); a stream without
one reserves 1 GiB and fails once it exceeds that. Pass signal to stop transfer, then
either resume or uploads.abort(id) to release the reservation. When put fails after
starting a session it aborts the session for you; pass { keepSessionOnFailure: true }
to keep it and resume later. put generates an idempotencyKey unless you pass
one and reuses it whenever it resends start, which start itself does after a
timeout, a lost reply or a 5xx: the server replays a start that committed, so it
never leaves an unnamed session holding one of your 20 pending uploads per
workspace. Small Blobs can use
{ method: 'simple' }; sequential resumable is the default. Parallel multipart is
not included. Do not log the session URL: it is a bearer credential.
resume and put compute a CRC32C over the whole source as they read it (from
byte zero, including the prefix a resume skips) and send it on completion; Atlas
aborts the session with 409 if the stored object differs, which is how a resume
with a source that yields different bytes is caught. If you transfer bytes yourself
with transferStorageUpload, pass its result: uploads.complete(id, { crc32c }).
Crc32c (incremental update(bytes) / digest()) and crc32c(bytes) are exported
for producers that want to declare checksum at start. The completion field is
optional today and becomes required once in-repo consumers ship this SDK.
Replace a file by supplying fileId and its expectedRevision. Concurrent stale
replacement fails with 409; reload before retrying. Completion is idempotent.
While a just-committed permission change is still being applied, Atlas answers
503 permissions_pending; while other changes hold the workspace, 503
storage_busy. The client retries both on reads, upload start/complete/abort,
createFolder and manifest apply (replay-safe) for ~10 seconds; other mutations are not retried,
because the change may already have been applied, so refresh before repeating one.
createFolder sends an idempotencyKey (generated per call, or pass
{ idempotencyKey } to cover your own retries), so a resend returns the folder
it already created instead of 409. createStorageClient bounds each metadata request with requestTimeoutMs
(default 30 s, 0 disables); a timeout surfaces as request_timeout.
versions, restoreVersion and deleteVersion manage immutable history. Restore
creates a new revision without copying bytes. update renames/moves metadata while
preserving file IDs. trash/restore are reversible; purge and workspace delete
return durable receipts from operations (yours by default; { scope: 'workspace' }
lists everyone's for workspace Admins). Poll these until succeeded; failed runs
requeue automatically, and retryOperation restarts one marked failed after
correcting its cause.
In an Artifact, use the trusted host's Blob transfer bridge:
import { createArtifactStorageClient } from '@sequenceholdings/artifact-studio'
const workspace = createArtifactStorageClient().workspace(workspaceId)
await workspace.uploads.put(input, file, {
onProgress: ({ uploadedBytes }) => updateProgress(uploadedBytes),
})Declare capabilities.storage as a map from workspace UUID to viewer, editor
or admin. The declaration only limits reach; the caller also needs resource
permissions. A folder-only grant is sufficient for that folder's resources.
In a Managed Function, use invocation headers, never a caller-supplied token or URL:
import { createFunctionStorageClient } from '@sequenceholdings/managed-functions'
const storage = createFunctionStorageClient({ baseUrl: configuredAtlasUrl, headers: req.headers })
const file = await storage.workspace(workspaceId).getNode(fileId)User mode is the default. { mode: 'service' } explicitly selects the manifest's
attached platform service account, which needs its own storage grants. Declare
service_account, capabilities.storage and hosted network egress to your Atlas
host and storage.googleapis.com. Delegation expires after 15 minutes; resume in a
fresh invocation when necessary. Signed download leases expire after 60 seconds.
For local integration use the normal local Atlas environment, seq-studio storage
and functions deploy -e local. File references in ORM remain application-owned;
there is no automatic row-to-file authorization or deletion cascade.
