dynamodb-toolkit

Opinionated zero-runtime-dependency micro-library for AWS DynamoDB — REST-shaped Adapter, expression builders, batch/transaction chunking, mass operations, and a framework-agnostic HTTP handler. Built on the AWS JS SDK v3.

v3 is a green-field rewrite. v2 consumers stay on v2 ([email protected]). The v3 API, naming, and packaging differ throughout — see Migration: v2 → v3 below and the wiki.

Highlights

• Zero runtime dependencies. AWS SDK v3 modules are peer dependencies (@aws-sdk/client-dynamodb, @aws-sdk/lib-dynamodb).
• ESM-only. Native import / export, hand-written .d.ts sidecars next to every .js file. No build step.
• TypeScript, CommonJS, Node/Deno/Bun — first-class TS typings via sidecars, CJS consumers can require() on current Node 20+, and the test suite runs on all three runtimes (see Compatibility).
• Declarative schema — typed keyFields with composite structural keys, an indices map for GSIs/LSIs (with sparse, indirect, and projection controls), opt-in technicalPrefix for adapter-managed namespaces, typeLabels / typeDiscriminator / typeField for type detection via adapter.typeOf(item) (with auto-populated type tags on write), and a filterable allowlist for the filter URL grammar.
• Adapter with hooks — prepare / revive / prepareKey / validateItem / checkConsistency / updateInput / prepareListInput, single-op → transactWriteItems auto-upgrade, Raw<T> bypass marker, indirect-index second-hop for keys-only GSIs. Canned stampCreatedAtISO() / stampCreatedAtEpoch() prepare-hook builders.
• Expression builders for UpdateExpression, ProjectionExpression, FilterExpression, ConditionExpression, and buildKeyCondition / adapter.buildKey for hierarchical Query key conditions (children by default; {self: true} to include the parent row; {partial} for prefix narrowing).
• Batch + transaction chunking — applyBatch / applyTransaction with UnprocessedItems / UnprocessedKeys retry, exponential backoff, {options} sentinel for transaction-level knobs (clientRequestToken, returnConsumedCapacity), and explainTransactionCancellation to map cancellation reasons back to input descriptors.
• Resumable mass operations — cursor-based {maxItems, resumeToken} across deleteListByParams / cloneListByParams / moveListByParams / editListByParams, opaque encodeCursor / decodeCursor, MassOpResult with {processed, skipped, failed, conflicts, cursor?} buckets, per-item ifNotExists / ifExists conditionals. adapter.edit for read-diff-update, rename + cloneWithOverwrite subtree macros.
• Cascade primitives — deleteAllUnder / cloneAllUnder{,By} / moveAllUnder{,By} rooted at a partial key; gated by an explicit relationships declaration (no cascade inference from composite keys).
• Optimistic concurrency + scope-freeze — opt-in versionField auto-conditions writes on attribute_not_exists(<pk>) OR <versionField> = :observed and bumps on success; opt-in createdAtField + {asOf} mass-op option AND-merges <createdAtField> <= :asOf for replay-safe scans.
• Marshalling helpers — dynamodb-toolkit/marshalling: marshallDateISO, marshallDateEpoch, marshallMap(m, valueTransform?), marshallURL with symmetric unmarshall* pairs.
• Filter + search URL grammar — structured filter clauses via ?<op>-<field>=<value> (eq / ne / lt / le / gt / ge / in / btw / beg / ct / ex / nx, first-character-delimited multi-values); free-form text search via ?search=<query> over the adapter's searchable mirror columns.
• Framework-agnostic REST core + node:http handler — pure parsers/builders/policy plus a standard route pack ready to drop into createServer. Koa / Express / Fetch / Lambda adapters in sibling packages.
• Table provisioning + CLI — dynamodb-toolkit/provisioning ships planTable (read-only plan) and ensureTable (plan + apply) plus verifyTable (structured diff, {throwOnMismatch} optional) driven by the same Adapter declaration. Opt-in descriptor record detects drift DescribeTable can't see. dynamodb-toolkit CLI loads an ESM adapter module and runs plan-table / ensure-table / verify-table.

"Toolkit", not "framework"

The pieces are independent — adopt as much or as little as you need. Every layer has a public surface and is useful on its own:

• Use buildUpdate / buildCondition to prepare a params object, then send it with the raw SDK UpdateCommand. No Adapter in sight.
• Hand-build your own params and pass them to applyBatch / applyTransaction for chunking, UnprocessedItems retry, and exponential backoff.
• Use the Adapter for CRUD + hooks, but swap in your own @aws-sdk/lib-dynamodb Command invocation anywhere you want raw control.
• Take the REST handler or leave it — the Adapter works standalone.

Two concrete payoffs: migration (adopt one piece at a time starting from raw-SDK code) and debugging (peel back layers when something looks off). The boundary between caller code and toolkit machinery stays explicit.

Install

npm install dynamodb-toolkit @aws-sdk/client-dynamodb @aws-sdk/lib-dynamodb

Requires Node 20 or newer (also works on the latest Bun and Deno).

Quick start

import {DynamoDBClient} from '@aws-sdk/client-dynamodb';
import {DynamoDBDocumentClient} from '@aws-sdk/lib-dynamodb';
import {Adapter} from 'dynamodb-toolkit';

const client = new DynamoDBClient({region: 'us-east-1'});
const docClient = DynamoDBDocumentClient.from(client, {
  marshallOptions: {removeUndefinedValues: true}
});

const adapter = new Adapter({
  client: docClient,
  table: 'planets',
  keyFields: ['name'],
  searchable: {climate: 1, terrain: 1}
});

await adapter.put({name: 'Tatooine', climate: 'arid', terrain: 'desert'}, {force: true});

const planet = await adapter.getByKey({name: 'Tatooine'});
// → {name: 'Tatooine', climate: 'arid', terrain: 'desert'}

const page = await adapter.getListByParams({}, {offset: 0, limit: 10});
// → {data: [...], offset: 0, limit: 10, total: N}

Declarative schema + hierarchical keys

For non-trivial data models, declare the structural key and indices up front. The Adapter composes the structural key on writes, strips adapter-managed fields on reads, and enforces the filterable allowlist on the wire.

const adapter = new Adapter({
  client: docClient,
  table: 'rentals',
  technicalPrefix: '_',
  keyFields: ['state', 'city', 'rentalName'], // string shorthand — each ≡ {name, type: 'string'}
  structuralKey: '_sk', // shorthand for {name: '_sk', separator: '|'}
  typeLabels: ['state', 'city', 'rental'],
  typeField: 'kind', // auto-populated on write ('state' / 'city' / 'rental')
  typeDiscriminator: 'kind', // read back what the built-in wrote
  indices: {
    'by-status-date': {
      type: 'gsi',
      pk: 'status', // string shorthand for {name, type: 'string'}
      sk: {name: 'createdAt', type: 'string'}, // full descriptor when you need a non-string type
      projection: 'all'
    }
  },
  filterable: {status: ['eq', 'in'], createdAt: ['ge', 'le', 'btw']},
  relationships: {structural: true}
});

// Query the subtree "all rentals in Austin, TX" — children default:
const page = await adapter.getListUnder({state: 'TX', city: 'Austin'}, {limit: 50});

// Or build the Query params yourself and use with the raw SDK:
const kc = adapter.buildKey({state: 'TX', city: 'Austin'}); // children only
const kcWithSelf = adapter.buildKey({state: 'TX', city: 'Austin'}, {self: true}); // + parent row

// Dispatch by hierarchy level (depth fallback + discriminator override):
adapter.typeOf({state: 'TX', city: 'Austin'}); // → 'city'

// Subtree rename with resumable two-phase idempotent writes:
await adapter.moveAllUnder({state: 'TX', city: 'Austin'}, {state: 'TX', city: 'Dallas'});

Provision the table from the same declaration, either programmatically or via the bundled CLI:

# Preview the CreateTable / UpdateTable plan (read-only, no writes):
npx dynamodb-toolkit plan-table ./my-adapter.js
# → Would CREATE table rentals
# →  + GSI by-status-date (status:HASH, createdAt:RANGE)

# Apply it:
npx dynamodb-toolkit ensure-table ./my-adapter.js

# Drift check:
npx dynamodb-toolkit verify-table ./my-adapter.js --strict

REST handler

import {createServer} from 'node:http';
import {createHandler} from 'dynamodb-toolkit/handler';

const handler = createHandler(adapter, {
  sortableIndices: {name: '-t-name-index'}
});

createServer(handler).listen(3000);

The handler ships a standard route pack — GET / POST /, GET PUT PATCH DELETE /:key, GET DELETE /-by-names, PUT /-load, PUT /-clone, PUT /-move, PUT /-clone-by-names, PUT /-move-by-names, PUT /:key/-clone, PUT /:key/-move — with envelope keys, status codes, and prefixes all configurable via options.policy.

Framework adapters

The bundled dynamodb-toolkit/handler is a pure node:http handler. Framework-specific bindings live in separate packages so the core stays zero-dep — each adapter is a thin wrapper that translates its framework's request/response shape into the toolkit's rest-core parsers + standard route pack. The wire contract (routes, query parameters, envelope keys, error mapping) is identical across all four.

| Package | Runtime / framework | Notes | | ------------------------------------------------------------------------------------ | -------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------- | | dynamodb-toolkit-koa | Koa 2.x | Middleware; koa as peer dep | | dynamodb-toolkit-express | Express 4.x / 5.x | Middleware / Router; express as peer dep | | dynamodb-toolkit-fetch | Fetch API — (Request) => Promise<Response> | Zero-framework; runs on Cloudflare Workers, Deno Deploy, Bun.serve, Hono, Node native fetch server | | dynamodb-toolkit-lambda | AWS Lambda handler | Four event shapes (API Gateway REST / HTTP, Function URL, ALB); ships local-debug bridges for running the handler on real HTTP without sam local |

Sub-exports

The package ships discrete, tree-shakable sub-exports for callers who want only the lower-level surface:

| Sub-export | What's inside | | ------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | dynamodb-toolkit | Adapter, Raw, raw(), stampCreatedAtISO, stampCreatedAtEpoch, ToolkitError + subclasses (CascadeNotDeclared, KeyFieldChanged, NoIndexForSortField, BadFilterField, BadFilterOp, CreatedAtFieldNotDeclared, ConsistentReadOnGSIRejected, TableVerificationFailed), TransactionLimitExceededError, type re-exports | | dynamodb-toolkit/expressions | buildUpdate, addProjection, buildSearch, buildFilterByExample, buildCondition, buildKeyCondition, cleanParams, cloneParams | | dynamodb-toolkit/batch | applyBatch, applyTransaction, explainTransactionCancellation, getBatch, getTransaction, backoff, TRANSACTION_LIMIT | | dynamodb-toolkit/mass | paginateList, iterateList, iterateItems, readList, readByKeys, writeItems, deleteList, deleteByKeys, copyList, moveList, getTotal, encodeCursor, decodeCursor, mergeMapFn, runPaged (plus deprecated aliases: readListByKeys, readOrderedListByKeys, deleteListByKeys, writeList) | | dynamodb-toolkit/marshalling | marshallDateISO, marshallDateEpoch, marshallMap, marshallURL, unmarshall* pairs, Marshaller<TRuntime, TStored> type | | dynamodb-toolkit/paths | getPath, setPath, deletePath, applyPatch, normalizeFields, subsetObject | | dynamodb-toolkit/rest-core | parseFields, parseSort, parseFilter (structured clauses), parseSearch (free-form text), parsePatch, parseNames, parsePaging, parseFlag, buildEnvelope, buildErrorBody, paginationLinks, defaultPolicy, mapErrorStatus, mergePolicy, buildListOptions, resolveSort, stripMount, coerceStringQuery, validateWriteBody | | dynamodb-toolkit/handler | createHandler, matchRoute, readJsonBody | | dynamodb-toolkit/provisioning | planTable, ensureTable, verifyTable, diffTable, planAddOnly, describeTable, executePlan, buildCreateTableInput, buildAddGsiInput, readDescriptor, writeDescriptor, compareDescriptor, extractDeclaration |

A bundled CLI (dynamodb-toolkit) wraps the provisioning helpers for scripted use — see Table provisioning in the wiki.

Full reference docs live in the wiki.

Compatibility

TypeScript. Hand-written .d.ts sidecars ship next to every .js — no build step, no typing-generation round-trip. Adapter<TItem, TKey> binds the item shape to method signatures; buildUpdate<T> / buildCondition<T> preserve caller-supplied params typing. A typed smoke test at tests/test-typed.ts exercises the consumer-facing surface; run it via npm run ts-test (Node 22+; tape-six runs .ts natively — no tsx / ts-node needed).

CommonJS. The package is ESM, but require('dynamodb-toolkit') works from .cjs on current Node 20+ (require(esm) shipped unflagged in Node 20.19 for the 20.x line and 22.12 for 22.x). No await import() needed — the source has no top-level await. A CJS smoke test at tests/test-smoke.cjs demonstrates the main entry and every sub-export; it runs as part of npm test under Node.

Runtimes. Tested on Node, Deno, and Bun. The same source tree runs under all three — .cjs tests are Node-only (require(esm) is the Node-specific story); everything else is portable.

| Runtime | Script | | ------- | ------------------- | | Node | npm test | | Deno | npm run test:deno | | Bun | npm run test:bun |

More detail lives on the Compatibility wiki page.

Migration: v2 → v3

v3 is not a drop-in upgrade. Highlights:

• AWS SDK v3 — @aws-sdk/client-dynamodb + @aws-sdk/lib-dynamodb peer-deps replace aws-sdk. Construct a DynamoDBDocumentClient and pass it as options.client.
• One data format — plain JS via lib-dynamodb middleware. The Raw / DbRaw distinction is gone; Raw<T> is now a single bypass marker (raw(item)).
• Options bags everywhere — put(item, {force: true}) instead of put(item, true); getByKey(key, fields, {consistent: true}); patch(key, patch, {delete: [...]}).
• Hooks renamed — prepareListParams → prepareListInput, updateParams → updateInput. The hooks bag (options.hooks) is the canonical extension point; subclassing still works.
• Patch wire format — _delete / _separator (single underscore) by default; configurable via policy.metaPrefix.
• REST layer split — dynamodb-toolkit/rest-core is framework-agnostic; dynamodb-toolkit/handler is the node:http adapter. Koa lives in a separate package.
• No more makeClient / getProfileName — use @aws-sdk/credential-providers (fromIni, fromNodeProviderChain) directly.

The v2 documentation snapshot lives in the wiki repo at the v2.3-docs git tag. The v2 source code remains available on npm as [email protected] and on GitHub at the matching git tag.

Status

3.x is the current actively-developed line. v2 receives no further changes.

License

BSD-3-Clause.