aio-lib-jobs
v1.1.1
Published
Submit long-running work from an Adobe App Builder web action, hand it off to a non-blocking worker action, and track its status - so you only write the job logic.
Readme
❯ aio-lib-jobs
Submit long-running work from an Adobe App Builder web action, hand it off to a non-blocking worker action, and track its status — so you only write the job logic.
Built with the tools and technologies:
Table of Contents
- Table of Contents
- Overview
- Features
- Design Notes
- Project Structure
- Getting Started
- Roadmap / Known Limitations
- Contributing
- License
Overview
In Adobe App Builder / I/O Runtime, exposing a long-running process as an API forces you to split work across two actions: web actions are synchronous with a 60 second ceiling, while non-blocking (async) invocations can run up to 180 minutes. Every team ends up hand-rolling the same boilerplate — a web action that accepts a request, fires a non-web worker action non-blocking, returns a job id immediately, and can be polled for status; plus a worker action that does the work and maintains that status.
aio-lib-jobs abstracts that handoff and status-tracking so you only write the job logic. It provides:
- a submit-side client (
init()) for kicking off a job and polling/cancelling/reporting on it, with an optional composable Hono router for the common case, and - a worker-side wrapper (
runWorker()) that turns a plain async function into a status-tracked worker action.
Features
- Non-blocking handoff -
submit()invokes the worker via theopenwhiskclient withblocking: falseand returns immediately with a job id. - Self-deriving job ids -
jobIdis<prefix>.<activationId>, whereprefixis auto-derived from the worker action's own fully-qualified name. No consumer-supplied ids to manage. - Race-free state model -
@adobe/aio-lib-statehas no compare-and-swap, so each piece of job state lives in its own key with exactly one writer: the worker owns the main record, the submitter ownssubmittedAt, and the cancel route owns the cancel flag. Nothing can silently stomp a result. - Cooperative cancellation -
cancelUrlsets a flag; a worker's job function can checkctx.isCancelled()and exit early. OpenWhisk has no API to force-stop a running activation, so this is deliberately best-effort, not a kill switch. - Read-time timing, not heartbeats -
queuedMs,elapsedMs, and a computedstaleflag are derived from stored timestamps at poll time; nothing needs a background heartbeat to detect a dead worker. - Prefix-scoped reporting -
report()lists every job for a given worker action via a single glob-scopedlist()call, including jobs stillqueued. - Bring your own auth -
router()returns a plain composableHonoinstance (it never callsToOpenWhiskAction()itself), so you can wrap it withhono/bearer-author any other Hono middleware before mounting it. - Zero-config status URLs -
statusUrl/cancelUrlare built from__OW_ACTION_NAME's own namespace/package/action segments ashttps://<namespace>.adobeioruntime.net/api/v1/web/<package>/<action>- no base-URL configuration and no header-sniffing. (Deliberately not__OW_API_HOST: that env var is Runtime's internal control-plane host, unreachable from outside the platform - confirmed by an actual deployed round-trip, not just the docs.)
Design Notes
This library assumes exactly one non-blocking worker invocation per job (no fan-out), that authentication on the submit/status/cancel/report routes is the consumer's responsibility, and that a job's result must fit within @adobe/aio-lib-state's 1MB value limit (large results should be written to @adobe/aio-lib-files with only a reference stored in job state).
Project Structure
└── aio-lib-jobs/
├── LICENSE
├── package.json
├── tsconfig.json
├── vite.config.ts
├── src
│ ├── index.ts # Public exports
│ ├── client.ts # init(): submit / getStatus / cancel / report / router
│ ├── worker.ts # runWorker(): status-tracked worker wrapper
│ ├── job-id.ts # Prefix derivation + jobId compose/parse
│ ├── keys.ts # State key builders (main / submittedAt / cancel)
│ ├── timing.ts # queuedMs / elapsedMs / stale computation
│ ├── urls.ts # statusUrl / cancelUrl construction
│ ├── codec.ts # JSON encode/decode for state values
│ ├── errors.ts # JobsLibError + error codes
│ ├── types.ts # Shared domain types
│ ├── state-client.ts # Narrow @adobe/aio-lib-state interface
│ └── ow-client.ts # Narrow openwhisk interface
└── test
├── *.test.ts # Unit tests per module above
└── helpers/ # In-memory fakes for StateClient / OwClientProject Index
Getting Started
Prerequisites
- Runtime: Node.js >= 18 (an Adobe App Builder / I/O Runtime action environment)
- Package Manager: npm
- Peer dependencies:
@adobe/aio-lib-state,openwhisk, and optionallyhono(only needed if you userouter()) - left as peer deps so your project controls their versions.
Installation
npm install aio-lib-jobs @adobe/aio-lib-state openwhisk honoUsage
Submit/poll action (web: "raw", built with Hono via hono-openwhisk-adapter):
import { Hono } from "hono/quick";
import { bearerAuth } from "hono/bearer-auth";
import { ToOpenWhiskAction } from "hono-openwhisk-adapter";
import { init } from "aio-lib-jobs";
const jobs = await init();
const app = new Hono();
app.use("/jobs/*", bearerAuth({ token: process.env.MY_TOKEN })); // optional - see "Auth" below for alternatives
app.route("/jobs", await jobs.router("my-package/my-worker"));
export const main = ToOpenWhiskAction(app);Worker action (non-web, plain params - runs the actual job logic):
import { runWorker } from "aio-lib-jobs";
import { init as initState } from "@adobe/aio-lib-state";
interface ProcessItemsParams {
items: string[];
}
interface ProcessItemsResult {
processed: number;
}
function delay(ms: number): Promise<void> {
return new Promise((resolve) => setTimeout(resolve, ms));
}
export const main = runWorker<ProcessItemsParams, ProcessItemsResult>(async (ctx) => {
let processed = 0;
for (const item of ctx.params.items) {
if (await ctx.isCancelled()) break; // cooperative - the job function opts in to stopping early
await delay(1_000 + Math.random() * 5_000); // stand-in for a slow downstream call (DB write, external API, ...)
processed++;
}
return { processed };
}, { state: await initState() });Validating job params
Neither submit() nor runWorker() validates the shape of params - it's passed straight through, as Record<string, unknown>. That matters here specifically because the submit/poll action and the worker action are two independent action bundles with no compile-time link between them; nothing catches a mismatch except a runtime failure inside the worker. Share one schema between both bundles and validate on each end - Zod works well for this, but any library that throws on failure does (npm install zod):
// schema.ts - imported by both the submit/poll action and the worker action
import { z } from "zod";
export const ProcessItemsSchema = z.object({ items: z.array(z.string()) });
export type ProcessItemsParams = z.infer<typeof ProcessItemsSchema>;Submit side - validate before invoking, so a bad request gets a clear 400 instead of silently reaching the worker. This means wiring the route yourself instead of using jobs.router()'s /submit (its other routes - status/cancel/report - still apply, since submit/getStatus/cancel/report are plain methods on jobs, not tied to the router):
app.post("/jobs/submit", async (c) => {
let body: unknown;
try {
body = await c.req.json(); // throws on malformed JSON - safeParse only validates shape, not syntax
} catch {
return c.json({ error: "invalid JSON body" }, 400);
}
const parsed = ProcessItemsSchema.safeParse(body);
if (!parsed.success) return c.json({ error: parsed.error.flatten() }, 400);
const result = await jobs.submit("my-package/my-worker", parsed.data);
return c.json(result, 202);
});Worker side - validate ctx.params at the top of the handler; a thrown ZodError is caught by runWorker like any other error and recorded as a failed job:
export const main = runWorker<ProcessItemsParams, ProcessItemsResult>(async (ctx) => {
const { items } = ProcessItemsSchema.parse(ctx.params);
// ...
});Enabling both actions
Both the submit/poll action and the worker action need entries in your App Builder app.config.yaml. The submit/poll action must be web: "raw" (required by hono-openwhisk-adapter); the worker action is a plain non-web action, invoked only via the openwhisk client's non-blocking invoke(), never directly over HTTP:
application:
runtimeManifest:
packages:
my-package:
actions:
my-submit-action:
function: actions/my-submit-action/index.ts
web: "raw" # required - see hono-openwhisk-adapter
runtime: nodejs:22
annotations:
require-adobe-auth: true # optional - see "Auth" below
my-worker:
function: actions/my-worker/index.ts
# no `web` key - a plain non-web action, invoked only via `submit()`'s non-blocking invoke()
runtime: nodejs:22
limits:
timeout: 10800000 # up to 10800000ms (180 min) - OpenWhisk's own ceiling for non-blocking actionsAuth
aio-lib-jobs never adds auth itself (see Design Notes) - router() just returns a plain Hono instance, so you're free to protect it however fits your project:
- In code, with any Hono middleware -
hono/bearer-authabove,hono/basic-auth, a custom IMS-token check, etc. Runs inside the action, so you control the failure response shape. - At the platform level, with the
require-adobe-auth: trueannotation above - Adobe I/O Runtime rejects unauthenticated requests before your action code ever runs. No application code needed, but it's an all-or-nothing gate on the whole action, and once set totrueit cannot be set back tofalse.
Both are optional; an unauthenticated router() is reachable by anyone who has the URL, same as any other web: "raw" action.
Testing
Uses the Vitest test framework. Run the test suite with:
npm testRun the whole verification pipeline (typecheck, tests, lint, build) with:
npx tsc --noEmit && npm test && npm run lint && npm run buildRoadmap / Known Limitations
- [ ] Consumer-supplied prefix override (currently auto-derived from the worker action name only)
- [ ] Fan-out: a job orchestrating multiple worker invocations (currently strictly 1 job : 1 invocation)
- [ ] Pagination on
report()(currently single-page, scoped per worker action) - [X] Cooperative cancellation via a dedicated
.cancelkey - [X] Read-time
stale/queuedMs/elapsedMscomputation
Contributing
- Fork the repository and clone your fork locally.
- Create a branch with a descriptive name.
- Make your changes, following the existing seams (pure logic in
src/*.ts, tests intest/*.test.tsusing the fakes intest/helpers/). - Verify:
npx tsc --noEmit && npm test && npm run lint && npm run build. - Commit using Conventional Commits (
feat:,fix:,feat!:/BREAKING CHANGE:footer, etc.) - enforced locally by commitlint via a huskycommit-msghook, and checked again in CI on pull requests. - Open a pull request describing the change and its motivation.
Releasing
Releases are fully automated by semantic-release on every push to main - the commit types since the last release determine the version bump, and it publishes to npm using Trusted Publishing (OIDC), so no NPM_TOKEN is stored anywhere. See .github/workflows/release.yml for the one-time npm-side setup this requires.
License
Licensed under the Apache License, Version 2.0.
