@render-lab/tasks-llm

⚠️ Experimental: proof of concept. This package is part of the Render Tasks POC and is published for testing only. It is not fully tested or production ready. Task names, inputs, outputs, and behavior can change or break in any release. Pin exact versions and expect breaking changes.

Durable, provider-agnostic LLM tasks for Render Workflows. Import the tasks, call them from your own workflow, get retries and dashboard lineage for free.

import { classify, summarize, complete, extract, embed } from "@render-lab/tasks-llm";

Nine namespaced tasks:

| Task | Input | Output | | --------------- | ------------------------------------------------------ | --------------------------------------------------- | | llm.classify | { text, labels: {name, description?}[], maxLabels? } | { labels: string[], reasoning?, model } | | llm.summarize | { text, instructions?, maxWords? } | { summary, model } | | llm.complete | { prompt, system?, maxTokens? } | { text, model, stopReason? } | | llm.extract | { text, schema, instructions?, maxTokens? } | { data, model } | | llm.chat | { messages: {role, content}[], system?, maxTokens? } | { text, model, stopReason? } | | llm.rerank | { query, documents: string[], topN? } | { ranked: {index, score}[], model } | | llm.moderate | { text, categories? } | { flagged, categories: string[], reasoning?, model } | | llm.translate | { text, targetLang, sourceLang? } | { text, targetLang, model } | | llm.embed | { texts: string[], model? } | { embeddings: number[][], model } |

stopReason is the provider stop cause normalized to "end" | "length" | "refusal" — "length" means the output hit maxTokens and is truncated (note that models with thinking enabled spend thinking tokens from the same budget, so this can happen well below the visible text length).

The wrapped task for llm.chat is exported as chatTurn (the name chat is the provider-agnostic adapter). llm.extract and llm.rerank ask the model for JSON and parse it with extractJson; extract throws if nothing parses.

llm.embed batches texts into one provider call and returns one vector per text, in input order. It defaults to openai/text-embedding-3-small (LLM_EMBED_MODEL to override) rather than LLM_MODEL, because embeddings models are a separate family. Anthropic has no embeddings endpoint, so anthropic/* models throw — use an openai/* model or an LLM_BASE_URL gateway.

Every task also takes an optional model field (see below) and returns the resolved model id.

Install

pnpm add @render-lab/tasks-llm @renderinc/sdk

@renderinc/sdk is a peer dependency — you install one copy, and every task package registers against that single copy's TaskRegistry. The vendor SDKs (@anthropic-ai/sdk, openai) are regular dependencies, fully encapsulated inside the task functions.

Environment contract

| Variable | Required | Purpose | | ------------------- | ------------------------------- | ----------------------------------------------------------------------- | | ANTHROPIC_API_KEY | for anthropic/* models | Anthropic API key (read by the Anthropic SDK). | | OPENAI_API_KEY | for openai/* models | OpenAI API key (read by the OpenAI SDK). | | LLM_BASE_URL | optional | Route all calls through an OpenAI-compatible gateway (OpenRouter, LiteLLM proxy, self-hosted). | | LLM_API_KEY | optional | Key for the gateway when LLM_BASE_URL is set (falls back to OPENAI_API_KEY). | | LLM_MODEL | optional | Default model when a call doesn't pass one. Defaults to anthropic/claude-opus-4-8. | | LLM_EMBED_MODEL | optional | Default embeddings model for llm.embed. Defaults to openai/text-embedding-3-small. | | REDIS_URL | only when a ledger is used | Connection string for the run-scoped cost ledger (see Cost tracking). Same contract as @render-lab/tasks-render-kv. Read lazily on the first ledger write; never required if you don't pass ledger. | | LLM_PRICING | optional | JSON object of { "provider/model": { inputUsdPerMTok, outputUsdPerMTok? } }, merged over (and overriding) the bundled pricing table. Malformed JSON throws at first use. | | LLM_COST_LEDGER_TTL_SECONDS | optional | TTL backstop for a cost ledger key, refreshed on every append. Defaults to 172800 (48h) so a crashed run's ledger cleans itself up. |

Provider-agnostic by design

The model is a provider-prefixed string, so the same llm.* task names work across vendors without changing your workflow:

await classify({ text, labels, model: "anthropic/claude-opus-4-8" }); // Anthropic SDK
await classify({ text, labels, model: "openai/gpt-4o" });             // OpenAI SDK
// With LLM_BASE_URL set, the full string is passed through to the gateway:
await classify({ text, labels, model: "anthropic/claude-3.5-sonnet" }); // via OpenRouter, etc.

Agnosticism lives in the task signature, not the caller — the impl can swap providers without breaking the task names your other workflows depend on.

Skills (Agent Skills as prompt context)

An Agent Skill is a SKILL.md instruction file; harnesses like Claude Code "run" one by loading its text into the model's context. This pack does that mechanical part for durable workflows — pull any skill you point it at, pinned, cached per process, injected verbatim:

// Inline on the free-form tasks:
await complete({
  prompt: "Edit this draft:\n\n" + draft,
  system: "House rules that win on conflict.",   // appended AFTER the skill text
  skills: [
    "no-ai-slop",                                            // installed via `npx skills add` (see below)
    "file:skills/tone/SKILL.md",                             // vendored in your service repo (cwd-relative)
    "github:petergyang/no-ai-slop@a80787f49925cb343f626b39", // pinned to a commit sha
    "https://example.com/skills/tone/SKILL.md",              // any URL serving the text
  ],
});

// Or standalone, to compose with anything else:
const skill = await loadSkill({ source: "github:anthropics/skills@<sha>#writing/humanize/SKILL.md" });
await extract({ text, schema, instructions: skill.text.slice(0, 2000) });

The bare-name form plugs into the skills CLI ecosystem — install once, commit, reference by name:

npx skills add petergyang/no-ai-slop --copy -y   # lands in .agents/skills/no-ai-slop/ (and per-agent dirs)
git add .agents .claude && git commit            # the commit IS the pin; `npx skills update` + review to bump

At runtime a bare name resolves <dir>/<name>/SKILL.md across SKILLS_PATH (path-delimiter separated) or the defaults .agents/skills → .claude/skills → skills/. The pack never shells out to the CLI and never touches the network for installed skills — it just reads the directory convention. Prefer --copy over symlinks so deploy tarballs don't need link support.

Semantics and limits:

• Order: skills are injected in list order, before the caller's system — your system comes last and wins on conflict.
• Pin your sources. github: requires an explicit @ref; use a full commit sha, not a branch — a skill is part of your prompt, treat it like a dependency. Prefer file: (vendored) to keep third-party repos out of the runtime prompt path entirely.
• Caps: text is capped at 32k chars (maxChars to change) so a runaway skill can't blow the prompt budget.
• Cached per process by source; pinned sources make the cache trivially correct.
• Instruction injection only (ADR-0026): no skill script execution, no references/ browsing — that needs a sandboxed agent loop (Claude Agent SDK), not a durable task.

Cost tracking

Every llm.* task takes an optional ledger?: string. Absent, zero cost-tracking code runs — nothing is constructed, nothing is written. Set it, and after the provider call the task appends one usage record to a run-scoped ledger in Render Key Value, so a deeply composed run (agent.loop: N steps × llm calls) still gets a complete breakdown — the only obligation on a task that composes llm.* calls is to forward the ledger id, never to fold usage itself. @render-lab/tasks-agent's loop / step / plan / reflect / route / compressHistory do exactly this (see its README's "cost tracking" note).

const ledger = await llm.openCostLedger();
const $llm = llm.withLedger(ledger);

const labels = await $llm.classify({ text, labels: LABELS });
const run = await agent.loop({ goal, ledger: ledger.ledgerId });

const cost = await $llm.costReport({ close: true });
// cost.lines  → [{ task: "llm.classify", model, calls, inputTokens, outputTokens, estimatedCostUsd }, ...]
// cost.totals → { calls, inputTokens, outputTokens, estimatedCostUsd }

• llm.openCostLedger() → { ledgerId } mints a replay-stable id (it's a task, so the SDK's checkpoint cache memoizes it — a workflow replay reuses the same id instead of minting a new one). You can also skip it and pass any string you already own (an issue id, a cron timestamp) as ledger — uniqueness is then yours to guarantee.
• llm.withLedger(ledgerOrId) is a pure helper (not a task): it returns the same registered llm.* tasks with ledger curried into every input. No re-registration, no new task names.
• llm.costReport({ ledger, close? }) → { lines, totals } reads the ledger and aggregates per (task, model). close: true deletes the key after reading; otherwise the TTL backstop cleans it up, so you can pull a report mid-run without destroying the ledger.

Semantics — read before you wire this into anything cost-sensitive:

• Estimates, not invoices. estimatedCostUsd comes from a bundled pricing table keyed by provider-prefixed model id, overridable via LLM_PRICING. A model not in the table reports inputTokens/outputTokens with estimatedCostUsd: undefined — never guessed.
• Appends are best-effort. A KV write failure logs a warning and does not fail the task — failing would trigger a durable retry that re-runs the provider call, spending real money again to fix an observability write. A KV outage can drop lines from the breakdown; it can never re-bill or break a workflow.
• Retries are visible. Each attempt that reaches the provider appends its own record, so a task that fails and is retried shows up as multiple lines reflecting actual spend — something usage carried only on the result DTO structurally can't see.
• The composition rule: accept ledger, forward ledger. Any pack composing llm.* calls should add ledger?: string to its own inputs and pass it straight through, unfolded. A pack that forgets shows up as visibly missing lines in the breakdown (detectable), not a silently low total.

Durability & retries

Each task ships a retry policy tuned for provider rate limits (maxRetries: 5, waitDurationMs: 2000, backoffScaling: 2 → ~2s, 4s, 8s, 16s, 32s). The vendor SDK's own client-side retries are disabled so the durable Workflows retry is the single source of truth: a 429 throws, the run fails, and Render re-runs it with the backoff above — every attempt a visible run in the dashboard.

Extending

Every operation exports the wrapped task and the raw impl:

import { classify, classifyImpl } from "@render-lab/tasks-llm";

• Wrap — chain classify(...) from your own task and add logic around it. Keeps upgrades.
• Reuse the impl — build your own task around classifyImpl with your own name and retry policy. The impl accepts an optional second argument { chat } for dependency injection.
• Eject — copy the source (one task per file) into your repo and own it. Never re-register under the llm.* name; always rename.

Testing

The impls take an injectable chat function, so they unit-test with no network and no API keys:

import { classifyImpl } from "@render-lab/tasks-llm";

const res = await classifyImpl(
  { text: "app crashes on launch", labels: [{ name: "bug" }] },
  { chat: async () => ({ text: '{"labels":["bug"]}', model: "test", provider: "anthropic" }) },
);
// res.labels === ["bug"]

Run the package tests with pnpm -C packages/tasks-llm test.