@ossy/media-tasks

Changestream-triggered media processing tasks for the Ossy platform.

Tasks in this package are registered with TaskService.RegisterTask() at API startup and fire automatically when matching ADR 0008 resource events appear on the event store change stream.

Tasks

| Task id | Definition trigger (ceiling) | Description | |---------|------------------------------|-------------| | @ossy/media-tasks/tasks/resize-common-web | file Created + image/* | Common web sizes + BlurHash → storage keys under the source id | | @ossy/media-tasks/tasks/extract-colors | file Created + image/* | Dominant color + palette JSON → storage | | @ossy/media-tasks/tasks/convert-common-image | file Created + image/* (+ primary action) | First-slice format convert via sharp: auto-run for HEIC/HEIF/TIFF/BMP/AVIF and jpeg/png/webp → configurable jpeg/png/webp (default jpeg; skips when already the target). Manual Storage → Tasks invoke accepts any image. Inputs format, quality. | | @ossy/media-tasks/tasks/visual-content-descriptors | file Created + image/* / video/* | OpenRouter vision → title/description/tags/alt JSON → storage. Input model (config or payload); default openai/gpt-4o-mini, overridable per trigger. | | @ossy/media-tasks/tasks/run-prompt | file Created + text/* / image/* / application/json, or @ossy/resources/schema/markdown Created | OpenRouter chat completion. Inputs prompt (required), model (default openai/gpt-4o-mini), optional temperature. File bytes or the Markdown document body are the context. | | @ossy/media-tasks/tasks/convert-common-audio | file Created + audio/*, or on_action | General audio format converter via ffmpeg: MP3 (default), AAC, OGG, WAV, AIFF, FLAC. Inputs format, quality. Skips uploads already in the target container MIME. | | @ossy/media-tasks/tasks/text-to-voice | on_action + Markdown Created + file Created + text/* | OpenRouter TTS → MP3. Inputs text (optional when a resource supplies body/bytes), voice (default en_paul_neutral), model (OpenRouter slug; default mistralai/voxtral-mini-tts-2603). |

All seven set configurable: true. Workspaces arm them on /automation with their own triggers[] (within the ceiling above) and optional placement folders. Until armed, they do not run.

Task outputs

1. Artifact — storage key {resourceId}:task-{concept}-{outputName}; read via GET /r/:id/tasks/:concept/:name
2. Placement — placeTaskArtifacts creates a document only when outputs[].prefix or workspace placementLocations are set (no default folder)

| Task | Output | Placed schema | |------|--------|---------------| | extract-colors | palette | @ossy/media-tasks/schema/extract-colors | | resize-common-web | sizes (+ binary keys) | @ossy/media-tasks/schema/resize-common-web (JSON meta placed; binaries via URL) | | convert-common-image | converted (+ image binary) | @ossy/media-tasks/schema/convert-common-image (JSON meta placed; bytes at …-image) | | visual-content-descriptors | descriptors | @ossy/media-tasks/schema/visual-content-descriptors | | run-prompt | completion | @ossy/platform/schema/text-completion (freeform text/md — a platform schema, not feature-owned) | | convert-common-audio | converted (+ audio binary) | @ossy/media-tasks/schema/convert-common-audio (JSON meta placed; bytes at …-audio) | | text-to-voice | speech (+ audio MP3) | @ossy/media-tasks/schema/text-to-voice (JSON meta placed; bytes at …-audio) |

See TaskOutput.md.

Image placeholders (BlurHash)

resize-common-web stores BlurHash + dimensions on the sizes JSON artifact. Consumers use resizeImageMeta / resizeArtifactSrc from @ossy/resources after media URL attach.

Task contract

Each *.task.js file exports:

export const metadata = {
  id: '@ossy/media-tasks/tasks/my-task',
  triggers: [{ type, event, match?, location? }],
  inputs: [{ name: 'model', type: 'text', default: 'openai/gpt-4o-mini', sources: ['payload', 'config'] }],
  outputs: [{ name, kind: 'location', schemaId, contentType }],
  configurable: true,
}

export async function run ({ event, sdk, inputs }) { ... }

configurable: true opts the task into Automation UI workspace triggers, input defaults, per-trigger overrides, and output placement. Omit it for system tasks. Definition triggers are the capability ceiling; workspaces author their own triggers[]. Each trigger’s config overrides instance config for that match only. run reads the resolved values from inputs.

Integrations

| Id | Credentials | Used by | |----|-------------|---------| | openrouter | OPENROUTER_API_KEY (required); optional OPENROUTER_SITE_URL, OPENROUTER_APP_TITLE, OPENROUTER_BASE_URL | visual-content-descriptors, run-prompt, and text-to-voice via integrations.get('openrouter') |

OpenRouter is OpenAI-compatible (baseURL https://openrouter.ai/api/v1). The VCD model input defaults to openai/gpt-4o-mini. A workspace can set another slug as the instance default, and a different slug on a single trigger (folder or MIME). An invoke payload still wins.

run-prompt sends the resolved prompt as the user message. A matching file is appended as context (text/JSON excerpt, or an image data URL). A Markdown document (@ossy/resources/schema/markdown) sends content.body instead. The completion is stored at {resourceId}:task-run-prompt-completion and placed only when the workspace set a folder. Run now invokes @ossy/media-tasks/actions/run-prompt with the editor values; without a resource id the completion is returned on the task run and not stored.

Provider decision (#898): TTS uses the shared openrouter integration (OpenRouter /audio/speech, OpenAI-compatible). Default model mistralai/voxtral-mini-tts-2603 (in OpenRouter's speech catalog); legacy tts-1 / tts-1-hd aliases map onto that slug. Writes an MP3 artifact. Character usage is not counted against the ADR 0017 LLM token cap.

text-to-voice resolves spoken text from (1) the text input, else (2) Markdown content.body, else (3) text-file bytes. Primary action @ossy/media-tasks/actions/text-to-voice supports manual Storage → Tasks invokes (epic #889 / #934).

Usage metering and a monthly token cap (ADR 0017)

Both OpenRouter tasks read response.usage and return it as result.usage = { model, promptTokens, completionTokens, totalTokens }. TaskRunService forwards this onto MeteringService.record() automatically (any task returning result.usage in this shape gets metered — no per-task wiring needed).

Before calling OpenRouter, each task calls assertLlmUsageBudget() (@ossy/platform/metering) — a hard, fail-closed monthly token cap per workspace (LLM_WORKSPACE_MONTHLY_TOKEN_CAP env var default 2_000_000, overridable per workspace via @ossy/workspaces/actions/set-llm-monthly-token-limit). Once a workspace crosses its effective limit, further runs throw { status: 402, code: 'LLM_TOKEN_QUOTA_EXCEEDED' } — no OpenRouter call made, no tokens spent. Named generically, not after OpenRouter specifically, so swapping the provider later needs no rename. See docs/adr/0017-llm-usage-metering-and-quota.md.

Dependencies

• sharp (peer) — image resizing + metadata
• blurhash — encode compact placeholders after resize
• openai (peer) — OpenAI SDK client pointed at OpenRouter (chat + TTS)
• ffmpeg-static (optional peer; also a media-tasks devDependency so CI/unit tests resolve a binary) — portable ffmpeg for convert-common-audio; otherwise FFMPEG_PATH or system ffmpeg on PATH
• @ossy/platform — StorageClient for reading/writing file bytes

Audio output formats (convert-common-audio)

convert-common-audio converts between these container formats via ffmpeg. Inputs: format (default mp3) and quality 1–100 for lossy codecs (ignored for wav/aiff/flac). Auto-run on audio/* Created skips uploads already in the target container MIME; Storage → Tasks (on_action) always offers the format/quality dialog.

| Format | Codec | Content-Type | Quality mapping (input 1–100) | |--------|-------|--------------|-------------------------------| | mp3 (default) | libmp3lame | audio/mpeg | VBR q:a 0–9 — Math.round((1 - quality/100) * 9) (100 → 0 best) | | aac | aac | audio/aac | Bitrate 64k–320k — 64 + (quality/100) * 256 | | ogg | libvorbis | audio/ogg | VBR q:a 1–10 — max(1, round(quality/10)) | | wav | pcm_s16le | audio/wav | Lossless PCM — quality ignored | | aiff | pcm_s16be | audio/aiff | Lossless PCM — quality ignored | | flac | flac | audio/flac | Lossless — fixed compression_level 5 |

JSON meta artifact converted stores format, contentType, sourceContentType, quality, size, audioKey; binary bytes live at …-audio. Storage → Tasks shows an inline player plus FORMAT · quality N. Binary Content-Type uses magic-byte sniffing (WAV/AIFF/OGG/FLAC/MP3/AAC). DJ-specific preprocessing (BPM/key, Camelot tags, EBU R128 for rekordbox/CDJ USB) is tracked separately in #951 (analyze-dj-audio, not shipped yet).