AEY Cloud — TypeScript SDK

Computer vision over images, video and cameras. Run a model from the catalogue — detection, segmentation, keypoints, embeddings — and get its own output back, or describe what matters as rules and get back only the events you asked for.

Paid from prepaid credits: every run and event records exactly what it cost. Queue time is reported and never charged, and failed runs are free.

Install

npm install @aeyvision-cloud/sdk

No dependencies — it uses the platform's own fetch.

A key

Create one in the dashboard at https://aeyvision.com, or with POST /v1/keys. The client reads AEYVISION_API_KEY from the environment if you do not pass one — the form that works in a container without putting a credential in an argument list.

import { AeyVision } from "@aeyvision-cloud/sdk";

const cv = new AeyVision({ apiKey: "aey_..." });

What it can do

import { AeyVision } from "@aeyvision-cloud/sdk";

const cv = new AeyVision(); // reads AEYVISION_API_KEY

Browse the catalogue. tasks explains what each capability returns and which parameters it honours; models is the menu.

const catalogue = await cv.listCatalogue();

for (const task of Object.values(catalogue.tasks ?? {})) {
  console.log(task.label, "—", task.summary);
}

for (const model of catalogue.models ?? []) {
  if (model.available) console.log(model.id, model.task, model.licence);
}

Detect objects in a clip.

const answer = await cv.createRun({
  model: "aey-nano",
  video_url: "https://example.com/forecourt.mp4",
  params: { classes: ["person", "car"], confidence: 0.4, sample_fps: 2 },
});

console.log(answer.run?.cost?.priceUsd, "for", answer.run?.frames, "frames");

for (const frame of answer.output?.frames ?? []) {
  for (const found of frame.detections ?? []) {
    console.log(frame.index, found.label, found.confidence, found.box);
  }
}

Ask a question instead of reading boxes. Rules turn detections into events; level is null when nothing you asked about happened.

const verdict = await cv.analyse({
  video_url: "https://example.com/yard.mp4",
  areas: [{ name: "driveway", points: [[0, 0.5], [1, 0.5], [1, 1], [0, 1]] }],
  rules: [{ when: { object: "car", state: "moving", area: "driveway" }, raise: "alert" }],
});

if (verdict.level === "alert") {
  for (const event of verdict.events ?? []) console.log(event.message);
}

Segment things with a text prompt, using AEY Vision Prompt Seg.

await cv.createRun({
  model: "aey-prompt-seg",
  video_url: "https://example.com/site.mp4",
  params: { prompt: ["forklift", "hard hat", "spilled liquid"] },
});

Redact footage before you share it: everyone blurred but the people you leave visible. The pixels are destroyed, not drawn over, and audio is removed.

const { upload } = await cv.createUpload(await readFile("door.mp4"), { filename: "door.mp4" });
let { redaction } = await cv.createRedaction({ upload_id: upload!.id! });
const wait = async (until: string) => {
  while (redaction!.status !== until) {
    await new Promise((resolve) => setTimeout(resolve, 5000));
    ({ redaction } = await cv.getRedaction(redaction!.id!));
  }
};

await wait("review");
// Every person and vehicle, as a track. Leave the first person visible.
const { analysis } = await cv.getRedactionAnalysis(redaction!.id!);
const keep = analysis!.tracks!.filter((t) => t.label === "face").slice(0, 1).map((t) => t.id!);
await cv.renderRedaction(redaction!.id!, { review: { keep, boxes: [] } });

await wait("ready");
await writeFile("door-redacted.mp4", Buffer.from(await cv.getRedactionVideo(redaction!.id!)));

Processing price

const cost = answer.run?.cost;

console.log(cost?.seconds, "s on", cost?.gpu);
console.log("price per hour", cost?.pricePerHour);
console.log("charged", cost?.priceUsd);
console.log("queued", cost?.queuedSeconds, "s — not billed");

Full logs of what the container did — the same ones the console renders:

const log = await cv.getRunLogs(answer.run!.id!);
for (const line of log.lines ?? []) console.log(line.t, line.level, line.message, line.fields);
for (const stage of log.stages ?? []) console.log(stage.name, stage.ms, "ms");

And spend across the account:

const usage = await cv.getUsage({ days: 30 });
console.log(usage.total?.runs, "runs,", usage.total?.priceUsd, "USD");

Watch out for

Two-stage models such as aey-object-search run their second stage once per detected object, not once per frame. A busy car park costs meaningfully more than an empty drive. The catalogue flags these with scales_with_objects, before the run rather than on the invoice.

sample_fps is the biggest lever on cost. It defaults to five frames a second and goes from 0.1 to 30; two answers most questions, and ten costs five times as much to answer them slightly better. output_fps sets the frame rate of the annotated clip, 1–60 and 25 by default, and is free: frames between analysed ones carry the latest result.

Where it runs

Node 18+, browsers, Cloudflare Workers, Deno, Bun — anywhere with fetch. Pass your own if you need an instrumented one:

const cv = new AeyVision({ fetch: myInstrumentedFetch, timeoutMs: 120_000 });

In a browser, never ship a key to the client. @aeyvision-cloud/sdk/proxy is a ready-made server route that forwards your pages' requests with the key added:

// app/api/aey/[...path]/route.ts (Next.js)
import { createProxy } from "@aeyvision-cloud/sdk/proxy";

const proxy = createProxy({ prefix: "/api/aey" }); // reads AEYVISION_API_KEY; GET only by default
export { proxy as GET, proxy as POST };

It is what the @aeyvision/ui components talk to. See https://aeyvision.com/docs/ui-components.

Every method

Training

| Method | Endpoint | What it does | | --- | --- | --- | | addTrainingImage | POST /v1/training/datasets/{id}/images | Add an image and its boxes. JSON, or the image as the body with ?split= and ?annotations= (URL-encoded JSON). | | bulkTrainingImages | POST /v1/training/datasets/{id}/images/bulk | Accept, reject, delete or move many images: by imageIds, or every image with a status. | | cancelTrainingJob | POST /v1/training/jobs/{id}/cancel | Stop a job. Charged for the time it ran; the rest of the hold is returned. | | createTrainingDataset | POST /v1/training/datasets | Create a trainingDataset. | | createTrainingJob | POST /v1/training/jobs | Fine-tune an AEY Vision detector on a dataset. Sets aside the most it can cost, and charges for the time it uses. | | createTrainingVersion | POST /v1/training/datasets/{id}/versions | Keep the dataset as it is now: its classes and every checked image's labels. | | deleteTrainingDataset | DELETE /v1/training/datasets/{id} | Delete a trainingDataset. | | deleteTrainingImage | DELETE /v1/training/datasets/{id}/images/{imageId} | Remove an image from a dataset. | | deleteTrainingJob | DELETE /v1/training/jobs/{id} | Delete a finished job. Its model stays. | | deleteTrainingVersion | DELETE /v1/training/datasets/{id}/versions/{versionId} | Delete a version. | | estimateTraining | POST /v1/training/estimate | What a training job would set aside and most likely cost, without starting it. | | exportTrainingDataset | GET /v1/training/datasets/{id}/export | A zip of the dataset, or of one version, in COCO or YOLO layout. | | getAutolabel | GET /v1/training/datasets/{id}/autolabel | The dataset's latest auto-labelling job and its progress. | | getTrainingDataset | GET /v1/training/datasets/{id} | Get one trainingDataset. | | getTrainingEvaluation | GET /v1/training/jobs/{id}/evaluation | Per-class results on the held-out images, and what the model saw in each. | | getTrainingImageFile | GET /v1/training/datasets/{id}/images/{imageId}/file | The image itself. | | getTrainingJob | GET /v1/training/jobs/{id} | A job's progress, metrics, charge and model. | | listTrainingDatasets | GET /v1/training/datasets | List trainingDatasets owned by the account. | | listTrainingImages | GET /v1/training/datasets/{id}/images | Every image in a dataset, with its boxes. | | listTrainingJobs | GET /v1/training/jobs | Training jobs, newest first. | | listTrainingVersions | GET /v1/training/datasets/{id}/versions | The dataset's versions, newest first. Every training job saves one. | | startAutolabel | POST /v1/training/datasets/{id}/autolabel | Have AEY Vision Prompt Seg (by description) or a model propose labels. Each image is an ordinary, charged run; proposals wait for review. | | stopAutolabel | POST /v1/training/datasets/{id}/autolabel/stop | Stop auto-labelling after the images in hand. | | updateTrainingDataset | PATCH /v1/training/datasets/{id} | Update a trainingDataset. | | updateTrainingImage | PATCH /v1/training/datasets/{id}/images/{imageId} | Replace an image's boxes, or move it between train and valid. |

Rules

| Method | Endpoint | What it does | | --- | --- | --- | | analyse | POST /v1/analyse | Grade media against rules sent with the request. Nothing saved unless asked. | | analyseWithDetector | POST /v1/detectors/{id}/analyse | Grade media against a saved detector. | | createDetector | POST /v1/detectors | Create a detector. | | deleteDetector | DELETE /v1/detectors/{id} | Delete a detector and everything it recorded. | | getDetector | GET /v1/detectors/{id} | One detector, with its twenty most recent events. | | getEvent | GET /v1/events/{id} | One event in full. | | getEventClip | GET /v1/events/{id}/clip | The annotated clip for an event. | | getEventImage | GET /v1/events/{id}/image | The annotated frame for an event. | | listDetectors | GET /v1/detectors | Every detector on this account. | | listEvents | GET /v1/events | Everything analysed, including the calls that raised nothing. | | listModels | GET /v1/models | The rules product's detector variants, and every class a rule may name. | | updateDetector | PATCH /v1/detectors/{id} | Change a detector. Only the fields you send. |

Analytics

| Method | Endpoint | What it does | | --- | --- | --- | | appendAnalyticsEvent | POST /v1/analytics/events | Record an analytics point. | | queryAnalytics | POST /v1/analytics/query | Query recalled analytics points with a structured filter. | | queryAnalyticsEvents | GET /v1/analytics/events | Recall analytics points stored for the account. |

Sites

| Method | Endpoint | What it does | | --- | --- | --- | | checkCameraHealth | POST /v1/cameras/{id}/health | Inspect what this camera is actually showing. | | createCamera | POST /v1/cameras | Create a camera. | | createCustomer | POST /v1/customers | Create a customer. | | createSite | POST /v1/sites | Create a site. | | deleteCamera | DELETE /v1/cameras/{id} | Delete a camera. | | deleteCustomer | DELETE /v1/customers/{id} | Delete a customer. | | deleteSite | DELETE /v1/sites/{id} | Delete a site. | | dialCamera | POST /v1/cameras/{id}/dial | Dial this camera's RTSP stream and take a frame. | | getCamera | GET /v1/cameras/{id} | Get one camera. | | getCameraFalseAlarms | GET /v1/cameras/{id}/false-alarms | Where this camera's false alarms come from, using outcomes stored on its events. | | getCameraSnapshot | GET /v1/cameras/{id}/snapshot | The most recent frame taken from this camera. | | getCameraZoneSuggestions | GET /v1/cameras/{id}/zones/suggestions | The last scene reading stored for this camera, without reading it again. | | getCustomer | GET /v1/customers/{id} | Get one customer. | | getSite | GET /v1/sites/{id} | Get one site. | | listCameraHealth | GET /v1/cameras/{id}/health | This camera's health history, newest first. | | listCameras | GET /v1/cameras | List cameras owned by the account. | | listCustomers | GET /v1/customers | List customers owned by the account. | | listCustomerSites | GET /v1/customers/{customerId}/sites | List sites belonging to one customer. | | listSiteCameras | GET /v1/sites/{siteId}/cameras | List cameras belonging to one site. | | listSiteHealth | GET /v1/sites/{siteId}/health | Every camera at this site, with its latest health answer. | | listSites | GET /v1/sites | List sites owned by the account. | | reportCameraFalseAlarms | POST /v1/cameras/{id}/false-alarms | Where this camera's false alarms come from, and which zones would stop them. | | suggestCameraZones | POST /v1/cameras/{id}/zones/suggest | Read the camera's scene into zone-shaped regions with AEY Vision Prompt Seg. | | suggestZones | POST /v1/zones/suggest | Read any frame into zone-shaped regions with AEY Vision Prompt Seg. Nothing is stored. | | updateCamera | PATCH /v1/cameras/{id} | Update a camera. | | updateCustomer | PATCH /v1/customers/{id} | Update a customer. | | updateSite | PATCH /v1/sites/{id} | Update a site. |

Object search

| Method | Endpoint | What it does | | --- | --- | --- | | clearSiteObjects | DELETE /v1/sites/{siteId}/objects | Delete this site's object index, and the crops with it. | | getObject | GET /v1/objects/{id} | One kept object crop. | | getObjectImage | GET /v1/objects/{id}/image | The crop itself, as a JPEG. | | listSiteObjects | GET /v1/sites/{siteId}/objects | List the object crops this site has kept. | | searchSiteObjects | POST /v1/sites/{siteId}/search | Find the objects this site has already seen that look like this one. |

Account

| Method | Endpoint | What it does | | --- | --- | --- | | createCheckoutSession | POST /v1/billing/checkout-session | Create a hosted Stripe Checkout Session for credits. | | createFreeTrialSession | POST /v1/billing/free-trial-session | Start claiming the free credits: a Stripe Checkout Session in setup mode. | | createKey | POST /v1/keys | Mint a key. The secret is returned once and never again. | | createPortalSession | POST /v1/billing/portal-session | Open Stripe's billing portal for the Volume plan: card, invoices, cancel. | | createVolumeSession | POST /v1/billing/volume-session | Subscribe to the Volume plan: a Stripe Checkout Session in subscription mode. | | getBilling | GET /v1/billing | Credit balance, billing fee minimum, and account entitlement. | | getBrand | GET /v1/brand | What the console is called on the domain this request was made to. | | getInvite | GET /v1/invites/{token} | Who a sub-account invitation is from, and for which address. | | health | GET /health | Liveness. Unauthenticated. | | inviteSubAccount | POST /v1/sub-accounts/invites | Invite an email address to become a sub-account. | | listKeys | GET /v1/keys | Every key on this account. Never the secrets. | | listSubAccounts | GET /v1/sub-accounts | Your sub-accounts, and invitations not yet accepted. | | me | GET /v1/me | The authenticated account, and how it authenticated. | | revokeKey | DELETE /v1/keys/{id} | Revoke a key. Immediate. | | revokeSubAccountInvite | DELETE /v1/sub-accounts/invites/{id} | Withdraw an invitation that has not been accepted. | | updateSubAccount | PATCH /v1/sub-accounts/{id} | Change a sub-account's customer or monthly cap, or suspend it. |

Email

| Method | Endpoint | What it does | | --- | --- | --- | | createEmailDestination | POST /v1/email/destinations | Create a emailDestination. | | deleteEmailDestination | DELETE /v1/email/destinations/{id} | Delete a emailDestination. | | getEmailDestination | GET /v1/email/destinations/{id} | Get one emailDestination. | | listEmailDestinations | GET /v1/email/destinations | List emailDestinations owned by the account. | | updateEmailDestination | PATCH /v1/email/destinations/{id} | Update a emailDestination. |

Models

| Method | Endpoint | What it does | | --- | --- | --- | | createModelRegistration | POST /v1/model-registrations | Create a modelRegistration. | | deleteModelRegistration | DELETE /v1/model-registrations/{id} | Delete a modelRegistration. | | getModelArtifact | GET /v1/model-registrations/{id}/artifact | Download your model's ONNX file, uploaded or trained here, to run anywhere. | | getModelRegistration | GET /v1/model-registrations/{id} | Get one modelRegistration. | | listModelRegistrations | GET /v1/model-registrations | List modelRegistrations owned by the account. | | putModelArtifact | PUT /v1/model-registrations/{id}/artifact | Upload the ONNX file for a model registration. | | updateModelRegistration | PATCH /v1/model-registrations/{id} | Update a modelRegistration. |

Redaction

| Method | Endpoint | What it does | | --- | --- | --- | | createRedaction | POST /v1/redactions | Redact an upload: find every face and number plate in it, to review. | | deleteRedaction | DELETE /v1/redactions/{id} | Delete a redaction and its original footage. Its runs, and the redacted video, stay under Runs. | | deleteRedactionFootage | DELETE /v1/redactions/{id}/footage | Delete the original footage and keep the redacted video. The redaction can no longer be edited. | | getRedaction | GET /v1/redactions/{id} | One redaction, brought up to date with the run it is waiting on. | | getRedactionAnalysis | GET /v1/redactions/{id}/analysis | Every person and vehicle found, as tracks of normalised boxes over time. | | getRedactionSource | GET /v1/redactions/{id}/source | The original footage. Honours Range, so a video player can seek. | | getRedactionVideo | GET /v1/redactions/{id}/video | The redacted video, once status is ready. Honours Range. | | listRedactions | GET /v1/redactions | Every redaction on this account, newest first. | | renderRedaction | POST /v1/redactions/{id}/render | Make the redacted video from a review. | | retryRedactionAnalysis | POST /v1/redactions/{id}/analyse | Analyse the footage again, after an analysis failed. | | updateRedaction | PATCH /v1/redactions/{id} | Rename a redaction, or save a review in progress as its draft. |

Platform

| Method | Endpoint | What it does | | --- | --- | --- | | createRun | POST /v1/runs | Run one model over one video or image. | | createUpload | POST /v1/uploads | Upload footage for a footage job, up to 4 GB. | | deleteUpload | DELETE /v1/uploads/{id} | Delete an upload: the footage itself, not just the record. | | getCustomerUsage | GET /v1/usage/customers | Spend split by your own customers, for rebilling. | | getPricing | GET /v1/pricing | Customer processing price per hour. | | getRates | GET /v1/rates | Public list prices per processing hour. | | getRun | GET /v1/runs/{id} | One run: its status, its counts and exactly what it cost. | | getRunClip | GET /v1/runs/{id}/clip | The annotated clip, when the run produced one. | | getRunLogs | GET /v1/runs/{id}/logs | Every line the container logged for this run. | | getRunOutput | GET /v1/runs/{id}/output | The model's full output — every frame, every detection. | | getUpload | GET /v1/uploads/{id} | One upload. | | getUploadFile | GET /v1/uploads/{id}/file | The uploaded footage itself. Honours Range, so a video player can seek. | | getUsage | GET /v1/usage | Spend, by day and by model. | | listCatalogue | GET /v1/catalogue | Every model this platform can run, and what each capability returns. | | listRuns | GET /v1/runs | Run history, newest first. | | listUploads | GET /v1/uploads | Footage this account has uploaded and not yet deleted. |

Scripts

| Method | Endpoint | What it does | | --- | --- | --- | | createScript | POST /v1/scripts | Create a script. | | deleteScript | DELETE /v1/scripts/{id} | Delete a script. | | getScript | GET /v1/scripts/{id} | Get one script. | | listScripts | GET /v1/scripts | List scripts owned by the account. | | updateScript | PATCH /v1/scripts/{id} | Update a script. | | writeScript | POST /v1/scripts/ai-write | Generate a reviewable Python video handler from a behavior brief. |

UI Components

| Method | Endpoint | What it does | | --- | --- | --- | | createUiComponent | POST /v1/ui-components | Create a uiComponent. | | deleteUiComponent | DELETE /v1/ui-components/{id} | Delete a uiComponent. | | getUiComponent | GET /v1/ui-components/{id} | Get one uiComponent. | | listUiComponents | GET /v1/ui-components | List uiComponents owned by the account. | | updateUiComponent | PATCH /v1/ui-components/{id} | Update a uiComponent. |

Events

| Method | Endpoint | What it does | | --- | --- | --- | | listAudit | GET /v1/audit | Query the account audit log. | | setEventOutcome | POST /v1/events/{id}/outcome | Record whether an event was a false alarm or a real one. |

Platform resources

The package also includes a typed PlatformClient for the domain resource contract: sites, cameras, models, scripts, events, training jobs and runs. These routes are configurable so a customer can proxy them through a different gateway without changing the generated client.

import { PlatformClient } from "@aeyvision-cloud/sdk";

const platform = new PlatformClient({ apiKey: "aey_..." });
const sites = await platform.listSites({ limit: 25 });

For TypeScript, import PlatformClient from @aeyvision-cloud/sdk. Both clients default to /v1/customers, /v1/sites, /v1/cameras, /v1/model-registrations, /v1/scripts, /v1/ui-components, /v1/events, /v1/training/jobs and /v1/runs; pass route overrides when your gateway uses a different mount.

Errors

Anything but a 2xx raises AeyVisionError, carrying status and — when the rules API rejected a configuration — problems, the per-rule complaints. Both are attributes rather than prose in the message, because code that retries on 429 and gives up on 400 should not have to parse English.

Requests are retried on 429 and 5xx with exponential backoff, twice by default. Nothing else is retried: a 400 will be a 400 again, and a run that failed on the GPU has already cost real money — repeating it spends more of it for the same answer.

Licences

Every catalogue entry carries its licence, and commercial is a field. An entry whose weights you cannot ship inside a closed product is marked commercial: false with a licence_note saying why, and the warning rides on every response that used one. Read output.warnings before you build on a result.

This package is generated

Everything here comes from functions/api/src/openapi.ts by way of scripts/generate-sdk.mjs. Run npm run sdk to rebuild it. Editing it by hand is how a generated client starts lying about the API it describes.