Amniscient Detection SDK for TypeScript

TypeScript/JavaScript client for the Amniscient Inference API over HTTPS: low-latency object detection behind a single call, client.detect(image, modelId).

Requires Node 18+ (native fetch) or a modern browser. No runtime dependencies.

Installation

npm install @amniscient/detection

Quick start

Option A — the snippet. Drop this into your app:

import { DetectionClient } from "@amniscient/detection";

const client = new DetectionClient({
  endpoint: "https://api.amniscient.com/inference",
  apiKey: "YOUR_KEY",
  organizationId: "YOUR_ORG",
});

const result = await client.detect("photo.jpg", "MODEL_ID");

console.log(result.detectionFound);                 // true / false
for (const d of result.detections) {
  console.log(d.classId, d.confidence.toFixed(2), d.bbox); // bbox: {x1,y1,x2,y2}, normalized 0-1
}

Option B — a runnable script. Save this as detect.ts, fill in the four values at the top, and run it with npx tsx detect.ts path/to/image.jpg:

import { DetectionClient } from "@amniscient/detection";

const ENDPOINT = "https://api.amniscient.com/inference";
const API_KEY = "YOUR_KEY";
const ORG_ID = "YOUR_ORG";
const MODEL_ID = "MODEL_ID";

const imagePath = process.argv[2];
if (!imagePath) {
  console.error("Usage: npx tsx detect.ts <image_path>");
  process.exit(1);
}

const client = new DetectionClient({ endpoint: ENDPOINT, apiKey: API_KEY, organizationId: ORG_ID });

const result = await client.detect(imagePath, MODEL_ID);
console.log(`Found ${result.detections.length} objects in ${imagePath}:\n`);
for (const d of result.detections.sort((a, b) => b.confidence - a.confidence)) {
  const b = d.bbox;
  console.log(`  ${(d.confidence * 100).toFixed(0)}%  ${d.classId}`);
  console.log(`     bbox (0-1): (${b.x1.toFixed(3)}, ${b.y1.toFixed(3)}) -> (${b.x2.toFixed(3)}, ${b.y2.toFixed(3)})`);
}

Top-level await needs "type": "module" in your package.json (or name the file detect.mts). A ready-to-run version that reads these from env vars lives in examples/basic-detection.ts.

detect() accepts a file path (Node), raw bytes (Uint8Array/ArrayBuffer), or a Blob (browser). Everything is async — the client is safe to construct once and reuse.

Endpoint. Use the HTTPS endpoint https://api.amniscient.com/inference. Your apiKey is your own application API key — validated by the API gateway on every request. (This SDK is HTTP-only; the NATS transport is internal and not exposed here.)

How it works

The whole surface is detect(). You give it an image and a model; it returns a DetectionResult. Everything in between is handled for you:

1. Auth is validated first — a missing API key throws AuthenticationError before any image work or network call happens. It is never skipped.
2. The image is prepared — loaded into bytes, optionally compressed, and validated.
3. The request is sent and, on a transient failure (network error or 5xx), retried with exponential backoff.
4. The reply becomes a DetectionResult, or a specific typed error is thrown (see Error Handling).

For contributors, the pieces are small and single-purpose: client.ts orchestrates the flow and owns retries + error mapping; types.ts holds the public shapes plus the filterBy* helpers; utils.ts is where a reply becomes a DetectionResult and where image bytes are loaded/validated; annotate.ts turns a payload into draw-ready boxes; errors.ts holds the typed error hierarchy and the HTTP-status mapping.

Features

• One call, detect() — connection handling, retries, and parsing done for you
• Zero runtime dependencies — built on native fetch/FormData/Blob, Node 18+ or a modern browser
• Pluggable compression — passthrough by default; supply a compress hook (e.g. backed by sharp or a <canvas>) to actually resize/re-encode
• Retry logic — exponential backoff for transient errors (network, 5xx)
• Validated auth — a missing/invalid API key throws AuthenticationError, never a silent failure
• Fully async — every method returns a Promise; fan out with Promise.all when you need concurrency
• Raw mode — detectRaw() returns the unparsed server payload when you need it
• Full type hints — .d.ts shipped for every export, IDE-friendly

Usage Examples

Different image inputs

import { readFile } from "node:fs/promises";
import { DetectionClient } from "@amniscient/detection";

const client = new DetectionClient({ endpoint: ENDPOINT, apiKey: API_KEY, organizationId: ORG_ID });

// From a file path (Node only)
const a = await client.detect("image.jpg", "MODEL_ID");

// From raw bytes (Uint8Array)
const bytes = new Uint8Array(await readFile("image.jpg"));
const b = await client.detect(bytes, "MODEL_ID");

// From an ArrayBuffer
const c = await client.detect(bytes.buffer, "MODEL_ID");

// From a Blob (browser, or Node 18+)
const blob = new Blob([bytes], { type: "image/jpeg" });
const d = await client.detect(blob, "MODEL_ID");

Filtering results

import { filterByConfidence, filterByClassId, filterByTag } from "@amniscient/detection";

const result = await client.detect("image.jpg", "MODEL_ID");

// Filter by confidence threshold
const highConf = filterByConfidence(result, 0.8);

// Filter by class id
const peopleOnly = filterByClassId(result, "person", "pedestrian");

// Filter by a tag key/value
const shelfA1 = filterByTag(result, "shelf", "A1");

Each filterBy* helper returns a new DetectionResult — the original is left untouched.

Raw responses

By default detect() parses the server reply into a DetectionResult. Use detectRaw() when you need the unparsed response instead — for logging/debugging, or when downstream code already depends on the original xywh contract:

const payload = await client.detectRaw("image.jpg", "MODEL_ID");
// payload == { inference_id: "...", detection_found: true,
//              xywh: [{ x, y, width, height, confidence, id, tag }, ...] }
for (const det of payload.xywh as Array<Record<string, unknown>>) {
  console.log(det.id, det.confidence);
}

Retries, error mapping, and auth validation are unchanged — only the return value skips parsing.

Barcode & QR reading

Pass detectBarcodes: true to decode barcodes/QR codes alongside object detection, in the same request:

const result = await client.detect("shelf.jpg", "MODEL_ID", { detectBarcodes: true });
// object detection is unchanged; barcodes ride alongside:
for (const code of result.barcodes) {
  console.log(code.type, code.value); // e.g. qr https://…  |  upc-a 0012345678905
}

It's off by default — pass detectBarcodes: true explicitly. Supported symbologies (wire type values): qr, upc-a, upc-e, code-39.

Notes:

• result.barcodes is [] when detectBarcodes is off, no codes are found, or the decode step fails or times out — it never throws and never affects result.detections.
• Numeric 1D codes (UPC-A/UPC-E) decode to their expanded digit string (e.g. "0012345678905"), so don't expect an exact round-trip of a shorter printed value. code-39 returns the encoded text as-is.

Pinning a model version

Models are versioned. By default a request runs whichever version is currently active, so results follow promotions automatically:

// active version — follows promotions
const result = await client.detect("shelf.jpg", "MODEL_ID");

// pinned — same weights and thresholds regardless of later promotions
const pinned = await client.detect("shelf.jpg", "MODEL_ID", { version: 3 });

Versions are 1-based. Pin when you need reproducible output — a benchmark, a regression suite, or a customer held to a specific model revision. Leave it unset to get improvements as they're promoted. The version also selects that version's confidence/IoU thresholds and detection-item tags, not just its weights. A version that doesn't exist throws ModelNotFoundError.

Multi-tenant / per-request credentials

The client can hold one credential for its lifetime, or none at all. If you're a multi-tenant proxy whose credentials arrive with each request and are never stored server-side, build one shared client with no stored credentials and pass apiKey / organizationId per detect() call — they ride each request, and nothing is kept between calls:

// one shared, long-lived client — no credentials stored
const client = new DetectionClient({ endpoint: ENDPOINT });

// each request carries its own tenant's credentials
const result = await client.detect(image, "MODEL_ID", {
  apiKey: tenantKey,
  organizationId: tenantOrg,
});

Validating credentials (login gate)

To accept an API key + org before a model is chosen (e.g. a web login step), use validate() — it needs no model id and no image:

if (!(await client.validate(organizationId, apiKey))) {
  throw new Error("Invalid API key or organization");
}

validate() resolves true/false — it calls GET /models, so an accepted key gets a 200 (a real positive signal) and a rejected key (401/403) → false. A missing route throws EndpointNotAvailableError rather than masquerading as valid, so you never get a false "looks valid" from a route that simply isn't there — inspect statusCode on the thrown DetectionError if you need to tell those apart. Both validate() and getDefaultModel() are HTTP only and authenticate through the API gateway.

Listing models (model picker)

To let a user pick a model instead of pasting a UUID, listModels() returns the org's selectable models — no model id or image needed:

const models = await client.listModels(organizationId, apiKey);
for (const m of models) {
  dropdown.add({ value: m.id, label: m.name }); // gray out non-TRAINED
  // each m: { id: "<uuid>", name: "<label>", status: "TRAINED" }
}

const dflt = await client.getDefaultModel(organizationId, apiKey); // model dict, or null

It returns top-level models (specialist sub-models are excluded), each with id, name, and status. HTTP only, authenticated through the gateway; organizationId / apiKey override the client's defaults.

Putting it together — a typical flow

The methods above compose into one flow: validate the key at login → list models to pick one → detect on each capture. How you hold the client depends on your architecture.

Single-tenant app (one org, credentials known at startup) — one client for its lifetime:

const client = new DetectionClient({ endpoint: ENDPOINT, apiKey, organizationId });

// 1. Login gate — reject bad credentials before showing the app
if (!(await client.validate())) {
  throw new Error("Invalid API key or organization");
}

// 2. Let the user choose a model (dropdown), instead of pasting a UUID
const models = await client.listModels();      // [{ id, name, status }, ...]
const modelId = chooseFrom(models);            // e.g. first TRAINED model

// 3. Run detection on each capture — Promise.all when you need concurrency
const results = await Promise.all(captures.map((frame) => client.detect(frame, modelId)));
results.forEach((result) => handle(result.detections));

Multi-tenant proxy (credentials arrive per request, nothing stored) — one shared client, credentials passed per call:

import type { ImageInput } from "@amniscient/detection";

const client = new DetectionClient({ endpoint: ENDPOINT }); // no stored credentials

async function handleRequest(req: { org: string; key: string; image: ImageInput; modelId: string }) {
  // validate this tenant's key at their login step...
  if (!(await client.validate(req.org, req.key))) {
    return http401();
  }
  // ...then serve detections, credentials riding each call
  return client.detect(req.image, req.modelId, {
    organizationId: req.org,
    apiKey: req.key,
  });
}

Distinguish a real credential failure from a gateway routing error at the login gate — see Error Handling for the 401 vs 403 pattern.

Annotating detections (draw boxes)

Turn a /detect response + the source image into draw-ready boxes. Pass the raw payload (detectRaw()), unchanged:

const payload = await client.detectRaw("shelf.jpg", "MODEL_ID");
const result = await client.annotateDetections("shelf.jpg", payload); // label by name (default)

for (const b of result.boxes) {          // draw-ready specs — render however you like
  console.log(b.label, b.box);           // box = {x1,y1,x2,y2} in pixels; also id/name/confidence/color
}

result.boxes (an AnnotationBox[]) is the primary product — pixel-space, label-resolved specs you can render in any toolkit (sharp, canvas, SVG). Options: { label?: "name" | "id", color?: string, render?: boolean, names?: Record<string, string> } (label defaults to "name", color defaults to #00FF00).

You can also build the same boxes directly from a payload + known image dimensions, without a client, via the free function:

import { buildAnnotation } from "@amniscient/detection";

const boxes = buildAnnotation(payload, imgWidth, imgHeight, { label: "id" });

result.toPng() renders those boxes onto the source image and resolves with PNG bytes — but it needs a canvas backend (a browser OffscreenCanvas/<canvas>, or createImageBitmap). In a plain Node process it rejects with a message telling you to render .boxes yourself:

// Browser (or any environment with a canvas backend):
const png = await result.toPng();   // ready-to-display PNG bytes

// Plain Node: no canvas backend — render `.boxes` yourself, e.g. with
// sharp or @napi-rs/canvas, using the pixel-space box/label/color fields.

Handle the no-detections case — an empty payload throws NoDetectionsError instead of returning a blank image:

import { NoDetectionsError } from "@amniscient/detection";

try {
  const result = await client.annotateDetections(image, payload);
} catch (e) {
  if (e instanceof NoDetectionsError) {
    // nothing detected — show the original image / a "no detections" state
  } else {
    throw e;
  }
}

Name labels are pulled by id from the model's detection-items (the metadata endpoint, a sibling of your inference endpoint), cached per model and falling back to the id. The SDK derives that URL from an https://…/inference endpoint; if yours differs, pass metadataEndpoint when constructing the client — or pass a names: { id: name } map to annotateDetections to skip the network call.

Configuration Options

ClientOptions passed to new DetectionClient({ ... }):

| Option | Default | Description | |--------|---------|--------------| | endpoint | — (required) | HTTPS endpoint URL, e.g. https://api.amniscient.com/inference | | apiKey | undefined | Your application API key; can be supplied per detect() call instead | | organizationId | undefined | Organization identifier; can be supplied per call instead | | timeoutMs | 30000 | Request timeout in milliseconds | | maxRetries | 3 | Max retry attempts for transient failures (network, 5xx) | | detectPath | /detect | Override the detect path; a {model_id} placeholder is substituted | | compression | true | Whether the compress hook (if any) runs before sending — a no-op passthrough when no hook is supplied | | highCompression | false | High-compression bounds (800x600) passed to the compress hook; overrides jpegQuality's effective value | | jpegQuality | 70 | JPEG quality (1-100) passed to the compress hook | | maxDimension | 3840 | Longest-side cap in pixels (4K UHD) passed to the compress hook | | compress | undefined (passthrough) | Pluggable ImageCompressor — supply one (e.g. backed by sharp in Node or a <canvas> in the browser) to actually resize/re-encode | | metadataEndpoint | derived from endpoint's /inference sibling | Base URL for annotateDetections object-name lookups |

The compression truth: the base SDK ships no image library (zero runtime dependencies), so compression is a passthrough by default — bytes are sent unchanged even with compression: true. This matches the Python SDK's behavior on a base install without Pillow. Supply compress (an ImageCompressor) to actually resize/re-encode; the jpegQuality / maxDimension / highCompression knobs are then passed to it as CompressOptions.

Error Handling

Every failure surfaces as a typed subclass of DetectionError, each carrying statusCode / responseBody (both undefined for client-side errors, i.e. ones that never reached an HTTP response):

| Error | Meaning | |-------|---------| | DetectionError | Base class for all of the below | | AuthenticationError | Missing/invalid API key or organization ID (a genuine 401/403) | | EndpointNotAvailableError | The route isn't deployed on this gateway/stage (a 403 MISSING_AUTHENTICATION_TOKEN) — the key is valid | | ModelNotFoundError | The requested model does not exist (404) | | ImageTooLargeError | Payload exceeds the server's size limit (413) | | RateLimitError | Too many requests (429) | | ServerError | Server-side (5xx) failure, after retries are exhausted | | ConnectionError | Network/timeout failure, or a retried 5xx, after retries are exhausted | | InvalidImageError | The image input was empty or its bytes couldn't be read | | NoDetectionsError | annotateDetections/buildAnnotation was given a payload with no detections |

import {
  AuthenticationError,
  ModelNotFoundError,
  ImageTooLargeError,
  RateLimitError,
  ServerError,
  ConnectionError,
} from "@amniscient/detection";

try {
  const result = await client.detect("image.jpg", "MODEL_ID");
} catch (e) {
  if (e instanceof AuthenticationError) console.error("Invalid API key or organization ID");
  else if (e instanceof ModelNotFoundError) console.error("Model not found:", e.message);
  else if (e instanceof ImageTooLargeError) console.error("Image too large:", e.message);
  else if (e instanceof RateLimitError) console.error("Rate limited — back off and retry");
  else if (e instanceof ServerError) console.error(`Server error (${e.statusCode})`);
  else if (e instanceof ConnectionError) console.error("Failed to connect to server");
  else throw e;
}

Transient failures (network errors, 5xx) are retried with exponential backoff before surfacing as ConnectionError or ServerError.

A missing route is not a rejected key. An API Gateway returns 403 MISSING_AUTHENTICATION_TOKEN for an unmatched route even when the key is valid. The SDK recognizes that signature and throws EndpointNotAvailableError (a distinct type — not AuthenticationError), so a route that simply isn't deployed never masquerades as "bad credentials":

import { EndpointNotAvailableError, AuthenticationError } from "@amniscient/detection";

try {
  const models = await client.listModels(organizationId, apiKey); // or detect(...)
} catch (e) {
  if (e instanceof EndpointNotAvailableError) {
    // the key is fine — the route just isn't exposed on this gateway/stage
    throw new Error("Inference route not available for this environment");
  } else if (e instanceof AuthenticationError) {
    throw new Error("Invalid API key or organization"); // genuine 401/403
  } else {
    throw e;
  }
}

validate() resolves true/false for a genuine accept/reject, and throws EndpointNotAvailableError when the route isn't deployed — it never answers a confident false for a route that isn't there.

Development

npm install
npm run build      # tsup -> dist (CJS + ESM + .d.ts)
npm test           # vitest run — 79 tests across 7 files
npm run typecheck  # tsc --noEmit

Run the example against a live env:

export AMNISCIENT_ENDPOINT="https://api.amniscient.com/inference"
export AMNISCIENT_API_KEY="..."; export AMNISCIENT_ORG_ID="..."; export AMNISCIENT_MODEL_ID="..."
npm run example -- path/to/image.jpg

Live sandbox test — proves a real detect() succeeds and a bad key is actually rejected, end to end against the sandbox environment:

bash scripts/run-sandbox-detect.sh [image]

(Requires an AWS SSO refresh first; re-run it if you hit an auth error.)

More detail lives alongside this package: docs/USER_GUIDE.md (task-oriented walkthrough), MIGRATION.md (moving off direct HTTP), BUILD.md (contributors), and tests/README.md (what each test file covers).

License

MIT