BabyAPI (JS SDK)

A tiny Node.js client for BabyAPI — an OpenAI-compatible API for hosted open-weight models.

• OpenAI-compatible endpoints:
  • POST /v1/completions
  • POST /v1/chat/completions
  • POST /v1/embeddings
  • POST /v1/rerank
• BabyAPI convenience endpoint:
  • POST /infer (simple text-in, text-out)
• Document conversion (Docling):
  • POST /docling/v1/convert/source  ·  POST /docling/v1/convert/file
  • Async variants + chunking (hybrid / hierarchical)
• OCR:
  • POST /v1/ocr/document  ·  POST /v1/ocr/document/async  ·  GET /v1/ocr/jobs/{job_id}

Minimal surface area. Calm defaults. You bring an API key — we handle the GPUs.

Install

npm install babyapi

Node 18 or newer. The examples read the key from the environment:

export BABYAPI_API_KEY=baby_sk_YOUR_API_KEY

Quick start

const { BabyAPI } = require('babyapi');

const client = new BabyAPI({
  apiKey: process.env.BABYAPI_API_KEY,
  // baseURL: 'https://api.babyapi.org', // optional
});

async function run() {
  const res = await client.chat.completions.create({
    model: 'mistral',
    messages: [{ role: 'user', content: 'Say hi in 5 words.' }],
  });

  console.log(res.choices?.[0]?.message?.content);
}

run().catch(console.error);

Configuration

const client = new BabyAPI({
  apiKey: process.env.BABYAPI_API_KEY,              // required
  baseURL: process.env.BABYAPI_BASE_URL,            // optional (default: https://api.babyapi.org)
  timeoutMs: 3_600_000,                             // per request (default 60 min; GETs wait 60 s at most)
  streamIdleTimeoutMs: 960_000,                     // streams: longest time without data (default 16 min)
  maxRetries: 2,                                    // see "Timeouts and retries"
  retryBaseDelayMs: 250,                            // exponential backoff base
  defaultModel: 'mistral',                          // used by client.baby.infer when model is omitted
  defaultHeaders: { 'x-app': 'my-sideproject' },     // extra headers for every request
});

Environment variables supported:

• BABYAPI_API_KEY (or BABY_API_KEY)
• BABYAPI_BASE_URL
• BABYAPI_DEFAULT_MODEL

OpenAI-compatible: Chat Completions

const res = await client.chat.completions.create({
  model: 'mixtral',
  messages: [
    { role: 'system', content: 'You are concise.' },
    { role: 'user', content: 'Give me 3 tagline ideas for a tiny LLM API.' },
  ],
  temperature: 0.7,
});

console.log(res.choices?.[0]?.message?.content);

OpenAI-compatible: Completions

const res = await client.completions.create({
  model: 'mistral',
  prompt: 'Write a friendly release note opener for BabyAPI.',
  max_tokens: 120,
  temperature: 0.7,
});

console.log(res.choices?.[0]?.text);

Streaming (SSE) for chat

.stream() returns an async iterator that yields SSE events.

A stream that finishes ends with one { done: true } event. A stream that does not finish throws a BabyAPIError, so a cut answer is never taken for a complete one:

| err.code | Meaning | |---|---| | STREAM_TRUNCATED | The stream ended before the API's [DONE] | | STREAM_ERROR | The connection failed mid-stream, or the API sent an error event (err.message is the API's) | | TIMEOUT | No data for streamIdleTimeoutMs | | ABORTED | Your AbortSignal fired |

A non-2xx answer throws with the API's status, code and message, like a .create() call. Events received before the failure have already been yielded.

for await (const evt of client.chat.completions.stream({
  model: 'mistral',
  messages: [{ role: 'user', content: 'Write a short poem about servers.' }],
})) {
  if (evt.done) break;

  // evt.data: parsed JSON when possible (otherwise null)
  // evt.raw:  raw "data:" payload string
  //
  // For OpenAI-style streams, you usually want:
  const delta = evt.data?.choices?.[0]?.delta?.content;
  if (delta) process.stdout.write(delta);
}
process.stdout.write('\n');

Telling a cut stream from a finished one:

let text = '';
try {
  for await (const evt of client.chat.completions.stream({ model: 'mistral', messages })) {
    if (evt.done) break;
    text += evt.data?.choices?.[0]?.delta?.content || '';
  }
  // finished: `text` is the whole answer
} catch (err) {
  // not finished: `text` holds what arrived before the failure
  console.error(err.code, err.message, err.requestId);
}

Streaming for completions

for await (const evt of client.completions.stream({
  model: 'mistral',
  prompt: 'Write 5 bullet points about calm APIs.',
})) {
  if (evt.done) break;

  const text = evt.data?.choices?.[0]?.text;
  if (text) process.stdout.write(text);
}
process.stdout.write('\n');

Embeddings

Generate vector embeddings for text inputs using OpenAI-compatible POST /v1/embeddings.

const res = await client.embeddings.create({
  model: 'qwen3-embedding',
  input: 'BabyAPI makes inference easy.',
});

console.log(res.data[0].embedding); // float array
console.log(res.usage);             // { prompt_tokens, total_tokens }

Batch embeddings

Pass an array of strings to embed multiple texts in one call:

const res = await client.embeddings.create({
  model: 'qwen3-embedding',
  input: [
    'First document to embed.',
    'Second document to embed.',
    'Third document to embed.',
  ],
});

for (const item of res.data) {
  console.log(`index ${item.index}:`, item.embedding.slice(0, 5), '…');
}

Embedding options

const res = await client.embeddings.create({
  model: 'qwen3-embedding',
  input: 'Hello world',
  encoding_format: 'float',         // "float" (default) or "base64"
  dimensions: 512,                   // reduce output dimensions (model-dependent)
  truncate_prompt_tokens: 8192,      // max tokens per input to avoid OOM
});

Reranking

Rerank a set of documents against a query using POST /v1/rerank (Jina / Cohere compatible).

const res = await client.rerank.create({
  model: 'qwen3-reranker',
  query: 'What is BabyAPI?',
  documents: [
    'BabyAPI is a tiny hosted LLM API.',
    'The weather in Lisbon is sunny.',
    'BabyAPI supports open-weight models.',
  ],
});

for (const result of res.results) {
  console.log(`#${result.index} — relevance: ${result.relevance_score}`);
}

Reranking options

const res = await client.rerank.create({
  model: 'qwen3-reranker',
  query: 'Calm inference APIs',
  documents: ['doc A …', 'doc B …', 'doc C …'],
  top_n: 2,                          // return only top 2 results
  return_documents: true,             // include document text in response
  truncate_prompt_tokens: 4096,       // max tokens per document
});

Timeouts and retries

An inference can run for a long time and is billed when it completes, even if your client has stopped waiting. The defaults follow from that.

Timeouts

| Call | Default | |---|---| | POST (chat, completions, embeddings, rerank, /infer, docling, OCR) | timeoutMs: 60 minutes | | GET (docling health / status / result, OCR job) | 60 seconds, or timeoutMs if that is shorter | | File uploads | timeoutMs, 5 minutes at least | | Streams | no total timeout; streamIdleTimeoutMs (16 minutes) without data |

A per-call timeoutMs / streamIdleTimeoutMs always wins. streamIdleTimeoutMs: 0 turns the idle timeout off.

Retries (maxRetries, default 2, exponential backoff, Retry-After honoured)

| | Retried | Not retried | |---|---|---| | POST | could not connect; 503; 429 from the rate limiter | timeout; connection lost after sending; 500, 502, 504; 429 MONTHLY_LIMIT_REACHED; other 4xx | | GET | the above, plus network errors, timeouts, 408, 502, 504 | other 4xx, 500 | | Streams | as POST, and only before the first byte | anything after the first byte | | File uploads | never | |

A POST is retried only when the request provably did not run, so a retry cannot run (and bill) the same inference twice. A call you abort is never retried. To decide yourself, pass shouldRetry: (err, { method, attempt }) => boolean; err.response is set when the API answered.

Abort / cancellation (AbortController)

const ac = new AbortController();

setTimeout(() => ac.abort(), 1500);

try {
  const res = await client.chat.completions.create(
    {
      model: 'mistral',
      messages: [{ role: 'user', content: 'Write a long story...' }],
    },
    { signal: ac.signal }
  );

  console.log(res.choices?.[0]?.message?.content);
} catch (err) {
  if (err.name === 'BabyAPIError') {
    console.error('BabyAPIError:', err.code, err.status, err.message);
  } else {
    console.error(err);
  }
}

BabyAPI convenience: client.infer(...) (routes chat vs completions)

If you prefer “just do the right thing”, use client.infer(...):

// Routes to /v1/chat/completions if messages[] exists
const chatRes = await client.infer({
  model: 'mistral',
  messages: [{ role: 'user', content: 'One-line slogan for BabyAPI?' }],
});
console.log(chatRes.choices?.[0]?.message?.content);

// Routes to /v1/completions if prompt exists
const compRes = await client.infer({
  model: 'mistral',
  prompt: 'Give 3 product names for a tiny LLM SDK.',
  max_tokens: 60,
});
console.log(compRes.choices?.[0]?.text);

BabyAPI convenience: client.baby.infer(...) (simple text-out)

This hits BabyAPI’s /infer endpoint and returns a normalized response:

const out = await client.baby.infer({
  model: 'mistral',
  prompt: 'Write a 1-line release note title.',
  maxTokens: 40,
  temperature: 0.5,
});

console.log(out.output);
console.log(out.usage);        // prompt_tokens / completion_tokens / total_tokens
console.log(out.finish_reason);

You can also pass a raw string (uses defaultModel if configured):

const client = new BabyAPI({ apiKey: process.env.BABYAPI_API_KEY, defaultModel: 'mistral' });

const out = await client.baby.infer('Explain BabyAPI in one sentence.');
console.log(out.output);

Supported options (aliases are accepted):

• max_tokens / maxTokens
• temperature
• top_p / topP
• top_k / topK
• stop
• presence_penalty / presencePenalty
• frequency_penalty / frequencyPenalty

Vision / image input (OpenAI-style)

If your selected model supports vision, you can send an image using OpenAI-style message content:

const res = await client.chat.completions.create({
  model: 'pixtral', // or another vision-capable model you expose
  messages: [
    {
      role: 'user',
      content: [
        { type: 'text', text: 'Describe the image in 2 sentences. Then list 3 distinct objects you can see.' },
        {
          type: 'image_url',
          image_url: {
            url: 'https://api.babyapi.org/images/banner.png',
          },
        },
      ],
    },
  ],
});

console.log(res.choices?.[0]?.message?.content);

Vision streaming

for await (const evt of client.chat.completions.stream({
  model: 'pixtral',
  messages: [
    {
      role: 'user',
      content: [
        { type: 'text', text: 'What is this image trying to communicate?' },
        { type: 'image_url', image_url: { url: 'https://api.babyapi.org/images/banner.png' } },
      ],
    },
  ],
})) {
  if (evt.done) break;
  const delta = evt.data?.choices?.[0]?.delta?.content;
  if (delta) process.stdout.write(delta);
}
process.stdout.write('\n');

Note: image support depends on the model you choose. If the model is text-only, the API may reject image inputs.

Docling: document conversion & chunking

The client.docling.* namespace wraps BabyAPI's Docling proxy. Use it to convert PDFs / DOCX / PPTX / images into Markdown, JSON, HTML, text, or doctags, and to chunk documents for downstream RAG pipelines.

All calls authenticate with your standard BABYAPI_API_KEY.

Health / version

await client.docling.health();   // { status: 'ok' }
await client.docling.ready();
await client.docling.version();

Convert from a URL (synchronous)

const res = await client.docling.convertSource({
  sources: [{ kind: 'http', url: 'https://arxiv.org/pdf/2408.09869' }],
  options: { to_formats: ['md'], do_ocr: false, page_range: [1, 10] },
});

console.log(res.document?.md_content);

Convert a local file (synchronous)

Files accept a path string, a Buffer, or a structured entry:

// path string
await client.docling.convertFile({ files: './invoice.pdf' });

// multiple files + options
await client.docling.convertFile({
  files: ['./a.pdf', './b.docx'],
  options: { to_formats: ['md', 'json'] },
});

// raw buffer
await client.docling.convertFile({
  files: { filename: 'report.pdf', content: buffer, contentType: 'application/pdf' },
});

Convert asynchronously (recommended for large docs)

When you submit a file asynchronously, BabyAPI routes the job to one specific docling node and returns an x-babyapi-docling-id header identifying it. The SDK captures this automatically and exposes it as _babyapiDoclingId on the response object. Pass it to waitForResult (or to every pollStatus / getResult call manually) so that all subsequent requests are pinned to the node that owns the task. Without it, round-robin routing may hit a different node and return a 404.

const task = await client.docling.convertFileAsync({ files: './big.pdf' });
// task._babyapiDoclingId — identifies which node accepted the job

// Easiest: poll + fetch in one call. Passing the task object pins the
// requests to the right node; with a plain task id, pass `doclingId` yourself.
const result = await client.docling.waitForResult(task, {
  intervalMs: 2000,
  timeoutMs: 10 * 60_000,
  onPoll: (s) => console.log('status:', s.status),
});

console.log(result.document?.md_content);

Low-level alternative:

const task = await client.docling.convertSourceAsync({
  sources: [{ kind: 'http', url: 'https://arxiv.org/pdf/2408.09869' }],
});

let status;
do {
  await new Promise((r) => setTimeout(r, 2000));
  status = await client.docling.pollStatus(task.task_id, {
    headers: { 'x-babyapi-docling-id': task._babyapiDoclingId },
  });
} while (!['success', 'failure', 'error'].includes(status.status));

if (status.status === 'success') {
  const result = await client.docling.getResult(task.task_id, {
    headers: { 'x-babyapi-docling-id': task._babyapiDoclingId },
  });
  console.log(result);
}

Cancel an async task

await client.docling.cancel(task.task_id);

Chunking

Same file/source shapes as conversion — output is a list of chunks suitable for embeddings.

// Hybrid chunking from a URL
await client.docling.chunk.hybridSource({
  sources: [{ kind: 'http', url: 'https://arxiv.org/pdf/2408.09869' }],
});

// Hierarchical chunking from a file
await client.docling.chunk.hierarchicalFile({ files: './handbook.pdf' });

Conversion options

Pass any docling-serve options via options. Common ones:

| Option | Default | Description | |---|---|---| | to_formats | ['md'] | md, json, html, text, doctags | | do_ocr | true | Run OCR on images/scanned pages | | force_ocr | false | Force OCR even on text-layer PDFs | | do_table_structure | true | Detect and extract table structure | | table_mode | 'accurate' | accurate or fast | | page_range | full | e.g. [1, 10] | | image_export_mode | 'embedded' | embedded or referenced | | do_formula_enrichment | false | | | do_picture_classification | false | |

See the Docling endpoints reference for the full list.

OCR

client.ocr.* wraps the OCR endpoints. They need an OCR model on your account.

// Synchronous: images and small PDFs
const res = await client.ocr.document({
  model: 'your-ocr-model',
  input: { /* the document, as the model expects it */ },
  tasks: ['ocr'],            // default
  output_format: 'json',     // default
  // pages: [1, 10],
});

// Asynchronous: large PDFs
const job = await client.ocr.documentAsync({ model: 'your-ocr-model', input: { /* ... */ } });
const done = await client.ocr.waitForResult(job, {
  intervalMs: 2000,
  timeoutMs: 10 * 60_000,
  onPoll: (j) => console.log('status:', j.status),   // queued | processing | completed | failed
});

// Or poll yourself
const current = await client.ocr.getJob(job.job_id);

waitForResult returns the finished job and throws OCR_JOB_FAILED when the job fails. A job id is only known to the API for about two hours and is lost if the API restarts: polling then throws with status 404, code JOB_NOT_FOUND.

Errors

All SDK errors are thrown as BabyAPIError:

| Field | | |---|---| | message | The API's own message when it sent one | | status | HTTP status, when the API answered | | code | The API's error code when it sends one (RATE_LIMITED, FREE_TIER_EXHAUSTED, MONTHLY_LIMIT_REACHED, SERVER_DRAINING...), else one of the SDK's: TIMEOUT, CONNECTION_ERROR, ABORTED, STREAM_ERROR, STREAM_TRUNCATED, POLL_TIMEOUT, DOCLING_TASK_FAILED, OCR_JOB_FAILED | | type | The API's error type, when present | | requestId | X-BabyAPI-Request-Id of the answer: the id of the request on your Requests page. Quote it when asking for support | | details | The error body as the API sent it |

try {
  await client.chat.completions.create({ model: 'mistral', messages: [] });
} catch (err) {
  if (err.name === 'BabyAPIError') {
    console.error({
      message: err.message,
      status: err.status,
      code: err.code,
      type: err.type,
      requestId: err.requestId,
    });
  } else {
    console.error(err);
  }
}

Request options (per-call)

Every .create(...) / .stream(...) call can override:

const res = await client.chat.completions.create(
  {
    model: 'mistral',
    messages: [{ role: 'user', content: 'Hello.' }],
  },
  {
    apiKey: process.env.BABYAPI_API_KEY,          // override key
    timeoutMs: 30_000,                           // this call only
    // streamIdleTimeoutMs: 120_000,             // .stream() only
    maxRetries: 0,                               // disable retries
    headers: { 'x-trace': 'abc123' },             // extra per-request headers
    // signal: new AbortController().signal,      // cancellation
  }
);

TypeScript

This package ships types via types.d.ts.

import { BabyAPI } from 'babyapi';

const client = new BabyAPI({ apiKey: process.env.BABYAPI_API_KEY! });

const res = await client.chat.completions.create({
  model: 'mistral',
  messages: [{ role: 'user', content: 'TypeScript works.' }],
});

Tests

npm test

The tests run offline against a local fake server (Node's built-in test runner, no extra dependencies).

License

MIT.