@piedpiperrh/sdk

TypeScript client for the Pied Piper distributed inference API — OpenAI-compatible chat completions served by community GPUs.

Zero dependencies. Node ≥ 18, Bun, Deno, and browsers.

Install

npm install @piedpiperrh/sdk

Quick start

Mint an API key in the dashboard (Settings → API keys), then:

import PiedPiper from "@piedpiperrh/sdk";

const client = new PiedPiper({ apiKey: process.env.PIEDPIPER_API_KEY! });

const models = await client.models.list();

const completion = await client.chat.completions.create({
  model: models[0].id,
  messages: [{ role: "user", content: "Explain WebGPU in one paragraph." }],
});

console.log(completion.choices[0].message.content);

Streaming

const stream = await client.chat.completions.create({
  model: "llama-3.2-3b-instruct-q4f16_1",
  messages: [{ role: "user", content: "Write a haiku about GPUs." }],
  stream: true,
});

for await (const chunk of stream) {
  process.stdout.write(chunk.choices[0]?.delta.content ?? "");
}

Routing options

Every request accepts a second options argument:

await client.chat.completions.create(params, {
  host: "provider-session-uuid", // pin to one specific host
  priority: true,                // jump the queue on that host (costs more)
  waitSeconds: 30,               // wait up to 30s for capacity instead of failing fast
  signal: abortController.signal,
});

Errors

Failed requests throw PiedPiperError with status (HTTP) and code (e.g. insufficient_credits, no_capacity, rate_limited).

import { PiedPiperError } from "@piedpiperrh/sdk";

try {
  await client.chat.completions.create(params);
} catch (err) {
  if (err instanceof PiedPiperError && err.code === "no_capacity") {
    // retry with { waitSeconds: 60 }
  }
}

Tool calling

Pass OpenAI-format tools; requests route to tool-capable console hosts automatically.

const r = await client.chat.completions.create({
  model: "llama-3.2-3b-instruct-q4f16_1",
  messages: [{ role: "user", content: "What's the weather in Paris?" }],
  tools: [{
    type: "function",
    function: {
      name: "get_weather",
      description: "Get current weather for a city",
      parameters: { type: "object", properties: { city: { type: "string" } }, required: ["city"] },
    },
  }],
});

if (r.choices[0].finish_reason === "tool_calls") {
  for (const call of r.choices[0].message.tool_calls!) {
    console.log(call.function.name, JSON.parse(call.function.arguments));
  }
}

Vision

Send images as inline data URLs in content-part arrays (remote image URLs are not fetched). Requests route to hosts serving vision models (llava, llama3.2-vision, …).

await client.chat.completions.create({
  model: "llava-1.6-mistral-7b-q4f16_1",
  messages: [{
    role: "user",
    content: [
      { type: "text", text: "What's in this image?" },
      { type: "image_url", image_url: { url: `data:image/png;base64,${b64}` } },
    ],
  }],
});

Supported parameters

model, messages (string or content-part arrays), stream, temperature, top_p, max_tokens, stop, tools, tool_choice. n must be 1 — multiple choices per request aren't supported.

Self-hosted / preview deployments

new PiedPiper({ apiKey, baseURL: "https://your-deployment.example.com" });