@molecule/api-model-hosting-runpod

Auto-generated, AI-first package reference for the molecule.dev ecosystem. It is written to be read by coding agents as much as by people, and is generated from this package's source — edit src/index.ts JSDoc, not this file.

RunPod Serverless model hosting for molecule.dev — run a model server container on a load-balancing GPU endpoint (16 GB class and up) that scales to zero, called at https://<endpoint-id>.api.runpod.ai with your RunPod API key.

Load-balancing endpoints run YOUR HTTP server (no RunPod handler code), so laya-serve, jeff-serve or any /v1/systemone server works unchanged.

Quick Start

import { setProvider, requireProvider } from '@molecule/api-model-hosting'
import { provider } from '@molecule/api-model-hosting-runpod'

setProvider(provider) // reads RUNPOD_API_KEY on first use

const endpoint = await requireProvider().deploy({
  name: 'jeff',
  image: 'ghcr.io/acme/jeff-serve:0.8b',
  port: 8000,
  healthPath: '/ping',
  accelerator: { kind: 'gpu', minVramGb: 16 },
  region: 'us',
  scaling: { minInstances: 0, maxInstances: 2, idleTimeoutSeconds: 60 },
  access: 'private',
})
const headers = await requireProvider().authHeaders(endpoint.id) // { authorization: 'Bearer <RUNPOD_API_KEY>' }

Type

provider

Installation

npm install @molecule/api-model-hosting-runpod @molecule/api-bond @molecule/api-i18n @molecule/api-model-hosting @molecule/api-secrets

API

Interfaces

RunPodConfig

Configuration for the RunPod model hosting provider.

interface RunPodConfig {
  /** API key. Defaults to `RUNPOD_API_KEY`. */
  apiKey?: string
  /** GraphQL `locations` for `'us'`. Default `'US'`. */
  usLocations?: string
  /** GraphQL `locations` for `'eu'`. Default `'CZ,FR,GB,NO,RO'`. */
  euLocations?: string
  /** Container disk per worker, in GB. Default 20. */
  containerDiskGb?: number
  /** REST base URL (tests). Default `https://rest.runpod.io/v1`. */
  restBaseUrl?: string
  /** GraphQL URL (tests). Default `https://api.runpod.io/graphql`. */
  graphqlUrl?: string
  /** Endpoint URL template (tests); `{id}` is replaced. Default `https://{id}.api.runpod.ai`. */
  endpointUrlTemplate?: string
  /** How often `deploy` probes the health path while workers start, in ms. Default 5000. */
  pollIntervalMs?: number
}

RunPodEndpoint

A RunPod serverless endpoint (only the fields read here).

interface RunPodEndpoint {
  id: string
  name: string
  templateId?: string
  workersMin?: number
  workersMax?: number
  idleTimeout?: number
  gpuTypeIds?: string[]
  dataCenterIds?: string[]
  createdAt?: string
  workers?: unknown[]
}

RunPodTemplate

A RunPod template (only the fields read here).

interface RunPodTemplate {
  id: string
  name: string
}

Classes

RunPodApiError

A RunPod API error, carrying the HTTP status.

RunPodClient

Thin RunPod client.

Functions

createProvider(config)

Creates a RunPod model hosting provider.

function createProvider(config?: RunPodConfig): ModelHostingProvider

• config — API key, locations, disk size.

Returns: A ModelHostingProvider backed by RunPod Serverless.

gpuIdsFor(spec)

The GPU pools a spec may land on: the named pool, or every pool with enough memory.

function gpuIdsFor(spec: ModelEndpointSpec): string

• spec — The endpoint spec.

Returns: Comma-separated gpuIds.

toGraphQLLiteral(value)

Serializes a plain object as a GraphQL input literal (unquoted keys, type values that are enums stay quoted strings — RunPod takes strings).

function toGraphQLLiteral(value: unknown): string

• value — The value.

Returns: The GraphQL literal.

toTemplate(spec, diskGb)

The serverless template body for a spec.

function toTemplate(spec: ModelEndpointSpec, diskGb: number): Record<string, unknown>

• spec — The endpoint spec.
• diskGb — Container disk.

Returns: TemplateCreateInput.

Constants

modelHostingRunPodSecretDefinitions

Secret definitions used by the RunPod model hosting bond.

const modelHostingRunPodSecretDefinitions: SecretDefinition[]

provider

The provider implementation — lazy, so env vars are read on first use.

const provider: ModelHostingProvider

RUNPOD_CAPABILITIES

What RunPod Serverless load-balancing endpoints offer here (GPU only).

const RUNPOD_CAPABILITIES: ProviderCapabilities

RUNPOD_GPU_POOLS

GPU pools by memory, smallest first (docs.runpod.io/sdks/graphql/manage-endpoints, 2026-09-30).

const RUNPOD_GPU_POOLS: readonly { id: string; vramGb: number }[]

RUNPOD_GRAPHQL

Default GraphQL URL.

const RUNPOD_GRAPHQL: 'https://api.runpod.io/graphql'

RUNPOD_REST

Default REST base URL.

const RUNPOD_REST: 'https://rest.runpod.io/v1'

Core Interface

Implements @molecule/api-model-hosting interface.

Bond Wiring

Setup function to register this provider with the core interface:

import { setProvider } from '@molecule/api-model-hosting'
import { provider } from '@molecule/api-model-hosting-runpod'

export function setupModelHostingRunpod(): void {
  setProvider(provider)
}

Injection Notes

Requirements

Peer dependencies:

• @molecule/api-bond ^1.0.1
• @molecule/api-i18n ^1.0.1
• @molecule/api-model-hosting ^1.0.0
• @molecule/api-secrets ^1.0.1

Environment Variables

• RUNPOD_API_KEY (required) — runpod api key
  • Setup: RunPod API key — Manages templates and serverless endpoints, and authorizes calls to them (Settings → API Keys). Workers stop when the account balance reaches $0.
  • Get it here: https://www.console.runpod.io/user/settings
  • Example: rpa_XXXXXXXXXXXXXXXXXXXXXXXX

Runtime Dependencies

• @molecule/api-bond

• @molecule/api-i18n

• @molecule/api-model-hosting

• @molecule/api-secrets

• Serve /ping on your port. RunPod's load balancer health-polls /ping (the HEALTH_CHECK_PATH override is documented but reported ignored — runpod/docs#853) and expects 200 healthy / 204 still loading. A server with only /health never becomes healthy; add /ping (or proxy it). The bond sets PORT, PORT_HEALTH and HEALTH_CHECK_PATH from the spec anyway, and deploy waits on YOUR healthPath through the public URL.

• Cold starts return "no workers available" until a worker is up; retry 3 times 5–10 s apart (RunPod's own guidance), or keep minInstances: 1.

• GPU only. minVramGb picks the smallest pool class that fits (AMPERE_16 → 16 GB; 24 GB → AMPERE_24,ADA_24; …); name one or a comma-separated set in accelerator.model. 'us' → US, 'eu' → CZ,FR,GB,NO,RO, or pass RunPod location codes.

• The endpoint is created with the GraphQL saveEndpoint mutation (type: "LB"), because the REST API cannot create load-balancing endpoints; updates, reads and deletes use REST. remove drains workers to 0 before deleting, then deletes the template.

• Anyone with your RunPod API key can call every endpoint — the key is both the admin credential and the endpoint credential. Keep it server-side.

• Workers stop when the account balance reaches $0 — a silent outage.

• list() returns every serverless endpoint on the account (RunPod does not say which are load-balancing).

• API errors throw RunPodApiError with the HTTP status.