@inferencesh/sdk — ai inference api for javascript & typescript

official javascript/typescript sdk for inference.sh — the ai agent runtime for serverless ai inference.

run ai models, build ai agents, and deploy generative ai applications with a simple api. access 150+ models including flux, stable diffusion, llms (claude, gpt, gemini), video generation (veo, seedance), and more.

Installation

npm install @inferencesh/sdk
# or
yarn add @inferencesh/sdk
# or
pnpm add @inferencesh/sdk

Getting an API Key

Get your API key from the inference.sh dashboard.

Quick Start

import { inference, TaskStatusCompleted } from '@inferencesh/sdk';

const client = inference({ apiKey: 'your-api-key' });

// Run a task and wait for the result
const result = await client.tasks.run({
  app: 'your-app',
  input: {
    prompt: 'Hello, world!'
  }
});

if (result.status === TaskStatusCompleted) {
  console.log(result.output);
}

Usage

Basic Usage

import { inference, TaskStatusCompleted } from '@inferencesh/sdk';

const client = inference({ apiKey: 'your-api-key' });

// Wait for result (default behavior)
const result = await client.tasks.run({
  app: 'my-app',
  input: { prompt: 'Generate something amazing' }
});

if (result.status === TaskStatusCompleted) {
  console.log('Output:', result.output);
}

With Setup Parameters

Setup parameters configure the app instance (e.g., model selection). Workers with matching setup are "warm" and skip setup:

const result = await client.tasks.run({
  app: 'my-app',
  setup: { model: 'schnell' },  // Setup parameters
  input: { prompt: 'hello' }
});

Fire and Forget

// Get task info immediately without waiting
const task = await client.tasks.run(
  { app: 'my-app', input: { prompt: 'hello' } },
  { wait: false }
);

console.log('Task ID:', task.id);
console.log('Status:', task.status);

Real-time Status Updates

const result = await client.tasks.run(
  { app: 'my-app', input: { prompt: 'hello' } },
  {
    onUpdate: (update) => {
      console.log('Status:', update.status);
      console.log('Progress:', update.logs);
    }
  }
);

Batch Processing

async function processImages(images: string[]) {
  const results = [];

  for (const image of images) {
    const result = await client.tasks.run({
      app: 'image-processor',
      input: { image }
    }, {
      onUpdate: (update) => console.log(`Processing: ${update.status}`)
    });

    results.push(result);
  }

  return results;
}

File Upload

// Upload from base64
const file = await client.files.upload('data:image/png;base64,...', {
  filename: 'image.png',
  contentType: 'image/png'
});

// Use the uploaded file in a task
const result = await client.tasks.run({
  app: 'image-app',
  input: { image: file.uri }
});

Cancel a Task

const task = await client.tasks.run(
  { app: 'long-running-app', input: {} },
  { wait: false }
);

// Cancel if needed
await client.tasks.cancel(task.id);

Sessions (Stateful Execution)

Sessions allow you to maintain state across multiple task invocations. The worker stays warm between calls, preserving loaded models and in-memory state.

// Start a new session
const result = await client.tasks.run({
  app: 'my-stateful-app',
  input: { prompt: 'hello' },
  session: 'new'
});

const sessionId = result.session_id;
console.log('Session ID:', sessionId);

// Continue the session with another call
const result2 = await client.tasks.run({
  app: 'my-stateful-app',
  input: { prompt: 'remember what I said?' },
  session: sessionId
});

Custom Session Timeout

By default, sessions expire after 60 seconds of inactivity. You can customize this with session_timeout (1-3600 seconds):

// Create a session with 5-minute idle timeout
const result = await client.tasks.run({
  app: 'my-stateful-app',
  input: { prompt: 'hello' },
  session: 'new',
  session_timeout: 300  // 5 minutes
});

// Session stays alive for 5 minutes after each call

Notes:

• session_timeout is only valid when session: 'new'
• Minimum timeout: 1 second
• Maximum timeout: 3600 seconds (1 hour)
• Each successful call resets the idle timer

For complete session documentation including error handling, best practices, and advanced patterns, see the Sessions Developer Guide.

Agent Chat

Chat with AI agents using client.agents.create().

Using a Template Agent

Use an existing agent from your workspace by its namespace/name@shortid:

import { inference } from '@inferencesh/sdk';

const client = inference({ apiKey: 'your-api-key' });

// Create agent from template
const agent = client.agents.create('my-org/assistant@abc123');

// Send a message with streaming
await agent.sendMessage('Hello!', {
  onMessage: (msg) => {
    if (msg.content) {
      for (const c of msg.content) {
        if (c.type === 'text' && c.text) {
          process.stdout.write(c.text);
        }
      }
    }
  }
});

// Clean up
agent.disconnect();

Creating an Ad-Hoc Agent

Create agents on-the-fly without saving to your workspace:

import { inference, tool, string } from '@inferencesh/sdk';

const client = inference({ apiKey: 'your-api-key' });

// Create ad-hoc agent
const agent = client.agents.create({
  coreApp: 'infsh/claude-sonnet-4@abc123',  // LLM to use
  systemPrompt: 'You are a helpful assistant.',
  tools: [
    tool('get_weather')
      .description('Get current weather')
      .params({ city: string('City name') })
      .handler(async (args) => {
        // Your tool logic here
        return JSON.stringify({ temp: 72, conditions: 'sunny' });
      })
      .build()
  ]
});

await agent.sendMessage('What is the weather in Paris?', {
  onMessage: (msg) => console.log(msg),
  onToolCall: async (call) => {
    // Tool handlers are auto-executed if defined
  }
});

Structured Output

Use output_schema to get structured JSON responses:

const agent = client.agents.create({
  core_app: { ref: 'infsh/claude-sonnet-4@latest' },
  output_schema: {
    type: 'object',
    properties: {
      summary: { type: 'string' },
      sentiment: { type: 'string', enum: ['positive', 'negative', 'neutral'] },
      confidence: { type: 'number' },
    },
    required: ['summary', 'sentiment', 'confidence'],
  },
  internal_tools: { finish: true },
});

const response = await agent.sendMessage('Analyze: Great product!');

Agent Methods

| Method | Description | |--------|-------------| | sendMessage(text, options?) | Send a message to the agent | | getChat(chatId?) | Get chat history | | stopChat(chatId?) | Stop current generation | | submitToolResult(toolId, resultOrAction) | Submit result for a client tool (string or {action, form_data}) | | streamMessages(chatId?, options?) | Stream message updates | | streamChat(chatId?, options?) | Stream chat updates | | disconnect() | Clean up streams | | reset() | Start a new conversation |

API Reference

inference(config)

Creates a new inference client.

| Parameter | Type | Required | Description | |-----------|------|----------|-------------| | config.apiKey | string | Yes | Your inference.sh API key | | config.baseUrl | string | No | Custom API URL (default: https://api.inference.sh) |

client.tasks.run(params, options?)

Runs a task on inference.sh.

Parameters:

| Parameter | Type | Required | Description | |-----------|------|----------|-------------| | params.app | string | Yes | App identifier (e.g., 'username/app-name') | | params.input | object | Yes | Input parameters for the app | | params.setup | object | No | Setup parameters (affects worker warmth/scheduling) | | params.infra | string | No | Infrastructure: 'cloud' or 'private' | | params.variant | string | No | App variant to use | | params.session | string | No | Session ID or 'new' to start a new session | | params.session_timeout | number | No | Session timeout in seconds (1-3600, only with session: 'new') |

Options:

| Option | Type | Default | Description | |--------|------|---------|-------------| | wait | boolean | true | Wait for task completion | | onUpdate | function | - | Callback for status updates | | autoReconnect | boolean | true | Auto-reconnect on connection loss | | maxReconnects | number | 5 | Max reconnection attempts | | reconnectDelayMs | number | 1000 | Delay between reconnects (ms) |

client.tasks.get(taskId)

Gets a task by ID.

client.tasks.cancel(taskId)

Cancels a running task.

client.files.upload(data, options?)

Uploads a file to inference.sh.

Parameters:

| Parameter | Type | Description | |-----------|------|-------------| | data | string \| Blob | Base64 string, data URI, or Blob | | options.filename | string | Filename | | options.contentType | string | MIME type | | options.public | boolean | Make file publicly accessible |

client.agents.create(templateOrConfig)

Creates an agent instance from a template or ad-hoc configuration.

Template mode:

const agent = client.agents.create('namespace/name@version');

Ad-hoc mode:

const agent = client.agents.create({
  coreApp: 'infsh/claude-sonnet-4@abc123',
  systemPrompt: 'You are helpful.',
  tools: [...]
});

Task Status Constants

import {
  TaskStatusQueued,
  TaskStatusRunning,
  TaskStatusCompleted,
  TaskStatusFailed,
  TaskStatusCancelled
} from '@inferencesh/sdk';

if (task.status === TaskStatusCompleted) {
  console.log('Done!');
}

TypeScript Support

This SDK is written in TypeScript and includes full type definitions. All types are exported:

import type { Task, ApiTaskRequest, RunOptions } from '@inferencesh/sdk';

Requirements

• Node.js 18.0.0 or higher
• Modern browsers with fetch support

resources

• documentation — getting started guides and api reference
• blog — tutorials on ai agents, image generation, and more
• app store — browse 150+ ai models
• discord — community support
• github — open source projects

license

MIT © inference.sh