@nebulaos/llm-gateway

NebulaOS LLM Gateway provider for the NebulaOS SDK. Provides OpenAI-compatible chat completions through pre-configured routes with automatic fallback, cost tracking, and access control.

Installation

npm install @nebulaos/llm-gateway

Usage

import { LLMGateway } from "@nebulaos/llm-gateway";
import { Agent, BufferMemory } from "@nebulaos/core";

const agent = new Agent({
  id: "assistente",
  name: "assistente",
  model: new LLMGateway({
    apiKey: process.env.NEBULAOS_API_KEY,
    baseUrl: "https://your-nebula-instance.com",
    model: "assistente", // Route alias configured in NebulaOS
    options: {
      reasoning_effort: "high",  // Enable deep reasoning (Gemini)
      temperature: 0.7,
    },
  }),
  memory: new BufferMemory(),
  instructions: "You are a helpful assistant.",
});

// Use the agent
const result = await agent.execute("Analyze this complex problem step by step");
console.log(result.content);

Configuration

| Option | Type | Required | Description | |--------|------|----------|-------------| | apiKey | string | Yes | API key from NebulaOS | | baseUrl | string | No | NebulaOS instance URL (defaults to localhost:4100) | | model | string | Yes | Route alias configured in NebulaOS | | logLevel | string | No | Logger verbosity ("debug", "info", "warn", "error", "none") | | options | object | No | Default model options for every call (see below) |

Model Options

Use options to configure model-specific parameters that apply to every call:

const model = new LLMGateway({
  apiKey: "your-api-key",
  model: "minha-rota",
  options: {
    temperature: 0.7,
    reasoning_effort: "high",  // Gemini 2.5+/3+ via LiteLLM
    topK: 40,
  }
});

Common Options

| Option | Type | Description | |--------|------|-------------| | temperature | number | Sampling temperature (0-2) | | topP | number | Nucleus sampling | | topK | number | Top-K sampling (Gemini) | | maxTokens | number | Max output tokens |

Provider-Specific Options

These are passed directly to the provider via LiteLLM:

Gemini (via LiteLLM): | Option | Values | Description | |--------|--------|-------------| | reasoning_effort | "low", "medium", "high" | Controls thinking depth (Gemini 2.5+/3+) | | thinking | { type: "enabled", budget_tokens: N } | Fine-grained thinking control |

OpenAI: | Option | Type | Description | |--------|------|-------------| | frequency_penalty | number | Reduce repetition (-2 to 2) | | presence_penalty | number | Encourage new topics (-2 to 2) |

Any extra options are passed through to the underlying provider.

Routes

Routes are configured in the NebulaOS UI and provide:

• Automatic fallback: If primary model fails, automatically tries backup models
• Cost tracking: All usage is automatically tracked and billed
• Access control: API keys can be restricted to specific routes
• Rate limiting: Configurable limits per route and user

Supported Features

• ✅ Text chat completions
• ✅ Streaming responses
• ✅ Tool/function calling
• ✅ JSON mode
• ✅ Multimodal (images)
• ✅ Automatic cost tracking
• ✅ Route-based access control

Error Handling

The provider will throw errors for:

• Invalid API keys
• Routes not found or inactive
• Access denied to routes
• Network issues with NebulaOS

All errors include appropriate HTTP status codes and messages.