@layr-labs/ai-gateway-provider

v1.0.1

Published

18 days ago

Vercel AI SDK provider for EigenCloud AI Gateway with JWT authentication

0High
0Medium
0Low

ai vercel ai-sdk llm jwt authentication

EigenCloud AI Gateway Provider for Vercel AI SDK

A custom provider for the Vercel AI SDK that connects to the EigenCloud AI Gateway with JWT authentication.

Features

Configured via env vars: BaseURL read from env vars, no configuration required
TEE Attestation Auth: Automatic JWT via TEE attestation when KMS_SERVER_URL and KMS_PUBLIC_KEY are set
OpenAI-Compatible: Works with any OpenAI-compatible API
Full AI SDK Support: Compatible with all Vercel AI SDK features (streaming, tool calls, structured output, etc.)
Image & Video Generation: Built-in support for image and video generation models
TypeScript: Fully typed for excellent developer experience

Installation

npm install @layr-labs/ai-gateway-provider ai

Quick Start

import { eigen } from '@layr-labs/ai-gateway-provider';
import { generateText } from 'ai';

// Use with AI SDK functions
const result = await generateText({
  model: eigen('anthropic/claude-sonnet-4.6'),
  prompt: 'What is the capital of France?',
});

console.log(result.text);

Usage

Basic Text Generation

import { generateText } from 'ai';
import { eigen } from '@layr-labs/ai-gateway-provider';

const { text } = await generateText({
  model: eigen('anthropic/claude-sonnet-4.6'),
  prompt: 'Explain quantum computing in simple terms.',
});

Streaming Responses

import { streamText } from 'ai';
import { eigen } from '@layr-labs/ai-gateway-provider';

const result = await streamText({
  model: eigen('anthropic/claude-sonnet-4.6'),
  prompt: 'Write a short story about a robot.',
});

for await (const chunk of result.textStream) {
  process.stdout.write(chunk);
}

Structured Output / JSON Mode

Request structured JSON responses with a schema:

import { generateText } from 'ai';
import { eigen } from '@layr-labs/ai-gateway-provider';

const result = await generateText({
  model: eigen('anthropic/claude-sonnet-4.6'),
  prompt: 'Extract the name and age from: "John is 30 years old."',
  responseFormat: {
    type: 'json',
    schema: {
      type: 'object',
      properties: {
        name: { type: 'string' },
        age: { type: 'number' },
      },
    },
  },
});

console.log(JSON.parse(result.text)); // { name: "John", age: 30 }

Using Multiple Models

import { generateText } from 'ai';
import { eigen } from '@layr-labs/ai-gateway-provider';

// Use different models
const sonnet = eigen('anthropic/claude-sonnet-4.6');
const gpt4 = eigen('openai/gpt-4o');

const [result1, result2] = await Promise.all([
  generateText({ model: sonnet, prompt: 'Hello' }),
  generateText({ model: gpt4, prompt: 'Hello' }),
]);

Image Generation

import { generateImage } from 'ai';
import { eigen } from '@layr-labs/ai-gateway-provider';

const result = await generateImage({
  model: eigen.image('dall-e-3'),
  prompt: 'A cat wearing a space helmet',
});

console.log(result.image); // base64 image data

Video Generation

import { eigen } from '@layr-labs/ai-gateway-provider';

const model = eigen.video('google/veo-3.0-generate-001');
// Video generation uses SSE streaming and returns video data

Tool Calls

import { generateText, tool } from 'ai';
import { eigen } from '@layr-labs/ai-gateway-provider';
import { z } from 'zod';

const result = await generateText({
  model: eigen('anthropic/claude-sonnet-4.6'),
  prompt: 'What is the weather in San Francisco?',
  tools: {
    getWeather: tool({
      description: 'Get the weather for a location',
      parameters: z.object({
        location: z.string().describe('The location to get weather for'),
      }),
      execute: async ({ location }) => {
        // Your weather API call here
        return { temperature: 72, condition: 'sunny' };
      },
    }),
  },
});

Without Authentication

If your gateway doesn't require authentication, you can omit the JWT:

import { createEigenGateway } from '@layr-labs/ai-gateway-provider';

const eigenGateway = createEigenGateway({
  baseURL: 'https://ai-gateway-dev.eigencloud.xyz',
});

Custom Headers

Add custom headers to all requests:

import { createEigenGateway } from '@layr-labs/ai-gateway-provider';

const eigenGateway = createEigenGateway({
  baseURL: 'https://ai-gateway-dev.eigencloud.xyz',
  headers: {
    'X-Custom-Header': 'value',
    'X-Organization-Id': 'org-123',
  },
});

Debug Mode

Enable debug logging to see request details:

const eigenGateway = createEigenGateway({
  baseURL: 'https://ai-gateway-dev.eigencloud.xyz',
  debug: true, // Logs requests and responses
});

You can also enable debug mode globally via the DEBUG=true environment variable.

Custom Timeout

Set a custom request timeout:

const eigenGateway = createEigenGateway({
  baseURL: 'https://ai-gateway-dev.eigencloud.xyz',
  timeout: 60000, // 60 seconds
});

Default timeouts: 30s for chat, 60s for images, 120s for video.

Authentication

In a TEE environment, authentication is automatic. The provider uses attestation to obtain and refresh JWTs when KMS_SERVER_URL and KMS_PUBLIC_KEY are set (these are exported automatically by the compute environment).

You can also provide a JWT manually via the jwt config option or KMS_AUTH_JWT env var to bypass attestation.

Using Environment Variables

The pre-configured eigen export automatically reads from environment variables:

# .env
EIGEN_GATEWAY_URL=https://ai-gateway-dev.eigencloud.xyz
# KMS_AUTH_JWT=optional-jwt-override
# DEBUG=true

import { eigen } from '@layr-labs/ai-gateway-provider';
import { generateText } from 'ai';

// Automatically configured from environment
const result = await generateText({
  model: eigen('anthropic/claude-sonnet-4.6'),
  prompt: 'Hello!',
});

Configuration Options

`EigenGatewayProviderConfig`

| Option | Type | Required | Default | Description | |--------|------|----------|---------|-------------| | baseURL | string | Yes | - | Base URL of your Eigen Gateway | | jwt | string | No | - | Optional JWT override (bypasses attestation) | | attestConfig | AttestClientConfig | No | - | Config for attestation-based JWT auth (auto-configured from env in TEE) | | headers | Record<string, string> | No | {} | Additional headers for all requests | | timeout | number | No | 30000 | Request timeout in milliseconds | | fetch | typeof fetch | No | globalThis.fetch | Custom fetch implementation | | debug | boolean | No | false | Enable debug logging |

Examples

Next.js API Route

// app/api/chat/route.ts
import { createEigenGateway } from '@layr-labs/ai-gateway-provider';
import { streamText } from 'ai';

const eigenGateway = createEigenGateway({
  baseURL: process.env.EIGEN_GATEWAY_URL!,
});

export async function POST(req: Request) {
  const { messages } = await req.json();

  const result = await streamText({
    model: eigenGateway('anthropic/claude-sonnet-4.6'),
    messages,
  });

  return result.toDataStreamResponse();
}

React Component

'use client';

import { useChat } from 'ai/react';

export default function Chat() {
  const { messages, input, handleInputChange, handleSubmit } = useChat({
    api: '/api/chat',
  });

  return (
    <div>
      {messages.map(m => (
        <div key={m.id}>
          <strong>{m.role}:</strong> {m.content}
        </div>
      ))}

      <form onSubmit={handleSubmit}>
        <input value={input} onChange={handleInputChange} />
        <button type="submit">Send</button>
      </form>
    </div>
  );
}

Node.js Script

import { createEigenGateway } from '@layr-labs/ai-gateway-provider';
import { generateText } from 'ai';

async function main() {
  const eigenGateway = createEigenGateway({
    baseURL: 'https://ai-gateway-dev.eigencloud.xyz',
  });

  const result = await generateText({
    model: eigenGateway('anthropic/claude-sonnet-4.6'),
    prompt: 'Hello, world!',
  });

  console.log(result.text);
}

main().catch(console.error);

Provider Class

For reusable configuration across multiple model types, use the EigenGatewayProvider class:

import { EigenGatewayProvider } from '@layr-labs/ai-gateway-provider';

const provider = new EigenGatewayProvider({
  baseURL: 'https://ai-gateway-dev.eigencloud.xyz',
});

const claude = provider.model('anthropic/claude-sonnet-4.6');
const dalle = provider.image('dall-e-3');
const veo = provider.video('google/veo-3.0-generate-001');

TypeScript Support

This package is written in TypeScript and provides full type definitions:

import type {
  EigenGatewayProviderConfig,
  ModelRequestOptions,
} from '@layr-labs/ai-gateway-provider';

Error Handling

The provider throws descriptive errors for common issues:

try {
  const result = await generateText({
    model: eigen('anthropic/claude-sonnet-4.6'),
    prompt: 'Hello',
  });
} catch (error) {
  if (error.message.includes('401')) {
    console.error('Authentication failed - check your JWT token');
  } else if (error.message.includes('429')) {
    console.error('Rate limit exceeded');
  } else {
    console.error('Request failed:', error);
  }
}

Vercel AI SDK Documentation