Foundry AI SDK Provider

A custom AI SDK provider for Palantir Foundry Language Model Service, enabling seamless integration with Vercel's AI SDK.

Features

✅ Multiple Model Support

• GPT models (GPT-4, GPT-5) with full tool calling support
• Gemini models (Gemini 2.0/2.5 Flash, Pro) for text and vision
• Automatic model detection and adapter selection

✅ Tool Calling

• Full tool/function calling support for GPT models
• Proper warnings for models that don't support tools
• Workaround for AI SDK v6 beta.93 schema serialization bug

✅ Streaming Support

• Text streaming for all models
• Real-time response generation

✅ Vision Support

• Image understanding with Gemini models
• Base64 encoded image support

✅ Production Ready

• Type-safe TypeScript implementation
• Comprehensive error handling
• Token usage tracking
• Request/response debugging

Installation

pnpm install

Quick Start

Basic Setup

import { createFoundry } from '@northslopetech/foundry-ai-sdk';
import { generateText } from 'ai';

const foundry = createFoundry({
  foundryToken: process.env.FOUNDRY_TOKEN!,
  baseURL: process.env.FOUNDRY_BASE_URL!,
});

// Simple text generation
const result = await generateText({
  model: foundry('Gemini_2_0_Flash'),
  prompt: 'What is 2+2?',
});

console.log(result.text);

Tool Calling with GPT Models

Important: Due to a bug in AI SDK v6 beta.93, tool schemas must be passed via providerOptions.foundry.parameters:

import { generateText } from 'ai';
import { createFoundry } from '@northslopetech/foundry-ai-sdk';

const foundry = createFoundry({
  foundryToken: process.env.FOUNDRY_TOKEN!,
  baseURL: process.env.FOUNDRY_BASE_URL!,
});

// Define your tool schema
const weatherToolSchema = {
  type: 'object' as const,
  properties: {
    location: {
      type: 'string' as const,
      description: 'The city and state, e.g. San Francisco, CA',
    },
    unit: {
      type: 'string' as const,
      enum: ['celsius', 'fahrenheit'],
    },
  },
  required: ['location'],
};

// WORKAROUND: Pass schema in both parameters AND providerOptions
const weatherTool = {
  description: 'Get the current weather in a given location',
  parameters: weatherToolSchema,
  providerOptions: {
    foundry: {
      parameters: weatherToolSchema, // Required for AI SDK beta.93
    },
  },
  execute: async ({ location, unit }) => {
    // Your tool implementation
    return {
      location,
      temperature: unit === 'celsius' ? 18 : 65,
      unit: unit || 'fahrenheit',
      conditions: 'Partly cloudy',
    };
  },
};

const result = await generateText({
  model: foundry('GPT_5'),
  prompt: 'What is the weather in San Francisco?',
  tools: { get_weather: weatherTool },
  maxTokens: 500,
});

console.log(result.text);

Streaming

import { streamText } from 'ai';

const result = streamText({
  model: foundry('Gemini_2_0_Flash'),
  prompt: 'Write a short poem about AI',
});

for await (const chunk of result.textStream) {
  process.stdout.write(chunk);
}

Vision (Image Understanding)

import { generateText } from 'ai';
import * as fs from 'fs';

const imageBuffer = fs.readFileSync('image.png');
const base64Image = imageBuffer.toString('base64');

const result = await generateText({
  model: foundry('Gemini_2_0_Flash'),
  messages: [
    {
      role: 'user',
      content: [
        { type: 'text', text: 'What is in this image?' },
        {
          type: 'image',
          image: base64Image,
          mimeType: 'image/png',
        },
      ],
    },
  ],
});

console.log(result.text);

Supported Models

GPT Models (with Tool Calling)

• GPT_4
• GPT_5
• Full tool/function calling support
• Vision support via gptChatWithVision

Gemini Models (Text & Vision)

• Gemini_2_0_Flash
• Gemini_2_5_Flash
• Gemini_2_5_Pro
• Note: Tool calling not supported by Foundry's Gemini implementation

Model Detection

The provider automatically detects the model family and applies the appropriate adapter:

// Automatically uses GPT adapter with tool support
const gptModel = foundry('GPT_5');

// Automatically uses generic adapter (no tools)
const geminiModel = foundry('Gemini_2_0_Flash');

Configuration

Environment Variables

Create a .env file:

FOUNDRY_TOKEN=your_foundry_token
FOUNDRY_BASE_URL=https://yourinstance.palantirfoundry.com

Provider Options

const foundry = createFoundry({
  foundryToken: 'your-token',           // Required
  baseURL: 'https://...',               // Required
  headers: {                            // Optional
    'Custom-Header': 'value',
  },
});

Model Settings

const model = foundry('GPT_5', {
  temperature: 0.7,    // Optional
  maxTokens: 1000,     // Optional
});

Debugging

Enable debug logging to see request/response details:

DEBUG_FOUNDRY=true pnpm tsx your-script.ts

This will log:

• Tool schemas being sent
• Request bodies
• Response parsing
• Error details

Known Issues & Workarounds

AI SDK v6 Beta.93 Tool Schema Bug

Issue: The AI SDK doesn't properly serialize tool schemas, sending empty inputSchema to providers.

Workaround: Pass schemas via providerOptions.foundry.parameters:

const tool = {
  description: '...',
  parameters: schema,
  providerOptions: {
    foundry: { parameters: schema }, // Add this
  },
  execute: async (args) => { ... },
};

Foundry GPT Quirks

1. tool_choice parameter: Foundry's GPT endpoint rejects the tool_choice parameter. The provider automatically omits it.

2. temperature parameter: Some GPT configurations reject temperature. The provider omits it for GPT requests.

3. Response format: Foundry wraps tool calls in a toolCall object. The provider handles both standard OpenAI and Foundry formats.

Architecture

┌─────────────────────────────────────────────┐
│           Vercel AI SDK (v6 beta)           │
│         (generateText, streamText)          │
└────────────────┬────────────────────────────┘
                 │
                 ▼
┌─────────────────────────────────────────────┐
│        FoundryLanguageModel                 │
│     (implements LanguageModelV2)            │
│                                             │
│  ┌─────────────────────────────────────┐   │
│  │  Model Detection                    │   │
│  │  (detectModel)                      │   │
│  └─────────────────────────────────────┘   │
│                 │                           │
│                 ▼                           │
│  ┌──────────────┴──────────────┐           │
│  │                              │           │
│  ▼                              ▼           │
│ GPT Adapter              Generic Adapter    │
│ ├─ Request Adapter       (Gemini, etc)      │
│ └─ Response Parser       - genericVision    │
│    - Tool calling           - No tools      │
│    - gptChat format         - Warnings      │
└─────────────────────────────────────────────┘
                 │
                 ▼
┌─────────────────────────────────────────────┐
│     Foundry Language Model Service          │
│   /llm/v3/completion/{modelId}              │
│   /llm/v3/completion/{modelId}/stream       │
└─────────────────────────────────────────────┘

API Reference

createFoundry(options)

Creates a Foundry provider instance.

Parameters:

• foundryToken: Your Foundry authentication token
• baseURL: Your Foundry instance URL
• headers?: Optional additional headers

Returns: FoundryProvider

Model Interface

interface FoundryLanguageModel extends LanguageModelV2 {
  provider: string;           // 'foundry'
  modelId: string;           // e.g., 'GPT_5'
  settings: {
    temperature?: number;
    maxTokens?: number;
  };

  doGenerate(options): Promise<GenerateResult>;
  doStream(options): Promise<StreamResult>;
}

Examples

See the examples/ directory for complete working examples:

• basic-text-completion.ts - Simple text generation
• gpt-tool-calling.ts - Tool calling with GPT models
• vision-completion.ts - Image understanding
• test-all-features.ts - Comprehensive feature test
• complete-workflow-test.ts - Full multi-step workflow

Run examples:

pnpm tsx examples/basic-text-completion.ts
pnpm tsx examples/gpt-tool-calling.ts

Testing

# Run all tests
pnpm test:all

# Run specific tests
pnpm test:basic
pnpm test:vision
pnpm test:gpt-tools

# Verify API connectivity
pnpm verify
pnpm verify:gpt

Development

Project Structure

src/
├── index.ts                      # Main exports
├── foundry-provider.ts           # Provider factory
├── foundry-language-model.ts     # Core model implementation
├── foundry-types.ts              # Foundry API types
├── foundry-settings.ts           # Configuration types
├── foundry-prompt-converter.ts   # Message format conversion
├── model-detector.ts             # Model family detection
└── adapters/
    ├── adapter-interface.ts      # Adapter contracts
    ├── gpt-request-adapter.ts    # GPT request conversion
    ├── gpt-response-parser.ts    # GPT response parsing
    └── generic-chat-request-adapter.ts

examples/
├── basic-text-completion.ts
├── gpt-tool-calling.ts
├── vision-completion.ts
└── test-all-features.ts

scripts/
├── verify-chat-completion.ts
├── verify-vision-completion.ts
└── verify-gpt-tool-calling.ts

Adding New Model Adapters

1. Create request adapter implementing RequestAdapter
2. Create response parser implementing ResponseParser
3. Add model detection logic in model-detector.ts
4. Register adapter in FoundryLanguageModel constructor

Example:

if (this.modelInfo.family === 'claude') {
  this.adapter = new ClaudeRequestAdapter();
  this.parser = new ClaudeResponseParser();
}

Contributing

Contributions welcome! Please ensure:

1. All tests pass: pnpm test
2. Code is formatted: pnpm format
3. Examples work: pnpm test:all

License

MIT

Support

For issues related to:

• This provider: Open an issue on GitHub
• Foundry LMS: Contact Palantir support
• AI SDK: Check Vercel AI SDK docs

Changelog

v0.1.0 (Current)

• ✅ Initial release
• ✅ GPT model support with tool calling
• ✅ Gemini model support (text & vision)
• ✅ Streaming support
• ✅ Vision support
• ✅ Comprehensive error handling
• ✅ AI SDK v6 beta compatibility
• ⚠️ Workaround for AI SDK beta.93 tool schema bug

Acknowledgments

Built for Northslope Technologies to enable AI-powered applications on Palantir Foundry.