Bedrock Limiter SDK - TypeScript/JavaScript

Drop-in replacement for AWS SDK v3's BedrockRuntimeClient that adds token limiting, user identification, and API key authentication.

Features

• Drop-in Replacement: Works exactly like AWS SDK v3's BedrockRuntimeClient
• Automatic Authentication: Injects X-User-ID and X-API-Key headers automatically
• Full Compatibility: All AWS SDK v3 Bedrock commands work (ConverseCommand, ConverseStreamCommand, etc.)
• Framework Support: Works with Langchain, Strands SDK, and any framework that accepts AWS SDK v3 clients
• Streaming Support: Fully supports streaming responses
• TypeScript: Full TypeScript support with type definitions

Installation

From GitHub (Recommended for Internal Use)

Add to your package.json:

{
  "dependencies": {
    "bedrock-limiter-sdk": "github:Tire-Rack-Innovation/token-limiter#main:sdk/typescript"
  }
}

Or install directly:

# Install from main branch
npm install github:Tire-Rack-Innovation/token-limiter#main:sdk/typescript

# Install specific version (when tagged)
npm install github:Tire-Rack-Innovation/token-limiter#v1.0.0:sdk/typescript

Requirements

• Node.js 18+
• @aws-sdk/client-bedrock-runtime ^3.0.0

Quick Start

import { BedrockClient } from 'bedrock-limiter-sdk';
import { ConverseCommand } from '@aws-sdk/client-bedrock-runtime';

// Create authenticated client
const bedrock = new BedrockClient({
  userId: '[email protected]',
  apiKey: 'your-api-key-here',
  endpointUrl: 'https://your-alb.elb.amazonaws.com'
});

// Use exactly like AWS SDK v3 client
const response = await bedrock.send(new ConverseCommand({
  modelId: 'anthropic.claude-3-5-sonnet-20241022-v2:0',
  messages: [
    {
      role: 'user',
      content: [{ text: 'Hello! How are you?' }]
    }
  ]
}));

console.log(response.output.message.content[0].text);

Usage Examples

Basic Conversation

import { BedrockClient } from 'bedrock-limiter-sdk';
import { ConverseCommand } from '@aws-sdk/client-bedrock-runtime';

const bedrock = new BedrockClient({
  userId: '[email protected]',
  apiKey: 'abc123...',
  endpointUrl: 'https://bedrock-limiter.elb.amazonaws.com'
});

const response = await bedrock.send(new ConverseCommand({
  modelId: 'anthropic.claude-3-haiku-20240307-v1:0',
  messages: [
    { role: 'user', content: [{ text: 'What is AWS Bedrock?' }] }
  ]
}));

console.log(response.output.message.content[0].text);

Streaming Responses

import { BedrockClient } from 'bedrock-limiter-sdk';
import { ConverseStreamCommand } from '@aws-sdk/client-bedrock-runtime';

const bedrock = new BedrockClient({
  userId: '[email protected]',
  apiKey: 'abc123...',
  endpointUrl: 'https://bedrock-limiter.elb.amazonaws.com'
});

const response = await bedrock.send(new ConverseStreamCommand({
  modelId: 'anthropic.claude-3-haiku-20240307-v1:0',
  messages: [
    { role: 'user', content: [{ text: 'Write a short poem' }] }
  ]
}));

// Process streaming events
if (response.stream) {
  for await (const event of response.stream) {
    if (event.contentBlockDelta?.delta?.text) {
      process.stdout.write(event.contentBlockDelta.delta.text);
    }
  }
}
console.log(); // Newline after stream completes

With Langchain

import { BedrockClientForLangchain } from 'bedrock-limiter-sdk';
import { ChatBedrock } from '@langchain/aws';

// Create authenticated client
const bedrockClient = new BedrockClientForLangchain({
  userId: '[email protected]',
  apiKey: 'abc123...',
  endpointUrl: 'https://bedrock-limiter.elb.amazonaws.com'
});

// Use with Langchain
const llm = new ChatBedrock({
  client: bedrockClient,
  model: 'anthropic.claude-3-haiku-20240307-v1:0',
  streaming: true
});

const response = await llm.invoke('What are the benefits of using AWS?');
console.log(response.content);

With Strands SDK

import { BedrockClientForStrands } from 'bedrock-limiter-sdk';
// Note: Exact Strands SDK integration depends on their API

const bedrockClient = new BedrockClientForStrands({
  userId: '[email protected]',
  apiKey: 'abc123...',
  endpointUrl: 'https://bedrock-limiter.elb.amazonaws.com'
});

// Use with Strands SDK
// (Integration details depend on Strands API)

JavaScript (CommonJS)

const { BedrockClient } = require('bedrock-limiter-sdk');
const { ConverseCommand } = require('@aws-sdk/client-bedrock-runtime');

const bedrock = new BedrockClient({
  userId: '[email protected]',
  apiKey: 'abc123...',
  endpointUrl: 'https://bedrock-limiter.elb.amazonaws.com'
});

async function main() {
  const response = await bedrock.send(new ConverseCommand({
    modelId: 'anthropic.claude-3-haiku-20240307-v1:0',
    messages: [
      { role: 'user', content: [{ text: 'Hello!' }] }
    ]
  }));

  console.log(response.output.message.content[0].text);
}

main();

API Reference

BedrockClient

Main client class that extends AWS SDK v3's BedrockRuntimeClient.

Constructor:

new BedrockClient(config: BedrockLimiterConfig)

Config Parameters:

• userId (string, required): Your user identifier (email, username, etc.)
• apiKey (string, required): Your API key (obtain from administrator)
• endpointUrl (string, required): Token limiter endpoint URL
• region (string, optional): AWS region (default: 'us-east-1')
• ...all other AWS SDK v3 BedrockRuntimeClientConfig options

Methods:

All standard AWS SDK v3 Bedrock methods via .send():

• ConverseCommand - Standard conversation
• ConverseStreamCommand - Streaming conversation
• InvokeModelCommand - Legacy API
• InvokeModelWithResponseStreamCommand - Legacy streaming API
• And all other bedrock-runtime commands

Helper Classes

BedrockClientForLangchain

class BedrockClientForLangchain extends BedrockClient

• Same as BedrockClient, optimized for Langchain integration
• Use when integrating with Langchain frameworks

BedrockClientForStrands

class BedrockClientForStrands extends BedrockClient

• Same as BedrockClient, optimized for Strands SDK integration
• Use when integrating with AWS Strands Agents

createBedrockClient() Function

Factory function for creating a client.

import { createBedrockClient } from 'bedrock-limiter-sdk';

const bedrock = createBedrockClient({
  userId: '[email protected]',
  apiKey: 'abc123...',
  endpointUrl: 'https://bedrock-limiter.elb.amazonaws.com'
});

Type Definitions

interface BedrockLimiterConfig extends Partial<BedrockRuntimeClientConfig> {
  userId: string;
  apiKey: string;
  endpointUrl: string;
  region?: string;
}

How It Works

The SDK extends AWS SDK v3's BedrockRuntimeClient and uses middleware to automatically inject authentication headers before each request:

1. You create a BedrockClient with your credentials
2. The SDK creates a standard AWS SDK client pointing to your token limiter endpoint
3. Before each API call, middleware injects X-User-ID and X-API-Key headers
4. Your token limiter middleware validates the request and tracks token usage
5. The request is proxied to AWS Bedrock
6. The response is returned normally to your application

Token Limiting

Token usage is tracked per user and per model. When you exceed your limit:

• Non-streaming requests: Throws error with details
• Streaming requests: Stream is interrupted with error event

Check with your administrator for your current limits.

Error Handling

import { BedrockClient } from 'bedrock-limiter-sdk';
import { ConverseCommand } from '@aws-sdk/client-bedrock-runtime';

const bedrock = new BedrockClient({
  userId: '[email protected]',
  apiKey: 'abc123...',
  endpointUrl: 'https://bedrock-limiter.elb.amazonaws.com'
});

try {
  const response = await bedrock.send(new ConverseCommand({
    modelId: 'anthropic.claude-3-haiku-20240307-v1:0',
    messages: [{ role: 'user', content: [{ text: 'Hello' }] }]
  }));
} catch (error: any) {
  if (error.name === 'TokenLimitExceeded') {
    console.error('Token limit exceeded:', error.message);
  } else if (error.name === 'Unauthorized') {
    console.error('Authentication failed:', error.message);
  } else {
    console.error('Error:', error.message);
  }
}

Troubleshooting

Module Not Found

If you get Cannot find module 'bedrock-limiter-sdk':

# Make sure you installed from GitHub
npm install github:Tire-Rack-Innovation/token-limiter#main:sdk/typescript

# Or check your package.json has the correct dependency

TypeScript Errors

If you get TypeScript errors:

1. Ensure @aws-sdk/client-bedrock-runtime is installed
2. Check that your tsconfig.json has proper module resolution
3. Verify Node.js version >= 18

Authentication Failed

If you get authentication errors:

1. Verify your API key is correct
2. Check that the endpoint URL is correct
3. Ensure your userId matches what's in the system

Connection Errors

If you get connection timeout or refused errors:

1. Verify the endpoint URL is accessible from your network
2. Check VPC/security group settings if using private endpoints
3. Ensure the token limiter service is running

Development

Building from Source

git clone https://github.com/Tire-Rack-Innovation/token-limiter.git
cd token-limiter/sdk/typescript

# Install dependencies
npm install

# Build the package
npm run build

# Link for local development
npm link

Using Locally Built Package

# In your project directory
npm link bedrock-limiter-sdk

Running Tests

# Unit tests
npm test

# Build and test
npm run build && npm test

Support

For issues or questions:

1. Check the TypeScript Installation Guide
2. Check the main repository documentation
3. Contact the platform team
4. File an issue on GitHub

License

MIT License - See LICENSE file for details

Version History

See CHANGELOG.md for version history and changes.