@railtownai/railengine

JavaScript/TypeScript SDK for Railtown AI Rail Engine - Retrieval package. This package provides a simple interface for reading and searching data from Rail Engine.

Installation

npm install @railtownai/railengine

Requirements

• Node.js 20+
• TypeScript 5.0+ (optional, but recommended)

Quick Start

Basic Usage

import { RailEngine } from "@railtownai/railengine";

// Initialize client
const client = new RailEngine({
  pat: "your-pat-token",
  engineId: "your-engine-id"
});

// Search vector store
const results = client.searchVectorStore("engine-id", "VectorStore1", "search query");
for await (const result of results) {
  console.log(result);
}

// Get storage document by EventId
const document = await client.getStorageDocumentByEventId("engine-id", "event-id");
console.log(document);

Using Environment Variables

import { RailEngine } from "@railtownai/railengine";

// ENGINE_PAT and ENGINE_ID are read from environment variables automatically
const client = new RailEngine();

// Use client.engineId if you need it
const results = client.searchVectorStore(client.engineId, "VectorStore1", "query");
for await (const result of results) {
  console.log(result);
}

Authentication Configuration

The retrieval package uses PAT (Personal Access Token) for authentication.

Environment Variables

• ENGINE_PAT - Personal Access Token (required)
• ENGINE_ID - Engine ID (optional, but required if not provided via constructor)
• RAILTOWN_API_URL - Base API URL (optional, defaults to https://cndr.railtown.ai/api)

Setting Environment Variables

Windows (PowerShell)

$env:ENGINE_PAT="<your-pat-token>"
$env:ENGINE_ID="<your-engine-id>"
$env:RAILTOWN_API_URL="https://cndr.railtown.ai/api"  # Optional

Windows (Command Prompt)

set ENGINE_PAT=<your-pat-token>
set ENGINE_ID=<your-engine-id>
set RAILTOWN_API_URL=https://cndr.railtown.ai/api

macOS/Linux

export ENGINE_PAT="<your-pat-token>"
export ENGINE_ID="<your-engine-id>"
export RAILTOWN_API_URL="https://cndr.railtown.ai/api"  # Optional

.env file

Create a .env file in your project root:

ENGINE_PAT=<your-pat-token>
ENGINE_ID=<your-engine-id>
RAILTOWN_API_URL=https://cndr.railtown.ai/api

Then use a package like dotenv to load it:

import "dotenv/config";
import { RailEngine } from "@railtownai/railengine";

const client = new RailEngine(); // Reads from environment variables

Configuration Priority

1. Constructor parameters (highest priority)
2. Environment variables
3. Default values (where applicable)

Note: If ENGINE_ID is set in environment variables, you don't need to specify it in client initialization. If ENGINE_ID is NOT set in environment variables, you MUST specify it in client initialization.

Zod Schema Support

You can provide a Zod schema during client initialization to automatically validate retrieved data:

import { z } from "zod";
import { RailEngine } from "@railtownai/railengine";

const FoodDiaryItemSchema = z.object({
  food_name: z.string(),
  calories: z.number(),
  carbs: z.number(),
  proteins: z.number(),
  fats: z.number()
});

type FoodDiaryItem = z.infer<typeof FoodDiaryItemSchema>;

// Initialize with schema
const client = new RailEngine({
  pat: "your-pat-token",
  engineId: "your-engine-id",
  schema: FoodDiaryItemSchema
});

// Results automatically validated against FoodDiaryItemSchema
const results = client.searchVectorStore("engine-id", "VectorStore1", "apple");
for await (const item of results) {
  // item is FoodDiaryItem
  console.log(item.food_name, item.calories);
}

Overriding Schema Per Call

You can override the default schema on individual method calls:

const FoodSummarySchema = z.object({
  name: z.string(),
  cal: z.number()
});

const results = client.searchVectorStore("engine-id", "VectorStore1", "apple", {
  schema: FoodSummarySchema // Override default schema
});

Client-Side Filtering

All retrieval methods support optional client-side filtering via filterFn:

// Filter results by CustomerKey pattern
const results = client.searchVectorStore("engine-id", "VectorStore1", "query", {
  filterFn: (item) => {
    const customerKey = (item as { CustomerKey?: string }).CustomerKey || "";
    return customerKey.startsWith("doc-");
  }
});

for await (const result of results) {
  console.log(result);
}

Filtering with Pagination

Filtering works seamlessly with pagination - filters are applied across all pages automatically:

// List documents and filter by version
const documents = client.listStorageDocuments("engine-id", {
  filterFn: (doc) => ((doc as { Version?: number }).Version || 0) > 1
});

for await (const doc of documents) {
  console.log(doc);
}

API Methods

Embeddings API

searchVectorStore

Search a vector store using semantic similarity.

const results = client.searchVectorStore(
  engineId: string,
  vectorStore: string,
  query: string,
  options?: {
    filterFn?: (item: T | Record<string, unknown>) => boolean;
    schema?: z.ZodSchema<unknown>;
  }
): AsyncIterable<T | Record<string, unknown>>

Example:

const results = client.searchVectorStore("engine-id", "VectorStore1", "apple");
for await (const result of results) {
  console.log(result);
}

Storage API

getStorageDocumentByEventId

Get a single storage document by EventId.

const document = await client.getStorageDocumentByEventId(
  engineId: string,
  eventId: string,
  options?: {
    filterFn?: (document: T | Record<string, unknown>) => boolean;
    schema?: z.ZodSchema<unknown>;
  }
): Promise<T | Record<string, unknown> | null>

Example:

const document = await client.getStorageDocumentByEventId("engine-id", "event-id");
if (document) {
  console.log(document);
}

getStorageDocumentByCustomerKey

Get storage documents by CustomerKey with automatic pagination.

const documents = client.getStorageDocumentByCustomerKey(
  engineId: string,
  customerKey: string,
  options?: {
    pageNumber?: number;  // Default: 1
    pageSize?: number;    // Default: 25, max: 100
    filterFn?: (item: T | Record<string, unknown>) => boolean;
    schema?: z.ZodSchema<unknown>;
  }
): AsyncIterable<T | Record<string, unknown>>

Example:

const documents = client.getStorageDocumentByCustomerKey("engine-id", "doc-123", {
  pageSize: 50
});

for await (const doc of documents) {
  console.log(doc);
}

queryStorageByJsonPath

Query storage documents using JSONPath.

const documents = client.queryStorageByJsonPath(
  engineId: string,
  jsonPathQuery: string,
  options?: {
    filterFn?: (item: T | Record<string, unknown>) => boolean;
    schema?: z.ZodSchema<unknown>;
  }
): AsyncIterable<T | Record<string, unknown>>

Example:

const documents = client.queryStorageByJsonPath("engine-id", "$.status:active");
for await (const doc of documents) {
  console.log(doc);
}

listStorageDocuments

List storage documents with automatic pagination.

const documents = client.listStorageDocuments(
  engineId: string,
  options?: {
    customerKey?: string;
    pageNumber?: number;  // Default: 1
    pageSize?: number;    // Default: 100, max: 100
    filterFn?: (item: T | Record<string, unknown>) => boolean;
    schema?: z.ZodSchema<unknown>;
  }
): AsyncIterable<T | Record<string, unknown>>

Example:

// List all documents
const documents = client.listStorageDocuments("engine-id");

// List documents filtered by customerKey
const filtered = client.listStorageDocuments("engine-id", {
  customerKey: "doc-123"
});

for await (const doc of documents) {
  console.log(doc);
}

Indexing API

searchIndex

Search index using Azure Search.

const results = client.searchIndex(
  projectId: string,
  engineId: string,
  query: Record<string, unknown>,
  options?: {
    filterFn?: (item: T | Record<string, unknown>) => boolean;
    schema?: z.ZodSchema<unknown>;
  }
): AsyncIterable<T | Record<string, unknown>>

Example:

const results = client.searchIndex("project-id", "engine-id", {
  search: "example query"
});

for await (const result of results) {
  console.log(result);
}

Pagination

Pagination is handled automatically by the SDK. Methods that return multiple results use async iterables that handle pagination internally:

// Automatically handles pagination - you don't need to manage pages manually
const documents = client.listStorageDocuments("engine-id", {
  pageSize: 100
});

// Iterate through all pages automatically
for await (const document of documents) {
  console.log(document);
}

Filtering is applied across all pages automatically:

// Filter is applied to all pages, not just the first page
const documents = client.listStorageDocuments("engine-id", {
  filterFn: (doc) => {
    const customerKey = (doc as { CustomerKey?: string }).CustomerKey || "";
    return customerKey.startsWith("doc-");
  }
});

for await (const doc of documents) {
  // All documents from all pages that match the filter
  console.log(doc);
}

Complete Examples

Example: Building a Simple Agent

import { RailEngine } from "@railtownai/railengine";
import { z } from "zod";

const DocumentSchema = z.object({
  id: z.string(),
  title: z.string(),
  content: z.string(),
  tags: z.array(z.string())
});

const client = new RailEngine({
  pat: process.env.ENGINE_PAT!,
  engineId: process.env.ENGINE_ID!,
  schema: DocumentSchema
});

async function searchDocuments(query: string) {
  // Search vector store
  const vectorResults = client.searchVectorStore(client.engineId, "VectorStore1", query, {
    filterFn: (doc) => {
      const tags = (doc as { tags?: string[] }).tags || [];
      return tags.includes("public");
    }
  });

  const results: unknown[] = [];
  for await (const result of vectorResults) {
    results.push(result);
  }

  return results;
}

// Usage
const docs = await searchDocuments("machine learning");
console.log(`Found ${docs.length} documents`);

Example: Retrieving and Filtering Documents

import { RailEngine } from "@railtownai/railengine";

const client = new RailEngine({
  pat: process.env.ENGINE_PAT!,
  engineId: process.env.ENGINE_ID!
});

// Get document by EventId
const doc = await client.getStorageDocumentByEventId("engine-id", "event-123");
if (doc) {
  console.log("Found document:", doc);
}

// Get documents by CustomerKey with filtering
const customerDocs = client.getStorageDocumentByCustomerKey("engine-id", "customer-123", {
  filterFn: (doc) => {
    const version = (doc as { Version?: number }).Version || 0;
    const status = (doc as { Status?: string }).Status;
    return version > 1 && status === "active";
  }
});

for await (const doc of customerDocs) {
  console.log("Customer document:", doc);
}

// Query using JSONPath
const activeDocs = client.queryStorageByJsonPath("engine-id", "$.status:active", {
  filterFn: (doc) => {
    const dateCreated = (doc as { DateCreated?: string }).DateCreated;
    return dateCreated ? new Date(dateCreated) > new Date("2024-01-01") : false;
  }
});

for await (const doc of activeDocs) {
  console.log("Active document:", doc);
}

API Reference

RailEngine

Main client class for retrieving data from Rail Engine.

Constructor

new RailEngine(options?: RailEngineOptions<T>)

Options:

• pat?: string - Personal Access Token. If not provided, reads from ENGINE_PAT environment variable.
• engineId?: string - Engine ID. If not provided, reads from ENGINE_ID environment variable.
• apiUrl?: string - Base API URL. If not provided, reads from RAILTOWN_API_URL environment variable or defaults to production.
• schema?: z.ZodSchema<T> - Optional Zod schema for validating retrieved data.
• timeout?: number - Request timeout in milliseconds. Defaults to 30000 (30 seconds).

Example:

const client = new RailEngine({
  pat: "your-pat-token",
  engineId: "your-engine-id",
  schema: MySchema,
  timeout: 60000
});

Properties

• pat: string - Personal Access Token for authentication
• engineId: string - Engine identifier
• apiUrl: string - Base API URL (normalized)

Methods

All methods are documented above in the API Methods section.

Error Handling

The SDK uses graceful error handling for read operations:

• Read operations return null or empty iterables on errors (graceful degradation)
• Configuration errors (missing PAT, engineId) throw exceptions

The SDK uses custom exception classes for different error scenarios:

• RailtownError - Base error class
• RailtownUnauthorizedError - Authentication failed (401)
• RailtownNotFoundError - Resource not found (404)
• RailtownBadRequestError - Bad request (400)
• RailtownConflictError - Conflict (409)
• RailtownServerError - Server error (500+)

Example:

import { RailEngine, RailtownUnauthorizedError, RailtownBadRequestError } from "@railtownai/railengine";

try {
  const client = new RailEngine({
    pat: "invalid-pat",
    engineId: "engine-id"
  });
} catch (error) {
  if (error instanceof RailtownBadRequestError) {
    console.error("Invalid configuration:", error.message);
  }
}

// Read operations return null/empty on error
const document = await client.getStorageDocumentByEventId("engine-id", "event-id");
if (document === null) {
  console.log("Document not found or error occurred");
}

License

See LICENSE file for details.