Ketrics SDK for Backend (@ketrics/sdk-backend)

Overview

The Ketrics SDK for Backend is a TypeScript type definition library that provides the complete interface for building tenant applications on the Ketrics platform. This SDK defines the ketrics global object—a comprehensive runtime API that tenant code uses to access platform features including storage, databases, secrets management, document generation, background jobs, and messaging.

Purpose: This package serves as the type contract between tenant application code and the Ketrics runtime execution environment. It is published as an npm package and consumed by tenant developers building applications that run in isolated VM sandboxes.

Architecture Context:

• Tenant applications run in ECS Fargate containers with Node.js 24+
• The global ketrics object is injected at runtime by the Ketrics platform
• This SDK provides TypeScript definitions so tenant code has IDE support and type safety
• The SDK is a pure type definition library (no implementation)

Key Responsibilities:

1. Define all TypeScript interfaces and types for the ketrics global object
2. Provide comprehensive error classes for all SDK operations
3. Establish the contract between tenant code and the Ketrics runtime
4. Enable type-safe access to platform capabilities (storage, databases, secrets, etc.)

Business Logic

What Problem Does This Solve?

Tenant applications need standardized, type-safe access to platform infrastructure without direct access to underlying services. This SDK enables developers to:

• Read/write files to S3-backed volumes with permission controls
• Query external databases with transaction support
• Retrieve encrypted secrets via KMS
• Generate PDF and Excel documents programmatically
• Execute long-running tasks asynchronously
• Send messages to users with multiple channels (inbox, push notifications)
• Make external API calls with a standardized HTTP client
• Access tenant and application metadata
• Log to CloudWatch with proper context

Core Workflows and Processes

1. Storage Access (Volumes)

• Connect to an S3-backed volume by code
• Get files (with content, metadata, etag)
• Put files (with metadata and conditional create)
• Delete files (single or by prefix)
• List files with pagination and filtering
• Copy/move files within volumes
• Generate presigned URLs for direct browser access

2. Database Operations

• Connect to external databases by code
• Execute SELECT queries with typed results
• Execute INSERT/UPDATE/DELETE with affected row counts
• Run queries within transactions (auto-commit on success, auto-rollback on error)
• Connection pooling and lifecycle management

3. Secrets Management

• Retrieve encrypted secrets by code
• Check if a secret exists before accessing
• Secrets are decrypted using tenant-specific KMS keys
• Subject to application access grants

4. Document Generation

• Excel: Read/parse existing Excel files, create new workbooks, add worksheets, format cells, write files
• PDF: Read/parse existing PDFs, create new documents, draw text/shapes/images, embed fonts, write files

5. Background Jobs

• Queue async function execution with payloads
• Support cross-app jobs with permission validation
• Track job status (pending, running, completed, failed)
• Idempotency keys to prevent duplicate execution
• Configurable timeouts (default 5min, max 15min)

6. User Messaging

• Send messages to individual users
• Bulk send to multiple users
• Send to group members (requires IAM-data permissions)
• Support for multiple channels: inbox (always) + push notifications (optional)
• Priority levels: LOW, MEDIUM, HIGH
• Custom action URLs for deep linking

Business Rules Implemented

Access Control:

• Volume access requires an access grant to the specific volume
• Database access requires an access grant with specific permissions
• Secrets access requires an access grant with decrypt permission
• Messages to groups require IAM-data tenant permissions (via application role)
• Cross-app jobs require application grants with background job permission

Data Boundaries:

• Requestor context distinguishes between users (type: "USER") and service accounts (type: "SERVICE_ACCOUNT")
• Requestor's application permissions determine what operations are allowed
• Each tenant's secrets are encrypted with tenant-specific KMS keys

Input/Output Expectations:

• File operations support streams, buffers, and strings
• Database queries return generic typed result objects
• Job operations return UUID job identifiers
• List operations support pagination with cursors
• All timestamps are ISO 8601 strings

Edge Cases Handled:

• File already exists with ifNotExists option
• File size limits enforced on put operations
• Content type restrictions on volumes
• Invalid path characters and path traversal detection
• Connection timeouts with retry semantics
• Null/undefined secret values
• Transaction rollback on error
• Job timeout after max execution time
• Bulk messaging with partial failures

Technical Details

Technology Stack and Dependencies

Runtime Environment:

• Node.js 24.0.0+ (specified in package.json engines)
• TypeScript 5.0.0+ for compilation
• ES2020 target with CommonJS module format

Package Dependencies:

• @types/node@^24.10.2 - Node.js type definitions for stream types (Readable)
• typescript@^5.0.0 - TypeScript compiler (dev dependency)

Key Type System Features:

• Strict mode enabled for type safety
• Declaration maps for source map debugging
• Module resolution set to "node" for standard npm resolution
• Force consistent casing for file imports

File Structure with Purpose

src/
├── index.ts                    # Main entry point, exports all public types and interfaces
├── context.ts                  # Runtime context types (tenant, app, requestor, environment)
├── console.ts                  # Console logger interface
├── http.ts                     # HTTP client interface for external API calls
├── databases.ts                # Database connection interfaces and types
├── database-errors.ts          # Database error class hierarchy
├── volumes.ts                  # Volume storage interfaces and types
├── errors.ts                   # Volume error class hierarchy
├── secrets.ts                  # Secret retrieval interfaces
├── secret-errors.ts            # Secret error classes
├── excel.ts                    # Excel workbook interfaces and types
├── excel-errors.ts             # Excel error classes
├── pdf.ts                      # PDF document interfaces and types
├── pdf-errors.ts               # PDF error classes
├── job.ts                      # Background job types and interfaces
├── job-errors.ts               # Job execution error classes
├── messages.ts                 # Message sending interfaces and types
├── messages-errors.ts          # Message error classes
└── dist/                       # Compiled JavaScript and .d.ts files (generated)

Key Functions/Classes and Their Purposes

Context Interfaces (context.ts):

• TenantContext: Tenant ID, code, and name
• ApplicationContext: Application ID, code, name, version, deployment ID
• RequestorContext: Identifies if request is from USER or SERVICE_ACCOUNT with permissions
• RuntimeContext: Node.js version, AWS region, runtime platform
• EnvironmentVariables: Tenant application environment variables

SDK Manager Classes (defined in index.ts as static interfaces):

1. Volume - S3-backed file storage

   • connect(volumeCode: string): Promise<IVolume>
   • Methods: get, put, delete, deleteByPrefix, list, copy, move, downloadUrl, uploadUrl

2. DatabaseConnection - External database access

   • connect(databaseCode: string): Promise<IDatabaseConnection>
   • Methods: query(), execute(), transaction(), close()

3. Secret - Encrypted secret retrieval

   • get(secretCode: string): Promise<string>
   • exists(secretCode: string): Promise<boolean>

4. Excel - Excel file operations

   • read(buffer: Buffer): Promise<IExcelWorkbook>
   • create(): IExcelWorkbook
   • Workbook methods: getWorksheet, addWorksheet, writeFile

5. Pdf - PDF document operations

   • read(buffer: Buffer): Promise<IPdfDocument>
   • create(): Promise<IPdfDocument>
   • rgb(r, g, b): PdfRgbColor
   • Document methods: page manipulation, drawing, embedding

6. Job - Background job execution

   • runInBackground(params: RunInBackgroundParams): Promise<string> - Returns job ID
   • getStatus(jobId: string): Promise<JobStatus>
   • list(params?: JobListParams): Promise<JobListResult>

7. Messages - User messaging

   • send(params: SendMessageParams): Promise<SendMessageResult>
   • sendBulk(params: SendBulkMessageParams): Promise<SendBulkMessageResult>
   • sendToGroup(params: SendGroupMessageParams): Promise<SendBulkMessageResult>

Error Class Hierarchy: Each feature domain (Volume, Database, Secret, Excel, PDF, Job, Messages) defines:

• Abstract base error class (extends Error)
• Specific error subclasses for different failure scenarios
• Type guards: is[Feature]Error(), is[Feature]ErrorType()

Example Volume errors:

• VolumeError (abstract base)
• VolumeNotFoundError, VolumeAccessDeniedError, VolumePermissionDeniedError
• FileNotFoundError, FileAlreadyExistsError, InvalidPathError
• FileSizeLimitError, ContentTypeNotAllowedError

Configuration Options and Environment Variables

Tenant Environment Variables (via ketrics.environment):

• Application-defined key-value pairs passed at deployment time
• All keys are uppercase with letters, numbers, underscores only
• Accessed at runtime for configuration (API keys, feature flags, etc.)

Volume Configuration:

• Volume codes define which S3 bucket is accessed
• Permissions defined per volume: ReadObject, CreateObject, UpdateObject, DeleteObject, ListObjects
• Content-type restrictions enforced on put operations
• File size limits enforced per volume

Database Configuration:

• Database codes identify external database servers (PostgreSQL, MySQL, etc.)
• Permissions per database: read, write, admin
• Connection pooling managed by runtime
• Query timeouts configurable

Excel Configuration:

• Tab colors, default row height, default column width configurable per worksheet
• Column definitions with header, key, width

PDF Configuration:

• Standard page sizes: A4, Letter, Legal, Tabloid, A3, A5
• Standard fonts: Courier variants, Helvetica variants, TimesRoman variants, Symbol, ZapfDingbats
• RGB color values (0-1 range)
• Drawing options: size, opacity, rotation, line height, max width

Job Configuration:

• Timeout range: 1ms to 900,000ms (15 minutes), default 300,000ms (5 minutes)
• Idempotency keys prevent duplicate job creation
• Cross-app execution requires application grants

Message Configuration:

• Subject max length: 200 chars
• Body max length: 10,000 chars (markdown supported)
• Priority levels: LOW, MEDIUM, HIGH
• Channels: inbox (always enabled) + push notifications (optional per message)

External Integrations

AWS S3 (Volumes):

• S3 buckets mapped to volume codes
• Presigned URLs for direct browser download/upload
• Versioning support via version IDs
• ETag support for content hashing

External Databases:

• PostgreSQL, MySQL, and other JDBC-compatible databases
• Connection strings and credentials never exposed to tenant code
• Parameterized queries to prevent SQL injection
• Transaction support with automatic rollback

AWS KMS (Secrets):

• Tenant-specific KMS keys for secret encryption
• Secrets decrypted at request time
• Decryption errors surface as SecretDecryptionError

CloudWatch (Logging):

• Console logs forwarded to CloudWatch with tenant/app/requestor context
• Proper log grouping and filtering capabilities

Application Job Queue:

• Background job execution in same or different applications
• Status tracking in centralized job store
• Timeout enforcement and error capture

User Messaging Platform:

• Inbox storage for messages
• Push notification delivery
• Group membership resolution via IAM service

Data Flow

How Data Enters the Component

1. Context Injection: At runtime, the Ketrics platform injects a global ketrics object with:

   • Tenant metadata (ID, code, name)
   • Application metadata (ID, code, version, deploymentId)
   • Requestor information (type, ID, permissions)
   • Runtime environment (Node.js version, AWS region)
   • Environment variables (key-value pairs)

2. Function Parameters: Tenant code receives:

   • HTTP request payloads (JSON, form data, etc.)
   • File uploads to volumes
   • Database query parameters
   • Job payloads
   • Message content from users

3. External Services: Data retrieved from:

   • S3 via volume.get()
   • Database queries via db.query()
   • Secrets via Secret.get()
   • Uploaded Excel/PDF files via Excel.read(), Pdf.read()

Transformations Applied

Volume Operations:

• Raw S3 objects → FileContent (buffer + metadata)
• List operations → Paginated FileInfo arrays with cursors
• Put operations → PutResult with etag and optional versionId
• Copy/move → CopyResult/MoveResult with operation status

Database Operations:

• Raw SQL results → DatabaseQueryResult with typed rows
• Execute operations → DatabaseExecuteResult with affectedRows and insertId
• Transactions → Implicit begin/commit/rollback with error handling

Document Operations:

• Raw file buffers → IExcelWorkbook/IPdfDocument with method interfaces
• Workbook operations → Cells, rows, worksheets with formatting
• PDF drawing → Rendered text, shapes, images on pages

Job Operations:

• RunInBackgroundParams → Job ID string
• Job polling → JobStatus with timestamps and error details
• Job listing → Paginated JobStatus arrays with cursor

Message Operations:

• SendMessageParams → SendMessageResult with messageId and status
• Bulk operations → Aggregated counts (sent, failed) + per-user results
• Group operations → Member resolution + bulk message delivery

How Data Exits or Is Persisted

Persistent Storage:

• Volume.put() → Data stored in S3 (encrypted at rest)
• Database execute() → Data persisted in external database
• Excel workbook → Buffer written to volume or returned to caller
• PDF document → Buffer written to volume or returned to caller
• Message → Stored in user inbox, push notification sent

Return Values:

• Query results → Returned synchronously to caller
• File content → Streamed or buffered to caller
• Generated documents → Returned as buffers
• Job IDs → Returned immediately, execution async
• Operation results → Status objects with metadata

External API Calls:

• ketrics.http → External API responses via HttpResponse
• Logging → CloudWatch via ketrics.console
• Notifications → Push delivery platform
• Messages → Inbox storage + notification delivery

Error Handling

Error Scenarios Considered

Each feature domain handles specific error scenarios:

Volume Errors:

• VolumeNotFoundError: Volume code doesn't exist in tenant's namespace
• VolumeAccessDeniedError: Application lacks access grant
• VolumePermissionDeniedError: Specific operation (read/write/delete/list) not permitted
• FileNotFoundError: Requested file doesn't exist
• FileAlreadyExistsError: File exists when ifNotExists: true
• InvalidPathError: Path contains invalid characters or traversal attempts (../)
• FileSizeLimitError: File exceeds volume's size limit
• ContentTypeNotAllowedError: MIME type not allowed for volume

Database Errors:

• DatabaseNotFoundError: Database code doesn't exist
• DatabaseAccessDeniedError: No access grant to database
• DatabaseConnectionError: Cannot establish connection (network, credentials, server down)
• DatabaseQueryError: SQL syntax error, constraint violation, etc.
• DatabaseTransactionError: Transaction commit/rollback failed

Secret Errors:

• SecretNotFoundError: Secret code doesn't exist
• SecretAccessDeniedError: Application lacks access grant
• SecretDecryptionError: KMS decryption failed (corrupted value, key revoked)

Excel Errors:

• ExcelParseError: Provided buffer is not valid Excel format
• ExcelWriteError: File write operation failed

PDF Errors:

• PdfParseError: Provided buffer is not valid PDF format
• PdfWriteError: File write operation failed

Job Errors:

• JobNotFoundError: Job ID doesn't exist
• InvalidFunctionError: Target function name invalid or missing
• CrossAppPermissionError: Application lacks permission for cross-app job
• EltJobExecutionError: Job failed during execution (timeout, runtime error)

Message Errors:

• MessageValidationError: Subject/body too long, invalid parameters
• GroupNotFoundError: Group code doesn't exist
• TenantGrantPermissionDeniedError: Application lacks IAM-data permissions for group access

Retry Logic or Fallback Mechanisms

Built-in Retry Behaviors:

• Database connection pooling automatically handles connection reuse
• Connection timeouts trigger error rather than unlimited retries (tenant responsibility to retry)
• Failed bulk messages don't prevent other messages in batch (partial success tracked)

Tenant-Implemented Retry Patterns:

// Retry with exponential backoff
async function retryWithBackoff(fn, maxAttempts = 3) {
  for (let attempt = 1; attempt <= maxAttempts; attempt++) {
    try {
      return await fn();
    } catch (error) {
      if (attempt === maxAttempts) throw error;
      const delay = Math.pow(2, attempt) * 1000;
      await new Promise(resolve => setTimeout(resolve, delay));
    }
  }
}

// Idempotency keys prevent duplicate job execution
const jobId = await ketrics.Job.runInBackground({
  function: 'processPayment',
  payload: { orderId: '123' },
  options: { idempotencyKey: 'order-123-payment' }
});

Logging Approach

Console Logging (ketrics.console):

• Methods: log(), error(), warn(), info(), debug()
• Forwarded to CloudWatch with context:
  • Tenant ID and code
  • Application ID and code
  • Requestor information (user ID or service account ID)
  • Timestamp
  • Log level

Error Logging:

• Error classes implement toJSON() for serialization
• Error properties: name, message, code (operation), timestamp, resource code
• Security note: Error messages never expose credentials, connection strings, or internal IDs

Example Logging Pattern:

try {
  const db = await ketrics.DatabaseConnection.connect('main-db');
  ketrics.console.log('Connected to database');
  const result = await db.query('SELECT * FROM users');
  ketrics.console.log(`Retrieved ${result.rowCount} users`);
} catch (error) {
  if (error instanceof ketrics.DatabaseQueryError) {
    ketrics.console.error('Database query failed', error.toJSON());
  } else {
    ketrics.console.error('Unexpected error', { message: error.message });
  }
}

Usage

How to Run/Deploy This Component

This is a type definition library. It does not run standalone but is consumed by tenant applications.

Installation in Tenant Application:

npm install @ketrics/sdk-backend

TypeScript Configuration (tsconfig.json):

{
  "compilerOptions": {
    "strict": true,
    "lib": ["ES2020"],
    "types": ["@ketrics/sdk-backend"]
  }
}

Building the SDK Package:

# Install dependencies
npm install

# Compile TypeScript to JavaScript
npm run build

# Output: dist/ folder with .js, .d.ts, .map files

Publishing:

npm publish

This publishes to npm registry at https://www.npmjs.com/package/@ketrics/sdk-backend

Example Invocations

Complete Tenant Application Example:

// handler.ts - Tenant application code

export async function processOrder(payload: { orderId: string }) {
  try {
    // Access context
    ketrics.console.log(`Processing for tenant: ${ketrics.tenant.name}`);
    ketrics.console.log(`Requestor: ${ketrics.requestor.name} (${ketrics.requestor.type})`);

    // Get secret
    const apiKey = await ketrics.Secret.get('stripe-api-key');

    // Query database
    const db = await ketrics.DatabaseConnection.connect('orders-db');
    try {
      const result = await db.query<Order>(
        'SELECT * FROM orders WHERE id = ?',
        [payload.orderId]
      );
      if (result.rowCount === 0) {
        return { error: 'Order not found' };
      }
      const order = result.rows[0];

      // Make external API call
      const response = await ketrics.http.post<ShipmentResponse>(
        'https://shipping-api.example.com/create-shipment',
        { orderId: order.id, address: order.shippingAddress },
        { headers: { 'Authorization': `Bearer ${apiKey}` } }
      );

      // Store file in volume
      const volume = await ketrics.Volume.connect('orders');
      const result = await volume.put(
        `shipments/${order.id}/label.pdf`,
        response.data.labelPdf,
        { contentType: 'application/pdf' }
      );

      // Update database in transaction
      await db.transaction(async (tx) => {
        await tx.execute(
          'UPDATE orders SET status = ?, shipment_id = ? WHERE id = ?',
          ['shipped', response.data.shipmentId, order.id]
        );
      });

      // Send message to user
      await ketrics.Messages.send({
        userId: order.userId,
        type: 'ORDER_SHIPPED',
        subject: 'Your order has shipped!',
        body: `Track your package at: ${response.data.trackingUrl}`,
        priority: 'HIGH',
        actionUrl: response.data.trackingUrl
      });

      // Queue async job
      const jobId = await ketrics.Job.runInBackground({
        function: 'sendShipmentNotifications',
        payload: { orderId: order.id },
        options: { timeout: 60000 }
      });

      return {
        success: true,
        shipmentId: response.data.shipmentId,
        jobId: jobId
      };
    } finally {
      await db.close();
    }
  } catch (error) {
    ketrics.console.error('Order processing failed', { error: error.message });
    throw error;
  }
}

export async function generateMonthlyReport() {
  try {
    // Create Excel workbook
    const workbook = ketrics.Excel.create();
    const sheet = workbook.addWorksheet('Sales Report');

    // Query all transactions
    const db = await ketrics.DatabaseConnection.connect('analytics-db');
    try {
      const result = await db.query<Transaction>(
        'SELECT * FROM transactions WHERE month = MONTH(NOW())'
      );

      // Populate worksheet
      sheet.addRow(['Date', 'Amount', 'Category']);
      for (const tx of result.rows) {
        sheet.addRow([new Date(tx.date), tx.amount, tx.category]);
      }

      // Write to volume
      const buffer = await workbook.writeFile();
      const volume = await ketrics.Volume.connect('reports');
      await volume.put(`monthly/${new Date().toISOString().split('T')[0]}.xlsx`, buffer);

      // Send to group (requires IAM-data permissions)
      await ketrics.Messages.sendToGroup({
        groupCode: 'finance-team',
        type: 'REPORT_READY',
        subject: 'Monthly Sales Report',
        body: 'Your monthly sales report is ready',
        priority: 'MEDIUM'
      });

      return { success: true };
    } finally {
      await db.close();
    }
  } catch (error) {
    if (error instanceof ketrics.GroupNotFoundError) {
      ketrics.console.error(`Group not found: ${error.message}`);
    } else if (error instanceof ketrics.ExcelWriteError) {
      ketrics.console.error('Failed to write Excel file');
    } else {
      throw error;
    }
  }
}

Error Handling Patterns:

// Type guards for error handling
try {
  const db = await ketrics.DatabaseConnection.connect('main-db');
} catch (error) {
  if (ketrics.isDatabaseError(error)) {
    // Specific database error
    console.log(`Database error: ${error.message}`);
  }

  if (ketrics.isDatabaseErrorType(error, 'DatabaseNotFoundError')) {
    // Exact error type check
    console.log(`Database not found: ${error.databaseCode}`);
  }

  if (error instanceof ketrics.DatabaseAccessDeniedError) {
    // Direct instanceof check
    console.log('Application lacks database access grant');
  }
}

Testing Approach

Unit Testing Tenant Code:

# Tenant applications test their handlers without running in Ketrics platform
npm install --save-dev jest @types/jest ts-jest

# Mock the global ketrics object for tests
jest.mock('@ketrics/sdk-backend', () => ({
  // Provide mock implementations
}));

Integration Testing (in Ketrics platform):

• Deploy application to staging environment
• Ketrics platform verifies type compatibility at build time
• Runtime validates against actual service permissions
• Test tenant isolation and error scenarios

Type Safety:

• TypeScript compilation enforces correct API usage at build time
• No runtime type checking (SDK is pure types)
• Type definitions include JSDoc comments with usage examples

Architecture Patterns

API Design Patterns

Static Factory Pattern:

• Volume.connect(), DatabaseConnection.connect(), Secret.get()
• No constructor calls, explicit factory methods
• Enables lazy connection pooling and resource management

Error Type Guards:

• isVolumeError(error) - Checks if error is any VolumeError
• isVolumeErrorType(error, 'VolumeNotFoundError') - Checks specific type
• Enables type-safe error narrowing without instanceof

Transaction Scope:

• db.transaction(async (tx) => { ... })
• Callback receives transaction connection
• Automatic commit on success, rollback on error

Pagination with Cursors:

• List operations return cursor for next page
• Enables efficient large result set handling
• Stateless pagination without offset/limit complexity

File Export Strategy

The index.ts file re-exports all types from feature modules:

1. Interfaces and types for tenant usage
2. Error classes for instanceof checking
3. Type guards for runtime error discrimination
4. Global KetricsSdkV1 interface defining the complete ketrics object

This central export point ensures:

• Single source of truth for SDK surface
• Complete type definitions for IDE autocomplete
• Proper TypeScript declaration file generation