@tetrascience-npm/ts-connectors-sdk

TetraScience Node.js Connectors SDK.

Provides base classes and utilities for building TetraScience Connectors in Node.js.

It is primarily intended to be used by TetraScience connectors running on the Tetra Data Platform (TDP).

Installation

This SDK relies on a small set of peer dependencies that your application must provide. These are not bundled with the SDK so that you can control the exact versions used in your environment.

Required peer dependencies:

• @aws-sdk/client-cloudwatch-logs
• @aws-sdk/client-s3
• @aws-sdk/client-sqs
• @aws-sdk/client-ssm
• @aws-sdk/lib-storage
• axios

If you are not already using these packages, you can install everything together:

Using npm:

npm install @tetrascience-npm/ts-connectors-sdk \
  @aws-sdk/client-cloudwatch-logs @aws-sdk/client-s3 \
  @aws-sdk/client-sqs @aws-sdk/client-ssm @aws-sdk/lib-storage axios

Using Yarn:

yarn add @tetrascience-npm/ts-connectors-sdk \
  @aws-sdk/client-cloudwatch-logs @aws-sdk/client-s3 \
  @aws-sdk/client-sqs @aws-sdk/client-ssm @aws-sdk/lib-storage axios

If your project already depends on the AWS SDK v3 clients or axios, you only need to add @tetrascience-npm/ts-connectors-sdk and ensure that your existing versions satisfy the peer dependency ranges in package.json.

This SDK requires Node.js 18, 20, or 22 (see the engines field in package.json). Node.js 16 support was dropped in v4.0.0.

Overview

The SDK exposes the following primary building blocks:

• TDPClient – wraps the Tetra Data Platform data-acquisition API and handles datalake uploads directly to S3, including metadata options and validation.
• Connector – base class that encapsulates common connector lifecycle concerns such as initialization, health reporting, metrics, and command handling.
• PollingConnector – extends Connector with a polling loop implementation suitable for connectors that primarily poll external systems on an interval.

All of these are exported from the package root, for example:

import { Connector, PollingConnector, TDPClient } from '@tetrascience-npm/ts-connectors-sdk';

Basic usage

TDPClient

In a typical connector environment, TDPClient is initialized with configuration derived from the connector runtime (environment variables, manifests, and connector settings). At a high level you use it to make API calls and upload files into the Tetra Data Platform:

import { TDPClient } from '@tetrascience-npm/ts-connectors-sdk';

async function uploadExample() {
  const client = new TDPClient(/* see source for available options */);

  await client.uploadFile({
    content: Buffer.from('example'),
    filepath: 'path/in/datalake/example.txt',
    // Optional metadata, tags, labels, and other fields are validated
    // against the platform metadata rules.
  });
}

Refer to the TDPClient source (src/tdp-client.ts) for the full set of options and behaviors.

createAwsClient()

After calling init(), you can create additional AWS SDK v3 clients that share the same proxy configuration and credentials as the SDK's built-in clients:

import { S3Client } from '@aws-sdk/client-s3';

const s3 = client.createAwsClient(S3Client);
// Optional: pass overrides for any AWS SDK client config
const s3WithOverrides = client.createAwsClient(S3Client, { region: 'us-west-2' });

All clients created this way use a generic request handler with a 120-second timeout, which is suitable for most AWS services.

For S3 specifically, the SDK already manages an internal S3Client tuned with socket idle timeouts instead of a fixed request timeout — a better fit for large or slow file transfers. If you need S3 access with that tuning, prefer either of these over createAwsClient(S3Client):

• uploadFile() — the primary way to upload files to the TDP datalake. Handles credentials, metadata, checksums, and multipart uploads automatically.

• assertAwsInitialized().s3Client — returns the SDK-managed S3Client directly, for cases where you need to make S3 calls beyond what uploadFile() covers:

  const { s3Client } = await client.assertAwsInitialized();

If you have a specific reason to create your own S3Client via createAwsClient and still want the S3-tuned handler, you can pass it as an override:

const s3 = client.createAwsClient(S3Client, {
  requestHandler: client.createProxyAWSNodeHttpHandlers().s3Handler,
});

Note that this creates new handler instances rather than sharing the SDK's internal ones, so the simpler assertAwsInitialized().s3Client is preferred when possible.

tdpDeploymentCertificates

A read-only getter that returns the TDP deployment certificates loaded during init(). These are automatically included in the CA bundle for axios-based HTTP requests (e.g. TDP API calls). You can inspect the loaded certificates:

await client.init();
console.log(`Loaded ${client.tdpDeploymentCertificates.length} TDP deployment certificate(s)`);

IAM_PROXY environment variable

Set IAM_PROXY to an HTTP proxy URL to route all AWS SDK traffic (S3, SQS, SSM, and other AWS clients created via createAwsClient()) through a specific proxy. This is useful in network-restricted deployments where AWS endpoints are only reachable through an IAM-aware proxy:

IAM_PROXY=http://iam-proxy.example.com:8080

When IAM_PROXY is not set, the SDK falls back to the standard http_proxy/https_proxy environment variables for per-request proxy selection.

Connector and PollingConnector

To build a connector, extend either Connector or PollingConnector and implement the appropriate hooks for your integration.

import { PollingConnector } from '@tetrascience-npm/ts-connectors-sdk';

class MyPollingConnector extends PollingConnector {
  // Implement required hooks such as poll(), onStartup(), onShutdown(), etc.
}

The base classes handle boilerplate such as:

• Connector lifecycle and shutdown behavior
• Health and metrics reporting
• Command reception and dispatch

Versioning and changelog

All notable changes are documented in CHANGELOG.md.

Contributing

For more details, see CONTRIBUTING.md.

License

This project is licensed under the Apache License 2.0.