@bernierllc/data-connector-base

Base interfaces and abstract classes for all data connectors in the BernierLLC data ingestion system.

Overview

This core package provides the foundational types and base class for implementing data connectors. It defines the common interface that all connector implementations must follow, ensuring consistency across different data source types (API, DOWNLOAD, FTP, SCRAPE).

Installation

npm install @bernierllc/data-connector-base

Usage

Extending BaseDataConnector

import { BaseDataConnector, ConnectorAccessMethod, FetchResult, FetchOptions } from '@bernierllc/data-connector-base';

class MyApiConnector extends BaseDataConnector {
  constructor(config: ConnectorConfig) {
    super({
      ...config,
      accessMethod: ConnectorAccessMethod.API
    });
  }

  async connect(): Promise<ConnectionResult> {
    // Implement connection logic
    return this.createConnectionResult(true);
  }

  async disconnect(): Promise<void> {
    // Implement disconnection logic
    this.connected = false;
  }

  async fetch(options?: FetchOptions): Promise<FetchResult> {
    // Implement data fetching logic
    return this.createFetchResult(true, data);
  }

  async validateConfig(): Promise<ValidationResult> {
    // Implement configuration validation
    return this.createValidationResult(true);
  }
}

Using Connector Types

import { DataConnector, ConnectorConfig, FetchResult } from '@bernierllc/data-connector-base';

async function useConnector(connector: DataConnector) {
  const status = await connector.getStatus();
  if (!status.connected) {
    await connector.connect();
  }
  
  const result = await connector.fetch();
  if (result.success) {
    console.log('Data:', result.data);
  }
}

API Reference

Types

• ConnectorAccessMethod - Enum of access method types (API, DOWNLOAD, FTP, SCRAPE)
• ConnectorConfig - Base configuration interface
• ConnectionResult - Result of connection attempt
• FetchResult<T> - Result of data fetch operation
• ConnectorStatus - Current connector status
• ConnectorError - Standardized error format
• ValidationResult - Configuration validation result

Classes

• BaseDataConnector - Abstract base class for all connectors

License

Bernier LLC - Limited Use License