AWS (Firehose) Destination for walkerOS

Source Code • NPM Package

walkerOS follows a source → collector → destination architecture. This AWS destination receives processed events from the walkerOS collector and streams them to AWS services like Firehose, enabling real-time data ingestion into AWS data lakes, warehouses, and analytics services for large-scale event processing and analysis.

Installation

npm install @walkeros/server-destination-aws

Quick Start

Configure in your Flow JSON:

{
  "version": 1,
  "flows": {
    "default": {
      "server": {},
      "destinations": {
        "firehose": {
          "package": "@walkeros/server-destination-aws",
          "config": {
            "settings": {
              "firehose": {
                "streamName": "your-firehose-stream-name",
                "region": "eu-central-1"
              }
            }
          }
        }
      }
    }
  }
}

Or programmatically:

import { startFlow } from '@walkeros/collector';
import { destinationFirehose } from '@walkeros/server-destination-aws';

const { elb } = await startFlow({
  destinations: [
    {
      destination: destinationFirehose,
      config: {
        settings: {
          firehose: {
            streamName: 'your-firehose-stream-name',
            region: 'eu-central-1',
            config: {
              credentials: {
                accessKeyId: 'your-access-key-id',
                secretAccessKey: 'your-secret-access-key',
              },
            },
          },
        },
      },
    },
  ],
});

Configuration

| Name | Type | Description | Required | Example | | ---------- | ---------------- | ----------------------------------- | -------- | ------------------------------------------------------ | | firehose | FirehoseConfig | AWS Firehose configuration settings | No | { streamName: 'walker-events', region: 'us-east-1' } |

Firehose Configuration

The firehose object has the following properties:

| Name | Type | Description | Required | Example | | ------------ | ---------------------- | ------------------------------------------------- | -------- | --------------------------------- | | streamName | string | Name of the Kinesis Data Firehose delivery stream | Yes | 'walker-events' | | client | FirehoseClient | Pre-configured AWS Firehose client instance | No | new FirehoseClient(config) | | region | string | AWS region for the Firehose service | No | 'us-east-1' | | config | FirehoseClientConfig | AWS SDK client configuration options | No | { credentials: awsCredentials } |

Type Definitions

See src/types/ for TypeScript interfaces.

Related

• Website Documentation
• Destination Interface

Contribute

Feel free to contribute by submitting an issue, starting a discussion, or getting in contact.

License

This project is licensed under the MIT License.