Pulumi Logtail Provider

A Pulumi provider for managing Logtail log management and analytics resources, dynamically bridged from the Terraform Logtail Provider.

Introduction

This package provides a Pulumi provider that enables you to manage your Logtail logging infrastructure using TypeScript, JavaScript, Python, Go, or C#. The provider is automatically generated from the Terraform Logtail provider, giving you access to all its functionality within the Pulumi ecosystem.

Features

• Log Sources: Create and manage log sources for collecting logs from various platforms and services
• Source Groups: Organize log sources into logical groups for better management
• Metrics: Define and configure custom metrics based on log data
• Log Aggregation: Centralized log collection and processing
• Log Analytics: Query and analyze log data with powerful search capabilities
• TypeScript Support: Full type safety with comprehensive TypeScript definitions

Installation

npm

npm install pulumi-logtail

yarn

yarn add pulumi-logtail

pnpm

pnpm add pulumi-logtail

Configuration

Before using the Logtail provider, you need to configure your API token. You can obtain an API token from your Logtail dashboard.

Environment Variables

export LOGTAIL_API_TOKEN="your-api-token-here"

Pulumi Configuration

pulumi config set logtail:apiToken "your-api-token-here" --secret

Usage

Log Source Management

import * as logtail from 'pulumi-logtail'

// Create a log source for application logs
const appSource = new logtail.Source('app-logs', {
  name: 'Application Logs',
  platform: 'javascript',
  retentionDays: 30,
})

// Create a source for system logs
const systemSource = new logtail.Source('system-logs', {
  name: 'System Logs',
  platform: 'syslog',
  retentionDays: 90,
})

// Create a source for Docker container logs
const dockerSource = new logtail.Source('docker-logs', {
  name: 'Docker Container Logs',
  platform: 'docker',
  retentionDays: 14,
})

Source Group Organization

import * as logtail from 'pulumi-logtail'

// Create a source group for production environment
const productionGroup = new logtail.SourceGroup('production', {
  name: 'Production Environment',
  description: 'All log sources from production infrastructure',
})

// Create a source group for development environment
const developmentGroup = new logtail.SourceGroup('development', {
  name: 'Development Environment',
  description: 'Log sources for development and testing',
})

// Create sources within groups
const prodAppSource = new logtail.Source('prod-app', {
  name: 'Production Application',
  platform: 'javascript',
  retentionDays: 90,
  sourceGroupId: productionGroup.id,
})

const devAppSource = new logtail.Source('dev-app', {
  name: 'Development Application',
  platform: 'javascript',
  retentionDays: 7,
  sourceGroupId: developmentGroup.id,
})

Custom Metrics Creation

import * as logtail from 'pulumi-logtail'

// Create a metric to count error occurrences
const errorMetric = new logtail.Metric('error-count', {
  name: 'Error Count',
  description: 'Count of application errors',
  sourceId: appSource.id,
  query: 'level:error',
  aggregation: 'count',
  tags: ['environment:production', 'team:backend'],
})

// Create a metric for response time monitoring
const responseTimeMetric = new logtail.Metric('response-time', {
  name: 'Average Response Time',
  description: 'Average HTTP response time',
  sourceId: appSource.id,
  query: 'http.response_time > 0',
  aggregation: 'avg',
  field: 'http.response_time',
  tags: ['environment:production', 'team:backend'],
})

// Create a metric for tracking specific events
const userSignupMetric = new logtail.Metric('user-signups', {
  name: 'User Signups',
  description: 'Count of user registration events',
  sourceId: appSource.id,
  query: 'event:user_signup',
  aggregation: 'count',
  tags: ['environment:production', 'team:growth'],
})

Advanced Log Source Configurations

import * as logtail from 'pulumi-logtail'

// Create a source for AWS CloudWatch logs
const cloudwatchSource = new logtail.Source('cloudwatch-logs', {
  name: 'AWS CloudWatch Logs',
  platform: 'cloudwatch',
  retentionDays: 60,
  configuration: {
    region: 'us-east-1',
    logGroups: ['/aws/lambda/my-function', '/aws/apigateway/my-api'],
  },
})

// Create a source for Kubernetes logs
const k8sSource = new logtail.Source('kubernetes-logs', {
  name: 'Kubernetes Cluster Logs',
  platform: 'kubernetes',
  retentionDays: 30,
  configuration: {
    namespace: 'production',
    labelSelectors: ['app=web', 'tier=frontend'],
  },
})

// Create a source for database logs
const dbSource = new logtail.Source('database-logs', {
  name: 'PostgreSQL Logs',
  platform: 'postgresql',
  retentionDays: 45,
  configuration: {
    logLevel: 'info',
    includeStatements: true,
  },
})

Log Analytics and Querying

import * as logtail from 'pulumi-logtail'

// Create metrics for monitoring application health
const healthMetrics = [
  new logtail.Metric('4xx-errors', {
    name: '4XX HTTP Errors',
    description: 'Count of 4XX HTTP status codes',
    sourceId: appSource.id,
    query: 'http.status_code >= 400 AND http.status_code < 500',
    aggregation: 'count',
    alertThreshold: 100,
  }),

  new logtail.Metric('5xx-errors', {
    name: '5XX HTTP Errors',
    description: 'Count of 5XX HTTP status codes',
    sourceId: appSource.id,
    query: 'http.status_code >= 500',
    aggregation: 'count',
    alertThreshold: 10,
  }),

  new logtail.Metric('slow-queries', {
    name: 'Slow Database Queries',
    description: 'Count of slow database queries',
    sourceId: dbSource.id,
    query: 'duration > 1000',
    aggregation: 'count',
    alertThreshold: 50,
  }),
]

Resources

Core Resources

• Source: Log sources for collecting logs from various platforms and services
• SourceGroup: Logical groups for organizing related log sources
• Metric: Custom metrics based on log data for monitoring and alerting

Data Sources

• getSource: Retrieve information about existing log sources
• getSourceGroup: Retrieve information about existing source groups
• getMetric: Retrieve information about existing metrics

API Reference

For detailed API documentation, type definitions, and examples, please refer to the API Reference documentation.

Authentication Setup

Obtaining an API Token

1. Log in to your Logtail account
2. Navigate to Settings → API Tokens
3. Click Create API Token
4. Provide a name for your token and select appropriate permissions
5. Copy the generated token (it will only be shown once)

Configuring the Provider

Set your API token using one of these methods:

Environment Variable (Recommended)

export LOGTAIL_API_TOKEN="logtail-api-token"

Pulumi Configuration

pulumi config set logtail:apiToken "logtail-api-token" --secret

Provider Configuration in Code

import * as logtail from 'pulumi-logtail'

const provider = new logtail.Provider('logtail', {
  apiToken: 'logtail-api-token',
})

// Use the provider with resources
const source = new logtail.Source(
  'my-source',
  {
    // ... configuration
  },
  { provider },
)

Platform Support

Logtail supports log collection from various platforms and services:

Application Platforms

• JavaScript/Node.js: Browser and server-side JavaScript applications
• Python: Python applications and scripts
• Ruby: Ruby applications and Rails
• Go: Go applications and services
• Java: Java applications and Spring Boot
• PHP: PHP applications and Laravel
• .NET: .NET Core and Framework applications

Infrastructure Platforms

• Docker: Container logs from Docker environments
• Kubernetes: Pod and container logs from Kubernetes clusters
• AWS CloudWatch: Logs from AWS services and Lambda functions
• Syslog: System logs from servers and network devices
• PostgreSQL: Database logs and query logs
• MySQL: Database logs and slow query logs
• Redis: Redis server logs
• Nginx: Web server access and error logs
• Apache: Apache web server logs

Cloud Services

• AWS: CloudWatch Logs, ELB, CloudFront, and other AWS services
• Google Cloud: Cloud Logging and other GCP services
• Azure: Azure Monitor and other Azure services
• Heroku: Application and system logs from Heroku dynos

Examples

You can find more examples in common use cases:

• Basic Log Collection: Setting up log sources for web applications
• Multi-Environment Setup: Organizing logs across development, staging, and production
• Metrics and Alerting: Creating custom metrics for monitoring application health
• Log Aggregation: Centralized logging for microservices architecture
• Platform Integration: Connecting various platforms and services to Logtail

Support

This provider is a derived work of the Terraform Provider distributed under MPL 2.0.

If you encounter a bug or missing feature, please consult the source terraform-provider-logtail repo.

For Pulumi-specific issues, please open an issue in the pulumi-any-terraform repository.

License

This package is distributed under the MIT License. The underlying Terraform provider is distributed under MPL 2.0.