@bernierllc/contentful-suite

Complete Contentful integration suite with ML training pipeline, headless CMS proxy, and content syndication.

Features

• ML Training Pipeline: Full content export + incremental sync to local database
• Headless CMS Proxy: API gateway with OAuth management and response caching
• Content Syndication: Real-time webhook processing and event routing
• Multi-Space Management: Centralized credential management for multiple spaces
• Migration Support: Content model migrations with history tracking
• NeverHub Integration: Service mesh integration for distributed systems

Installation

npm install @bernierllc/contentful-suite

Usage

Quick Start

Basic Setup

import {
  ContentfulSuite,
  InMemoryContentStorage,
  InMemorySyncStateStorage,
  InMemoryMigrationStorage
} from '@bernierllc/contentful-suite';

const suite = new ContentfulSuite({
  oauth: {
    clientId: process.env.CONTENTFUL_CLIENT_ID!,
    clientSecret: process.env.CONTENTFUL_CLIENT_SECRET!,
    redirectUri: 'http://localhost:3000/oauth/callback'
  },
  gateway: {
    enabled: true,
    port: 3000,
    apiKeys: ['your-internal-api-key']
  },
  sync: {
    enabled: true,
    interval: 300000, // 5 minutes
    spaces: [
      {
        spaceId: 'your-space-id',
        environmentId: 'master',
        accessToken: 'your-cda-token'
      }
    ]
  },
  webhooks: {
    enabled: true,
    port: 3001,
    secret: 'your-webhook-secret'
  },
  cache: {
    enabled: true,
    backend: 'memory',
    ttl: 300
  },
  migration: {
    enabled: true
  },
  storage: {
    contentStorage: new InMemoryContentStorage(),
    syncStateStorage: new InMemorySyncStateStorage(),
    migrationStorage: new InMemoryMigrationStorage()
  }
});

await suite.initialize();
await suite.start();

Use Cases

1. ML Training Pipeline

Export all content from Contentful to a local database for machine learning model training:

import { ContentfulSuite } from '@bernierllc/contentful-suite';

// Initialize suite with sync enabled
const suite = new ContentfulSuite({
  sync: {
    enabled: true,
    spaces: [
      {
        spaceId: 'your-space-id',
        environmentId: 'master',
        accessToken: 'your-cda-token'
      }
    ]
  },
  storage: {
    contentStorage: new InMemoryContentStorage(),
    syncStateStorage: new InMemorySyncStateStorage()
  },
  // ... other config
});

await suite.initialize();

// Get all content for training
const trainingData = await suite.getMLTrainingData('your-space-id');

// Filter by content type
const articles = await suite.getMLTrainingData('your-space-id', 'article');

// Train your ML model with the data
trainModel(trainingData);

Benefits:

• No API rate limits during training (data is local)
• Automatic incremental updates via webhooks + Sync API
• Full content history and relationships preserved

2. Headless CMS Proxy

Create an API gateway that manages OAuth tokens and caches responses:

import { ContentfulSuite } from '@bernierllc/contentful-suite';

const suite = new ContentfulSuite({
  oauth: {
    clientId: process.env.CONTENTFUL_CLIENT_ID!,
    clientSecret: process.env.CONTENTFUL_CLIENT_SECRET!,
    redirectUri: 'http://localhost:3000/oauth/callback'
  },
  gateway: {
    enabled: true,
    port: 3000,
    apiKeys: ['internal-api-key-1', 'internal-api-key-2']
  },
  cache: {
    enabled: true,
    backend: 'redis',
    ttl: 300
  },
  webhooks: {
    enabled: true,
    port: 3001,
    secret: 'webhook-secret'
  },
  // ... storage config
});

await suite.initialize();
await suite.start();

// Gateway is now running at http://localhost:3000
// Use internal API key to make requests:
// GET http://localhost:3000/cda/space-id/master/entries
// Header: X-API-Key: internal-api-key-1

Benefits:

• Centralized OAuth token management (no token refresh in clients)
• Response caching for better performance
• Webhook-based cache invalidation
• Rate limiting per client
• Multi-space routing

3. Content Syndication

Process webhooks in real-time and route content changes to subscribers:

import { ContentfulSuite } from '@bernierllc/contentful-suite';

const suite = new ContentfulSuite({
  webhooks: {
    enabled: true,
    port: 3001,
    secret: 'your-webhook-secret'
  },
  sync: {
    enabled: true,
    spaces: [{ spaceId: 'your-space-id', environmentId: 'master', accessToken: 'token' }]
  },
  // ... storage config
});

await suite.initialize();

// Subscribe to content events
const webhookService = suite.getWebhookService();
webhookService?.on('webhook.received', async (data) => {
  console.log('Content changed:', data);
  // Syndicate to other platforms
  await publishToTwitter(data);
  await publishToLinkedIn(data);
});

await suite.start();

// Configure Contentful webhook to POST to:
// http://your-domain:3001/webhooks/contentful

Benefits:

• Real-time content propagation
• Reliable delivery with retry logic
• Dead letter queue for failed events
• Event filtering and routing

4. Multi-Space Management

Manage multiple Contentful spaces from a single suite instance:

import { ContentfulSuite } from '@bernierllc/contentful-suite';

const suite = new ContentfulSuite({
  sync: {
    enabled: true,
    spaces: [
      {
        spaceId: 'space-1',
        environmentId: 'master',
        accessToken: 'token-1'
      },
      {
        spaceId: 'space-2',
        environmentId: 'staging',
        accessToken: 'token-2'
      }
    ]
  },
  migration: {
    enabled: true
  },
  // ... storage config
});

await suite.initialize();

// Run migration on specific space
await suite.runMigration('space-1', migrationScript, {
  id: 'migration-1',
  name: 'Add author field'
});

// Get sync service for specific space
const syncService = suite.getSyncService('space-1');
await syncService?.initialSync();

Storage Implementations

The suite uses abstract storage interfaces, allowing you to provide your own implementations:

In-Memory Storage (Included)

For testing and simple use cases:

import {
  InMemoryContentStorage,
  InMemorySyncStateStorage,
  InMemoryMigrationStorage
} from '@bernierllc/contentful-suite';

const config = {
  storage: {
    contentStorage: new InMemoryContentStorage(),
    syncStateStorage: new InMemorySyncStateStorage(),
    migrationStorage: new InMemoryMigrationStorage()
  }
};

Custom Storage Implementation

Implement the storage interfaces for your database:

import { ContentStorage, SyncStateStorage, MigrationStorage } from '@bernierllc/contentful-suite';

class PostgresContentStorage implements ContentStorage {
  async storeEntries(entries: ContentfulEntry[]): Promise<void> {
    // Store entries in PostgreSQL
  }

  async storeAssets(assets: ContentfulAsset[]): Promise<void> {
    // Store assets in PostgreSQL
  }

  // ... implement other methods
}

const suite = new ContentfulSuite({
  storage: {
    contentStorage: new PostgresContentStorage(),
    syncStateStorage: new PostgresSyncStateStorage(),
    migrationStorage: new PostgresMigrationStorage()
  }
});

Configuration

OAuth Configuration

Required for gateway service:

{
  oauth: {
    clientId: 'your-client-id',
    clientSecret: 'your-client-secret',
    redirectUri: 'http://localhost:3000/oauth/callback'
  }
}

Gateway Configuration

API gateway with internal authentication:

{
  gateway: {
    enabled: true,
    port: 3000,
    apiKeys: ['key1', 'key2'] // Internal API keys for authentication
  }
}

Sync Configuration

Content sync with local storage:

{
  sync: {
    enabled: true,
    interval: 300000, // Polling interval in ms (default: 5 minutes)
    spaces: [
      {
        spaceId: 'space-id',
        environmentId: 'master',
        accessToken: 'cda-token'
      }
    ]
  }
}

Webhooks Configuration

Webhook receiving and processing:

{
  webhooks: {
    enabled: true,
    port: 3001,
    secret: 'webhook-secret' // For signature validation
  }
}

Cache Configuration

Response caching:

{
  cache: {
    enabled: true,
    backend: 'memory' | 'redis',
    ttl: 300 // Time to live in seconds
  }
}

Migration Configuration

Content model migrations:

{
  migration: {
    enabled: true
  }
}

API Reference

ContentfulSuite

Main orchestrator class.

Methods

initialize(): Promise<void>

Initialize all enabled services.

start(): Promise<void>

Start all services (gateway, webhooks, sync polling).

stop(): Promise<void>

Stop all services gracefully.

getMLTrainingData(spaceId: string, contentType?: string): Promise<any[]>

Get content from local storage for ML training.

runMigration(spaceId: string, script: string, options: MigrationOptions): Promise<void>

Run a migration on a specific space.

getHealth(): SuiteHealth

Get health status of all services.

Service Getters

• getGateway(): ContentfulGatewayService | undefined
• getSyncService(spaceId: string): ContentfulSyncService | undefined
• getWebhookService(): ContentfulWebhookService | undefined
• getCacheService(): ContentfulCacheService | undefined
• getMigrationService(spaceId: string): ContentfulMigrationService | undefined

Integration Status

This package integrates with the following BernierLLC services:

• Logger: Integrated - Uses @bernierllc/logger for structured logging across all services
• NeverHub: Optional - Automatically detects and integrates with NeverHub service mesh when available

Logger Integration

The suite uses @bernierllc/logger for consistent structured logging across all services:

• Gateway service logs API requests and OAuth events
• Sync service logs content synchronization operations
• Webhook service logs incoming webhook events
• Migration service logs schema changes

NeverHub Integration

The suite automatically integrates with NeverHub if available:

// NeverHub events published:
- contentful.suite.started
- contentful.suite.stopped

// NeverHub events subscribed:
- system.shutdown (graceful shutdown)
- config.updated (reload configuration)

Dependencies

This suite bundles all Contentful packages:

Core Packages

• @bernierllc/contentful-types
• @bernierllc/contentful-client-core
• @bernierllc/contentful-auth
• @bernierllc/contentful-cma-client
• @bernierllc/contentful-cda-client
• @bernierllc/contentful-graphql-client
• @bernierllc/contentful-webhook-handler
• @bernierllc/contentful-export-adapter
• @bernierllc/contentful-rich-text

Service Packages

• @bernierllc/contentful-gateway-service
• @bernierllc/contentful-sync-service
• @bernierllc/contentful-webhook-service
• @bernierllc/contentful-cache-service
• @bernierllc/contentful-migration-service

Infrastructure

• @bernierllc/config-manager
• @bernierllc/logger
• @bernierllc/neverhub-adapter

License

Copyright (c) 2025 Bernier LLC. See LICENSE file for details.