HyperConn

Multi-tenant MongoDB connection manager with AWS Secrets Manager integration

HyperConn is a powerful NestJS package that simplifies multi-tenant database management by providing seamless connections to multiple MongoDB databases for different organizations. Each organization operates within its own isolated schema while leveraging AWS Secrets Manager for secure credential management.

🚀 Features

• Multi-tenant Architecture: Connect to multiple databases with organization-specific schemas
• AWS Secrets Manager Integration: Secure credential management with automatic rotation support
• Schema Isolation: Each organization operates in its own database schema namespace
• Connection Pooling: Optimized database connections with configurable pool settings
• Type Safety: Full TypeScript support with comprehensive interfaces
• NestJS Compatible: Built specifically for NestJS applications
• Flexible Configuration: Support for multiple services and collections per organization
• Production Ready: Built with enterprise-scale applications in mind

📦 Installation

# Using npm
npm install hyperconn

# Using yarn
yarn add hyperconn

# Using pnpm
pnpm add hyperconn

🔧 Prerequisites

1. AWS Secrets Manager Setup

Ensure AWS Secrets Manager is configured with database credentials for each organization. Format for secret name must be: -

Service name will be the name of the microservice. Each secret should contain below JSON in stringified format:

{
  "<microserviceName>-mongodb": "db_url",
}

Example - Name can be - martech-orgId1 Stringified JSON can be - '{"corMicroservice-mongodb": "db_url"}'

2. Environment Variables

Create a .env file in the project root directory with your AWS credentials:

AWS_SECRETS_MANAGER_ACCESS_KEY_ID=your_access_key_id
AWS_SECRETS_MANAGER_SECRET_ACCESS_KEY=your_secret_access_key
AWS_SECRETS_MANAGER_REGION=your_aws_region

🚦 Quick Start

1. Make a file with name "connectHypercon" and Import Required Components

import { MongoDbManagerService, requiredParamsInterface } from 'hyperconn';
import eventsSchema from "./yourpath/eventsSchema"
import userSchema from "./yourpath/userSchema"
import profileSchema from "./yourpath/profileSchema"

2. Initialize and Configure

export class DatabaseService {
  private dbManager: MongoDbManagerService;

  constructor() {
    this.dbManager = new MongoDbManagerService();
  }

  async initializeDatabases() {
    const params: requiredParamsInterface = {
      secretManager: {
        secretsManagerAccessKeyId: process.env.AWS_SECRETS_MANAGER_ACCESS_KEY_ID,
        secretsManagerSecretAccessKey: process.env.AWS_SECRETS_MANAGER_SECRET_ACCESS_KEY,
        secretsManagerRegion: process.env.AWS_SECRETS_MANAGER_REGION
      },
      mongoDbServiceManager: [
        {
          serviceName: "service1",
          collections: [
            {
              name: "events",
              schema: eventsSchema
            },
            {
              name: "users",
              schema: userSchema
            }
          ]
        },
        {
          serviceName: "service2",
          collections: [
            {
              name: "events",
              schema: eventsSchema
            },
            {
              name: "profiles",
              schema: profileSchema
            }
          ]
        }
      ],
      projectName: "martech",
      mongoConnectionOptions: {
        maxPoolSize: 10,
        connectTimeoutMS: 10000,
        socketTimeoutMS: 45000,
        retryWrites: true,
        readPreference: 'primary'
      }
    };

    await this.dbManager.connectToDatabasesAndModels(params);
  }

  // Get model for specific organization
  getEventModel(orgId: string, serviceName, modelName) {
    return this.dbManager.getModel(serviceName, orgId, modelName);
  }

  // Get all organization IDs
  getAllOrganizations() {
    return await this.dbManager.getOrgIds();
  }
}

3. In main.ts, initialize connectHypercon.ts

import { NestFactory } from '@nestjs/core';
import { AppModule } from './app.module';
import { connectMongoWithHypercon } from './database/connectHyperconn';

async function bootstrap() {
  // Initialize HyperConn before creating the app
  await connectMongoWithHypercon.connect();
  
  const app = await NestFactory.create(AppModule);
  await app.listen(3000);
  
  console.log('🚀 Application is running on: http://localhost:3000');
}

bootstrap();

4. Use in Your NestJS Service

@Injectable()
export class EventsService {
  constructor(private readonly databaseService: DatabaseService) {}

  async createEvent(orgId: string, eventData: any) {
    const EventModel = this.databaseService.getEventModel(orgId, 'service1', 'events');
    const event = new EventModel(eventData);
    return await event.save();
  }

  async getEventsByOrganization(orgId: string, filters: any = {}) {
    const EventModel = this.databaseService.getEventModel(orgId, 'service1', 'users');
    return await EventModel.find(filters).exec();
  }

  async getOrganizationStats() {
    const organizations = await this.databaseService.getAllOrganizations();
    const stats = [];

    for (const orgId of organizations) {
      const EventModel = this.databaseService.getEventModel(orgId, 'service1', 'events');
      const count = await EventModel.countDocuments();
      stats.push({ orgId, eventCount: count });
    }

    return stats;
  }
}

📚 API Reference

MongoDbManagerService

The main service class for managing multi-tenant database connections.

Methods

connectToDatabasesAndModels(params: requiredParamsInterface): Promise<void>

Establishes connections to all configured databases and initializes models.

Parameters:

• params - Configuration object containing all required parameters

getEventModel(orgId: string, serviceName: string, modelName: string): Model<any>

Retrieves a Mongoose model for a specific organization and schema.

Parameters:

• serviceName - Name of the service (e.g., 'core', 'segment')
• orgId - Organization identifier
• modelName - Name of the collection/schema

Returns: Mongoose Model instance

getOrgIds(): Promise<string[]>

Retrieves organisation ids.

Returns: All the organisation ids for which db connections are made

Array of all available organization IDs.

requiredParamsInterface

Configuration interface for database connections.

interface requiredParamsInterface {
  secretManager: {
    secretsManagerAccessKeyId: string;
    secretsManagerSecretAccessKey: string;
    secretsManagerRegion: string;
  };
  mongoDbServiceManager: Array<{
    serviceName: string;
    collections: Array<{
      name: string;
      schema: Schema;
    }>;
  }>;
  projectName: string;
  mongoConnectionOptions?: ConnectOptions;
}

🔧 Configuration Options

MongoDB Connection Options

mongoConnectionOptions: {
  // Connection pool settings
  maxPoolSize: 10,                    // Maximum number of connections in pool
  minPoolSize: 5,                     // Minimum number of connections in pool
  
  // Timeout settings
  connectTimeoutMS: 10000,            // Connection timeout (10 seconds)
  socketTimeoutMS: 45000,             // Socket timeout (45 seconds)
  serverSelectionTimeoutMS: 5000,     // Server selection timeout
  
  // Retry and reliability
  retryWrites: true,                  // Enable retry writes
  retryReads: true,                   // Enable retry reads
  
  // Read preferences
  readPreference: 'primary',          // Read from primary by default
  
  // Buffer settings
  bufferMaxEntries: 0,                // Disable mongoose buffering
  bufferCommands: false,              // Disable mongoose buffering
  
  // Authentication
  authSource: 'admin',                // Authentication database
  
  // SSL/TLS
  ssl: true,                          // Enable SSL
  sslValidate: true,                  // Validate SSL certificates
}

🏗️ Architecture

┌─────────────────────────────────────────────────────────────────┐
│                        HyperConn Package                        │
├─────────────────────────────────────────────────────────────────┤
│  MongoDbManagerService                                          │
│  ├── AWS Secrets Manager Integration                           │
│  ├── Connection Pool Management                                │
│  └── Model Registry (Org-specific)                             │
└─────────────────────────────────────────────────────────────────┘
                                │
                                ▼
┌─────────────────────────────────────────────────────────────────┐
│                    AWS Secrets Manager                          │
├─────────────────────────────────────────────────────────────────┤
│  org-1-db-credentials │ org-2-db-credentials │ org-n-db-creds   │
└─────────────────────────────────────────────────────────────────┘
                                │
                                ▼
┌─────────────────────────────────────────────────────────────────┐
│                     MongoDB Clusters                            │
├─────────────────────────────────────────────────────────────────┤
│ Org 1 DB    │    Org 2 DB     │    Org N DB                    │
│ └─ Schema A │    └─ Schema A   │    └─ Schema A                 │
│ └─ Schema B │    └─ Schema B   │    └─ Schema B                 │
└─────────────────────────────────────────────────────────────────┘

🔒 Security Best Practices

1. AWS Credentials Management

• Never hardcode AWS credentials in your code
• Use IAM roles when running on AWS infrastructure
• Rotate AWS access keys regularly
• Follow the principle of least privilege for IAM permissions

2. Database Security

• Enable MongoDB authentication and authorization
• Use SSL/TLS for database connections
• Implement network-level security (VPC, Security Groups)
• Regularly update database versions and security patches

3. Environment Variables

# Use strong, unique credentials
AWS_SECRETS_MANAGER_ACCESS_KEY_ID=AKIA...
AWS_SECRETS_MANAGER_SECRET_ACCESS_KEY=...
AWS_SECRETS_MANAGER_REGION=us-east-1

## 🚀 Advanced Usage

### Custom Error Handling

```typescript
import { MongoDbManagerService } from 'hyperconn';

class CustomDatabaseService {
  private dbManager: MongoDbManagerService;

  async safeGetModel(serviceName: string, orgId: string, modelName: string) {
    try {
      return this.dbManager.getModel(serviceName, orgId, modelName);
    } catch (error) {
      console.error(`Failed to get model for ${orgId}:${serviceName}:${modelName}`, error);
      throw new Error(`Database model not available for organization ${orgId}`);
    }
  }
}

Health Checks

@Controller('health')
export class HealthController {
  constructor(private readonly databaseService: DatabaseService) {}

  @Get('database')
  async checkDatabaseHealth() {
    const organizations = await this.databaseService.getAllOrganizations();
    const healthStatus = {};

    for (const orgId of organizations) {
      try {
        const EventModel = this.databaseService.getEventModel(orgId, 'service1', "events");
        await EventModel.findOne().limit(1).exec();
        healthStatus[orgId] = 'healthy';
      } catch (error) {
        healthStatus[orgId] = 'unhealthy';
      }
    }

    return { status: 'ok', organizations: healthStatus };
  }
}

Batch Operations

async bulkCreateEvents(orgId: string, events: any[]) {
  const EventModel = this.databaseService.getEventModel(orgId, 'service1', "events");
  return await EventModel.insertMany(events, { ordered: false });
}

async aggregateEventsByType(orgId: string) {
  const EventModel = this.databaseService.getEventModel(orgId, 'service1', "events");
  return await EventModel.aggregate([
    { $group: { _id: '$eventType', count: { $sum: 1 } } },
    { $sort: { count: -1 } }
  ]);
}

🧪 Testing

Unit Testing Example

import { Test, TestingModule } from '@nestjs/testing';
import { MongoDbManagerService } from 'hyperconn';

describe('DatabaseService', () => {
  let service: DatabaseService;
  let dbManager: MongoDbManagerService;

  beforeEach(async () => {
    const module: TestingModule = await Test.createTestingModule({
      providers: [
        DatabaseService,
        {
          provide: MongoDbManagerService,
          useValue: {
            connectToDatabasesAndModels: jest.fn(),
            getModel: jest.fn(),
            orgIds: ['org1', 'org2']
          }
        }
      ],
    }).compile();

    service = module.get<DatabaseService>(DatabaseService);
    dbManager = module.get<MongoDbManagerService>(MongoDbManagerService);
  });

  it('should be defined', () => {
    expect(service).toBeDefined();
  });
});

📊 Performance Considerations

Connection Pooling

• Default Pool Size: 10 connections per database
• Recommended: Monitor connection usage and adjust based on load
• Scaling: Consider connection limits per MongoDB instance

Memory Usage

• Each organization connection maintains its own model registry
• Monitor memory usage with multiple organizations
• Implement model caching strategies for frequently accessed schemas

Query Optimization

// Use indexes for better query performance
const eventSchema = new Schema({
  userId: { type: String, required: true, index: true },
  eventType: { type: String, required: true, index: true },
  timestamp: { type: Date, default: Date.now, index: true }
});

// Compound indexes for complex queries
eventSchema.index({ userId: 1, timestamp: -1 });
eventSchema.index({ eventType: 1, timestamp: -1 });

🐛 Troubleshooting

Common Issues

Connection Timeout

Error: MongoServerSelectionError: connection timeout

Solution: Increase connectTimeoutMS and serverSelectionTimeoutMS values.

AWS Credentials Error

Error: Missing AWS credentials

Solution: Verify environment variables and IAM permissions.

Schema Not Found

Error: Schema not found for organization

Solution: Ensure the organization exists in AWS Secrets Manager and the schema is properly registered.

Debug Mode

// Enable debug logging
const params: requiredParamsInterface = {
  // ... other params
  mongoConnectionOptions: {
    // ... other options
    debug: true, // Enable Mongoose debug mode
  }
};

🤝 Contributing

We welcome contributions! Please follow these steps:

1. Fork the repository
2. Create a feature branch (git checkout -b feature/amazing-feature)
3. Commit your changes (git commit -m 'Add amazing feature')
4. Push to the branch (git push origin feature/amazing-feature)
5. Open a Pull Request

Development Setup

# Clone the repository
git clone https://github.com/TCG-LS/HyperConn.git
cd hyperconn

# Install dependencies
npm install

# Run tests
npm test

# Build the package
npm run build

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.

📞 Support

• Documentation: GitHub Wiki
• Issues: GitHub Issues
• Discussions: GitHub Discussions

🎯 Roadmap

Version 1.1.0

• [ ] Redis caching integration
• [ ] Connection health monitoring
• [ ] Automatic failover support

Version 1.2.0

• [ ] PostgreSQL support
• [ ] GraphQL integration
• [ ] Advanced monitoring dashboard

Version 2.0.0

• [ ] Microservices architecture support
• [ ] Kubernetes operator
• [ ] Real-time synchronization

🏆 Acknowledgments

• Built with NestJS
• Powered by Mongoose
• Secured by AWS Secrets Manager

Made with ❤️ by the HyperConn Team