@kitiumai/docker

Enterprise-ready Docker package for Kitium AI services with production-grade configurations, multi-stage builds, Docker Compose orchestration, and complete networking setup.

Features

✅ Multi-stage Builds - Optimized Docker images with minimal production footprint ✅ Docker Compose - Complete local development and production environments ✅ Custom Networking - Isolated bridge network for inter-service communication ✅ Volume Management - Persistent data storage and log management ✅ Security Best Practices - Non-root users, health checks, resource limits ✅ Production Ready - Environment-specific configurations and scaling options ✅ Comprehensive Monitoring - Health checks, logging, and service status ✅ Database Integration - PostgreSQL with initialization scripts and backups ✅ Caching Layer - Redis for session and cache management ✅ Supply-chain Security - SBOM generation, image signing, Conftest policies, and secret mounts ✅ Observability - Vector log shipping, OpenTelemetry collector, Prometheus + Grafana dashboards ✅ Cloud Ready - Kustomize bases and Helm chart with ingress, autoscaling, and TLS ✅ Developer Ergonomics - Dev proxy with HTTPS, VS Code devcontainer, hot reload profiles

Project Structure

tooling/docker/
├── Dockerfile.template        # Reusable template for all services
├── docker-compose.yml         # Base configuration
├── docker-compose.dev.yml     # Development overrides
├── docker-compose.prod.yml    # Production overrides
├── .env.example               # Comprehensive environment template
├── .dockerignore              # Docker build exclusions
├── package.json               # NPM scripts for Docker management
├── scripts/
│   ├── init-db.sql            # Database initialization
│   ├── backup-database.sh     # Backup script
│   ├── restore-database.sh    # Restore script
│   └── health-check.sh        # Health monitoring
├── volumes/                   # Data persistence
│   ├── postgres_data/
│   ├── redis_data/
│   ├── api_logs/
│   └── web_logs/
└── README.md                  # This file

apps/
├── api/                       # Node.js Express API service
│   ├── Dockerfile             # Multi-stage build (based on template)
│   ├── src/
│   │   └── index.ts           # API entry point
│   ├── package.json
│   ├── tsconfig.json
│   └── .dockerignore
└── website/                   # Next.js web application
    ├── Dockerfile             # Multi-stage build (based on template)
    ├── package.json
    ├── next.config.js
    └── .dockerignore

Docker Setup Overview

Key Files

| File | Purpose | |------|---------| | Dockerfile.template | Reusable template for creating service Dockerfiles | | docker-compose.yml | Base Docker Compose configuration | | .env.example | Comprehensive environment variable template | | init-db.sql | Database schema and initial data |

Dockerfile Strategy

All service Dockerfiles are based on Dockerfile.template and include:

• Multi-stage builds - Separate build and production stages
• Development target - Hot reload with all dev dependencies
• Production target - Minimal image with only runtime deps
• Workspace support - Works with pnpm monorepo structure

Each app (api, website) has its own Dockerfile in its directory that extends the template pattern.

Quick Start

Prerequisites

• Docker Engine 20.10+
• Docker Compose 2.0+
• Git
• Node.js 20+ (for local development)
• pnpm 9+ (for monorepo management)

Installation

1. Navigate to KitiumAI root directory

   cd KitiumAI
   cd tooling/docker

2. Create environment file from template

   cp .env.example .env
   # Edit .env with your configuration
   # ⚠️  Change all default passwords in production!

3. Build and start services (Development)

   docker-compose -f docker-compose.yml -f docker-compose.dev.yml build
   docker-compose -f docker-compose.yml -f docker-compose.dev.yml up

   Optional overlays:

   # Add observability sidecars
   docker-compose -f docker-compose.yml -f docker-compose.dev.yml -f docker-compose.observability.yml up
   
   # Enable secrets and Vault dev server
   docker-compose -f docker-compose.yml -f docker-compose.security.yml up
   
   # Local HTTPS proxy (requires mkcert certificates in ./secrets/mkcert)
   docker-compose -f docker-compose.yml -f docker-compose.dev.yml -f docker-compose.devproxy.yml up

4. Verify services are running

   docker-compose ps
   npm run healthcheck

5. Access services

   • API: http://localhost:3000
   • Web: http://localhost:3001
   • Database: localhost:5432 (psql)
   • Redis: localhost:6379 (redis-cli)

Available Commands

Development Environment

# Start services in development mode
npm run dev

# Build all services
npm run dev:build

# Stop all services
npm run dev:down

# View real-time logs
npm run logs

# Run tests
npm run test

# Run linter
npm run lint

Production Environment

# Start services in production mode
npm run prod

# Build for production
npm run prod:build

# Stop production services
npm run prod:down

# Check health of all services
npm run healthcheck

# Clean up containers and volumes
npm run clean

Database Operations

# Backup database
./scripts/backup-database.sh

# Point-in-time recovery WAL archive
./scripts/backup-wal.sh

# Restore from backup
./scripts/restore-database.sh backups/kitium_db_backup_*.sql.gz

# Restore WAL archives
./scripts/restore-wal.sh backups/wal_*.tar.gz

# Run migrations with prisma (override via MIGRATION_COMMAND)
./scripts/run-migrations.sh api

# Load redacted QA data
./scripts/seed-redacted-data.sh

# Health check all services
./scripts/health-check.sh

Supply Chain and Compliance

# Generate SBOM for an image (requires syft)
./scripts/generate-sbom.sh ghcr.io/kitium-ai/api:latest

# Sign an image with cosign (COSIGN_KEY env optional for key-based signing)
./scripts/sign-image.sh ghcr.io/kitium-ai/api:latest

CI/CD runs lint/test, Hadolint, Trivy, Syft, Cosign, and Conftest automatically via .github/workflows/ci.yml.

Configuration

Environment Variables

Create a .env file based on .env.example:

# Node.js Environment
NODE_ENV=development
LOG_LEVEL=info

# Database
POSTGRES_USER=kitium
POSTGRES_PASSWORD=secure_password
POSTGRES_DB=kitium_dev
POSTGRES_PORT=5432

# Redis
REDIS_PASSWORD=redis_password
REDIS_PORT=6379

# API Service
API_PORT=3000
CORS_ORIGIN=*

# Observability / OTEL
OTEL_EXPORTER_AUTH_TOKEN=replace-me

# Dev proxy
DEV_DOMAIN=kitium.localhost

# Web Service
WEB_PORT=3001
NEXT_PUBLIC_API_URL=http://localhost:3000

Docker Compose Overrides

• docker-compose.yml - Base configuration with all services
• docker-compose.dev.yml - Development overrides (volume mounts, debug logging)
• docker-compose.prod.yml - Production overrides (resource limits, restart policies)

Multi-stage Docker Build Targets

Each service Dockerfile includes multiple build targets:

# Development target (with hot reload)
docker build --target development .

# Production target (minimal image)
docker build --target production .

Creating New Service Dockerfiles

To add a Dockerfile for a new service:

1. Copy the template to your service directory

   cp tooling/docker/Dockerfile.template apps/my-service/Dockerfile

2. Customize the Dockerfile

   • Update the package filter name: @kitium-ai/my-service
   • Adjust the health check endpoint as needed
   • Update the entry point if not dist/index.js

3. Example for a new service

   # For apps/my-service/
   RUN pnpm --filter=@kitium-ai/my-service run build
   CMD ["node", "apps/my-service/dist/index.js"]

4. Add to docker-compose.yml

   my-service:
     build:
       context: ../..
       dockerfile: ./apps/my-service/Dockerfile
       target: ${BUILD_TARGET:-production}
     container_name: kitium-my-service
     ports:
       - "${MY_SERVICE_PORT:-3002}:3002"
     # ... rest of configuration

5. Add environment variables to .env

   MY_SERVICE_PORT=3002
   # ... other service-specific vars

Services

API Service (Node.js/Express)

• Port: 3000
• Health Check: GET /health
• Endpoints:
  • GET /version - Service version
  • GET /api/v1/status - API status

Web Service (Next.js)

• Port: 3001
• Health Check: GET /
• Features: Server-side rendering, static generation

PostgreSQL Database

• Port: 5432
• Version: 15-Alpine
• Database: kitium_dev
• Initialization: Automatic schema and tables creation
• Persistence: Named volume postgres_data

Redis Cache

• Port: 6379
• Version: 7-Alpine
• Password: Configured via REDIS_PASSWORD
• Persistence: AOF enabled for data durability

Docker Networking

All services communicate via a custom bridge network (kitium-network):

                    ┌─────────────────┐
                    │   kitium-web    │
                    │   (3001:3001)   │
                    └────────┬────────┘
                             │
                    ┌────────▼────────┐
                    │   kitium-api    │
                    │   (3000:3000)   │
                    └────┬────────┬───┘
                         │        │
         ┌───────────────┘        └──────────────┐
         │                                       │
    ┌────▼─────────┐              ┌──────────────▼──┐
    │   postgres   │              │     redis      │
    │  (5432:5432) │              │  (6379:6379)  │
    └──────────────┘              └────────────────┘

Network: kitium-network (172.20.0.0/16)

Service Hostnames

Services can communicate using hostnames:

• api:3000 - API service
• web:3001 - Web service
• postgres:5432 - PostgreSQL
• redis:6379 - Redis

Volume Management

Named Volumes

| Volume | Mount Point | Purpose | |--------|------------|---------| | postgres_data | /var/lib/postgresql/data | PostgreSQL database storage | | redis_data | /data | Redis persistence | | api_logs | /app/logs | API service logs | | web_logs | /app/.next/logs | Next.js logs |

Volume Locations

All volumes are stored in the ./volumes/ directory on the host:

volumes/
├── postgres_data/      # PostgreSQL data files
├── redis_data/        # Redis dump files
├── api_logs/          # API application logs
└── web_logs/          # Web application logs

Security

Best Practices Implemented

1. Non-root Users - All services run with non-root user (uid: 1000)
2. Read-only Filesystems - Where applicable
3. Resource Limits - CPU and memory constraints
4. Health Checks - Automated service health verification
5. Network Isolation - Services on custom bridge network
6. Secrets Management - Environment-based configuration
7. Security Headers - Helmet.js for HTTP headers
8. CORS Configuration - Configurable CORS policies

Security Scanning

# Scan for vulnerabilities in dependencies
docker run --rm -v /var/run/docker.sock:/var/run/docker.sock \
  aquasec/trivy image kitium-api:latest

Monitoring and Logging

Service Status

# View all containers
docker-compose ps

# View service logs
docker-compose logs -f api
docker-compose logs -f web
docker-compose logs -f postgres

# Filter logs by timestamp
docker-compose logs --since 2024-01-01 api

Health Checks

Each service includes automated health checks:

# Run comprehensive health check
npm run healthcheck

# Manual API health check
curl http://localhost:3000/health

# Manual web health check
curl http://localhost:3001

Performance Monitoring

# View resource usage
docker stats

# View detailed service stats
docker-compose stats

Database Management

Initialize Database

# Run initialization script automatically on startup
docker-compose up postgres

# Manual initialization
docker-compose exec postgres psql -U kitium -d kitium_dev -f /docker-entrypoint-initdb.d/init.sql

Backup and Restore

# Create backup
./scripts/backup-database.sh

# Restore from backup
./scripts/restore-database.sh backups/kitium_db_backup_20240101_120000.sql.gz

# List backups
ls -lh backups/

Database Access

# Connect to database
docker-compose exec postgres psql -U kitium -d kitium_dev

# Execute SQL query
docker-compose exec postgres psql -U kitium -d kitium_dev -c "SELECT version();"

Performance Optimization

Multi-stage Build Benefits

• Development: Full features with hot reload
• Production: Minimal image size, optimized dependencies
• Layer Caching: Faster builds by caching dependencies

Resource Limits

Configure in docker-compose.yml:

deploy:
  resources:
    limits:
      cpus: '2'
      memory: 1024M
    reservations:
      cpus: '0.5'
      memory: 512M

Volume Mount Caching

Development volumes use cached option for performance:

volumes:
  - ./services/api/src:/app/src:cached

Troubleshooting

Services Won't Start

# Check logs for errors
docker-compose logs api
docker-compose logs web
docker-compose logs postgres

# Validate configuration
npm run validate

Database Connection Issues

# Check database health
docker-compose exec postgres pg_isready -U kitium

# Check network connectivity
docker-compose exec api ping postgres
docker-compose exec api curl http://redis:6379

High Memory Usage

# Check container resource usage
docker stats

# Reduce limits in .env and docker-compose override
export POSTGRES_MEMORY=256M

Port Already in Use

# Change port in .env
API_PORT=3002
WEB_PORT=3003

# Or kill the process using the port
lsof -i :3000
kill -9 <PID>

Production Deployment

Pre-deployment Checklist

• [ ] All environment variables configured
• [ ] Database backups tested
• [ ] SSL/TLS certificates obtained
• [ ] Resource limits appropriate for server
• [ ] Health checks passing consistently
• [ ] Load balancer configured (if applicable)

Docker Swarm Deployment

# Initialize swarm (single node)
docker swarm init

# Deploy stack
docker stack deploy -c docker-compose.yml kitium

# View services
docker service ls

# Scale services
docker service scale kitium_api=3

Observability Stack

• docker-compose.observability.yml adds Vector (log shipping), OpenTelemetry collector (traces/metrics/logs), Prometheus (scrapes cAdvisor + OTEL), and Grafana.
• Configuration lives in observability/ and can be customized for downstream APM backends.

Access points:

• Prometheus: http://localhost:9090
• Grafana: http://localhost:3003 (admin/admin)
• OTLP endpoints: gRPC 4317 / HTTP 4318

Secrets, SBOMs, and Image Signing

• docker-compose.security.yml mounts Docker secrets for API/DB/Redis and starts a Vault dev server for local experiments.
• Use SOPS + age or cloud KMS to store files under ./secrets/; referenced secrets are not committed.
• SBOM + signing automation: scripts/generate-sbom.sh and scripts/sign-image.sh mirror the CI pipeline so artifacts can be uploaded to registries with provenance.

Kubernetes and Helm Deployments

• deploy/kubernetes provides a Kustomize base (namespace, ConfigMap, Secret, StatefulSets/Deployments, Ingress, HPA) with overlays for dev/stage/prod.

  kubectl apply -k deploy/kubernetes/overlays/dev

• deploy/helm ships a lightweight chart mirroring Compose defaults.

  helm install kitium deploy/helm --set env.postgresPassword=$POSTGRES_PASSWORD --set env.redisPassword=$REDIS_PASSWORD

Developer Experience

• .devcontainer/devcontainer.json enables instant VS Code Dev Containers with Docker-in-Docker and pnpm preinstalled.
• docker-compose.devproxy.yml brings Traefik with automatic HTTPS for kitium.localhost (compatible with mkcert certificates in ./secrets/mkcert).
• Continue using docker-compose.dev.yml for hot reload; buildx multi-arch builds are available via the CI workflow or docker buildx build --platform linux/amd64,linux/arm64 ... locally.

API References

• API service base path: ${API_BASE_PATH:-/api} with health endpoint at /health.
• Web service exposed on / with proxy-friendly headers via Traefik/Ingress.

Contributing

1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Test locally with all environments
5. Submit a pull request

License

MIT License - See LICENSE file for details

Support

• Issues: https://github.com/kitium-ai/docker/issues
• Discussions: https://github.com/kitium-ai/docker/discussions
• Email: [email protected]

Changelog

See CHANGELOG.md for version history and breaking changes.

Last Updated: 2024-01-21 Version: 1.0.0 Maintainer: Kitium AI Team