@gokiteam/goki-dev

"LocalStack for Goki Services"

A unified local development platform providing mock implementations of external services (GCP Pub/Sub, AWS IoT MQTT, Redis, PostgreSQL, Google Cloud Logging, Firestore) with a centralized Web UI.

Install

# Global install (CLI available everywhere)
npm install -g @gokiteam/goki-dev

# Or as a project dev dependency
npm install --save-dev @gokiteam/goki-dev

Usage

CLI — Manage local infrastructure

# Interactive menu
goki-dev

# Start all services (Docker)
goki-dev start

# Stop services
goki-dev stop

# Check status
goki-dev status

# MCP server for Claude Code integration
goki-dev-mcp

Programmatic — DevToolsClient for tests

import { DevToolsClient } from '@gokiteam/goki-dev'

const devTools = new DevToolsClient()
const traceId = devTools.generateTraceId('user-reg')

// Verify Pub/Sub messages
await devTools.pubsub.waitForMessage({
  filter: { topic: 'user-events' },
  timeout: 5000,
  traceId
})

// Query PostgreSQL
await devTools.postgres.query({
  database: 'auth_dev',
  query: 'SELECT * FROM users',
  traceId
})

// Check Redis
await devTools.redis.get({ key: 'session:user-001', traceId })

// Assert no errors in logs
const errors = await devTools.logging.assertNoErrors({ traceId })

Or use the pre-configured singleton:

import { devTools } from '@gokiteam/goki-dev'

const traceId = devTools.generateTraceId('test')
const message = await devTools.pubsub.waitForMessage({ ... })

Docker (Direct)

docker compose up -d
docker compose logs -f
docker compose down

Access

Production (Docker)

• Web UI: http://localhost:9001 (nginx + React)
• Backend API: http://localhost:9000/v1/*
• Pub/Sub Emulator: http://localhost:8085
• AWS IoT Core HTTPS API: http://localhost:8086 (POST /topics/:topicName)
• Firestore Emulator: http://localhost:8081
• Cloud Logging: http://localhost:8087
• PostgreSQL: localhost:5432 (user: postgres, password: postgres)
• Redis: localhost:6379
• MQTT Broker: localhost:8883

Development

• Web UI (dev): http://localhost:9001 (React dev server)
• Backend API (dev): http://localhost:9000/v1/*

Architecture

The platform uses a microservices architecture with separate frontend and backend containers:

Production Architecture (Docker)

┌─────────────────────────────────────────────────────────────┐
│                     Docker Network (goki-network)            │
├─────────────────────────────────────────────────────────────┤
│                                                               │
│  ┌──────────────────┐         ┌───────────────────┐         │
│  │  Frontend (9001) │         │  Backend (9000)   │         │
│  │  nginx + React   │◄────────│  Node.js API      │         │
│  │                  │   API   │  + Emulators      │         │
│  └──────────────────┘  calls  └─────────┬─────────┘         │
│                                          │                    │
│                                          │                    │
│         ┌────────────────────────────────┼────────┐          │
│         │                                │        │          │
│         ▼                                ▼        ▼          │
│  ┌─────────────┐  ┌───────────────┐  ┌──────┐  ┌──────┐   │
│  │ Pub/Sub     │  │ Firestore     │  │Redis │  │Postgres│  │
│  │ (8085)      │  │ (8081)        │  │(6379)│  │(5432)│    │
│  └─────────────┘  └───────────────┘  └──────┘  └──────┘    │
│                                                               │
└─────────────────────────────────────────────────────────────┘

Services

Frontend Container (dev-tools-frontend):

• Base: nginx:1.25-alpine
• Port: 9001 → 80 (nginx)
• Purpose: Serve React UI with optimized caching and compression
• Features:
  • Gzip compression
  • 1-year cache for static assets
  • No-cache for index.html
  • Security headers
  • SPA routing support

Backend Container (dev-tools-backend):

• Base: node:20-alpine + Java 21
• Ports: 9000 (API), 8085 (Pub/Sub), 8086 (AWS IoT), 8087 (Logging), 8883 (MQTT)
• Purpose: API server and emulators
• Features:
  • RESTful API endpoints
  • GCP Pub/Sub emulator
  • AWS IoT Core HTTPS API emulation
  • Google Cloud Logging emulator
  • MQTT broker (Aedes) with Pub/Sub bridge
  • SQLite for Pub/Sub/Logging persistence

Dependencies:

• PostgreSQL 15 (port 5432)
• Redis 7 (port 6379)
• Pub/Sub Emulator (port 8085)
• Firestore Emulator (port 8081)

Development Architecture

• Backend: Hot-reload with nodemon
• Frontend: React dev server with fast refresh
• Separate containers for each service
• Volume mounts for live code updates

API Endpoints

Health

• GET /v1/health/liveness - Service is alive
• GET /v1/health/readiness - Service is ready to handle requests

Dashboard

• POST /v1/dashboard/stats - Get platform statistics

Services

• POST /v1/services/list - List all services
• POST /v1/services/status - Get service status
• POST /v1/services/restart - Restart a service

Pub/Sub

• POST /v1/pubsub/topics/list - List topics
• POST /v1/pubsub/topics/create - Create topic
• POST /v1/pubsub/messages/publish - Publish message
• POST /v1/pubsub/messages/pull - Pull messages
• GET /v1/pubsub/messages/stream - Stream messages (SSE)

PostgreSQL

• POST /v1/postgres/connection/test - Test connection
• POST /v1/postgres/databases/list - List databases
• POST /v1/postgres/tables/list - List tables
• POST /v1/postgres/rows/list - List rows
• POST /v1/postgres/query/execute - Execute query

Redis

• POST /v1/redis/connection/test - Test connection
• POST /v1/redis/info - Get info
• POST /v1/redis/keys/scan - Scan keys
• POST /v1/redis/keys/get - Get key value
• POST /v1/redis/keys/update - Update key
• POST /v1/redis/keys/delete - Delete key

Firestore

• POST /v1/firestore/collections/list - List collections
• POST /v1/firestore/documents/list - List documents
• POST /v1/firestore/documents/get - Get document
• POST /v1/firestore/query/execute - Execute query

Export/Import

• POST /v1/data/export - Export all data
• POST /v1/data/import - Import data
• POST /v1/data/clear - Clear all data

MQTT and AWS IoT Core Emulation

The platform provides full MQTT broker and AWS IoT Core HTTPS API emulation for local development and testing with backend services like device-simulator.

Architecture (Docker)

┌─────────────────────────────────────────────────────────────┐
│                     Docker Network (goki-network)            │
├─────────────────────────────────────────────────────────────┤
│                                                               │
│  device-simulator container                                  │
│    ↓ [MQTT to dev-tools-backend:8883]                       │
│  dev-tools-backend: Aedes MQTT Broker                       │
│    ↓ [Pub/Sub Bridge]                                        │
│  pubsub-emulator (port 8085)                                 │
│    ↓ [Topic: mqttMessageReceived]                           │
│  device-simulator (subscribes via pubsub-emulator:8085)     │
│                                                               │
│  device-simulator                                            │
│    ↓ [POST to dev-tools-backend:8086/topics/...?qos=1]     │
│  dev-tools-backend: AWS IoT Core HTTPS API                   │
│    ↓ [Publishes to Aedes broker]                            │
│  device-simulator (subscribed to response topic via MQTT)   │
│                                                               │
└─────────────────────────────────────────────────────────────┘

Key Service Names (Docker networking):

• dev-tools-backend - MQTT broker (8883) + AWS IoT API (8086)
• pubsub-emulator - Pub/Sub emulator (8085)
• redis - Redis (6379)
• postgres - PostgreSQL (5432)

From Host Machine: Use localhost instead of service names

MQTT Broker (Port 8883)

• Implementation: Aedes MQTT broker (pure Node.js)
• Protocol: MQTT 3.1.1
• Authentication: Relaxed mode (accept all connections for dev)
• Features:
  • QoS levels 0, 1
  • Last Will Testament (LWT)
  • Retained messages
  • Persistent connections

Connect with MQTT client:

From Docker container (using service name):

import mqtt from 'mqtt'

const client = mqtt.connect('mqtt://dev-tools-backend:8883', {
  clientId: 'test-device-123'
})

client.on('connect', () => {
  client.subscribe('rs/+/test-device-123')
  client.publish('rq/test/test-device-123', Buffer.from('packet'), { qos: 1 })
})

From host machine (using localhost):

import mqtt from 'mqtt'

const client = mqtt.connect('mqtt://localhost:8883', {
  clientId: 'test-device-123'
})

AWS IoT Core HTTPS API (Port 8086)

Mimics AWS IoT Core's HTTPS publishing API for backend services.

Endpoint: POST http://localhost:8086/topics/:topicName?qos=1

• Body: Binary payload (Buffer)
• Query: qos (optional, defaults to 1)
• Response: Empty 200 (matching AWS IoT Core)
• Auth: SigV4 headers accepted but not validated (dev mode)

Example usage (device-simulator):

When device-simulator runs in Docker (on goki-network):

# device-simulator Docker environment
AWS_IOT_CORE_MQTT_ENDPOINT=dev-tools-backend:8086  # Use service name
PUBSUB_EMULATOR_HOST=pubsub-emulator:8085          # Use service name

When device-simulator runs on host machine:

# device-simulator local development
AWS_IOT_CORE_MQTT_ENDPOINT=localhost:8086  # Use localhost
PUBSUB_EMULATOR_HOST=localhost:8085        # Use localhost

Publishing (no code changes needed):

const response = await axios.post(
  `http://${AWS_IOT_CORE_MQTT_ENDPOINT}/topics/rs/test/device-123?qos=1`,
  binaryPacket,  // Buffer
  { headers: { 'Content-Type': 'application/octet-stream' } }
)

MQTT → Pub/Sub Bridge

Automatically routes MQTT messages to Pub/Sub (mimics AWS IoT Core behavior).

Topic: mqttMessageReceived Message Format:

{
  "clientId": "test-device-123",
  "topicName": "rq/test/test-device-123",
  "packet": "base64-encoded-binary",
  "receivedAtMilliseconds": 1234567890
}

Lifecycle Events:

• Topic: simulator/connected - Client connected
• Topic: simulator/disconnected - Client disconnected

Topic Patterns (Goki Conventions)

• Request: rq/{action}/{clientId}
• Response: rs/{action}/{clientId}
• Notification: nf/{action}/{clientId}
• Will: w/{action}/{clientId}
• Lifecycle: simulator/connected, simulator/disconnected

Testing

End-to-End Tests

Comprehensive E2E test suite using Playwright:

# Run all E2E tests
npm run test:e2e

# Run UI tests only
npm run test:e2e:ui

# Run workflow tests only
npm run test:e2e:workflows

# Run with browser UI
npm run test:e2e:headed

# Debug mode
npm run test:e2e:debug

See E2E_TEST_GUIDE.md for complete testing documentation.

Integration Tests

# Run all integration tests
npm run test:integration

# Run specific test categories
npm run test:api
npm run test:emulation

Docker Images

Production Images

• Backend: dev-tools-dev-tools-backend (~1.19GB)

  • Includes Java 21 for Firebase emulator
  • Python3, make, g++ for native dependencies
  • Firebase CLI

• Frontend: dev-tools-dev-tools-frontend (~464MB)

  • Multi-stage build (React → nginx)
  • Optimized static assets

Development Images

• Backend: Uses Dockerfile.dev with hot-reload
• Frontend: Uses ui/Dockerfile.dev with React dev server

Environment Variables

Backend

• NODE_ENV - Environment (development/production)
• WEB_UI_PORT - Backend port (default: 9000)
• PUBSUB_EMULATOR_HOST - Pub/Sub emulator host
• FIRESTORE_EMULATOR_HOST - Firestore emulator host
• POSTGRES_HOST - PostgreSQL host
• POSTGRES_PORT - PostgreSQL port
• POSTGRES_USER - PostgreSQL user
• POSTGRES_PASSWORD - PostgreSQL password
• REDIS_HOST - Redis host
• REDIS_PORT - Redis port
• DATA_DIR - Data directory for SQLite

Frontend

• REACT_APP_API_URL - Backend API URL (default: http://localhost:9000)

Troubleshooting

Backend won't start

# Check logs
docker logs goki-dev-tools-backend

# Common issue: Corrupted SQLite database
rm -f data/dev-tools.db*
docker compose restart dev-tools-backend

Frontend won't load

# Check logs
docker logs goki-dev-tools-frontend

# Verify backend is running
curl http://localhost:9000/v1/health/readiness

# Rebuild frontend
docker compose build dev-tools-frontend
docker compose up -d dev-tools-frontend

PostgreSQL/Redis not connecting

# Check if services are running
docker compose ps

# Verify ports are accessible
nc -zv localhost 5432  # PostgreSQL
nc -zv localhost 6379  # Redis

# Restart services
docker compose restart postgres redis

Project Status

✅ Production Ready - Complete with comprehensive E2E tests

Features

• ✅ Pub/Sub message management with real-time streaming
• ✅ PostgreSQL database viewer with query execution
• ✅ Redis key-value browser with type support
• ✅ Firestore document viewer with queries
• ✅ Google Cloud Logging viewer with filters
• ✅ MQTT broker monitoring
• ✅ Service control dashboard
• ✅ Export/Import functionality
• ✅ Docker auto-start for CLI
• ✅ Split frontend/backend architecture

See plans/platform-specification.md for complete specification.

Documentation

• Docker Setup
• E2E Testing Guide