Gateway Type 3 - Seller-Managed Gateway

Overview

Type 3 Gateway is a seller-managed gateway client that runs on the seller's infrastructure. It connects to the marketplace via WebSocket and uses the seller's own API keys to fulfill requests.

Architecture

• Deployment: Seller's own infrastructure
• Credentials: Seller's own API keys
• Connection: WebSocket client to marketplace
• Control: Fully managed by seller

Features

• Persistent WebSocket connection
• Automatic reconnection with exponential backoff
• Heartbeat monitoring
• Request/response routing to provider APIs
• Version management with auto-upgrade notifications
• Interactive CLI UI with chat interface and model management
• Intelligent Image Generation with keyword-based auto-detection
• Terminal Image Preview for generated images
• Automation Mode: Full session management with bash, SSH, and system tools

🚀 Quick Install (Recommended)

Linux/macOS

Install the gateway on any Linux/macOS server with a single command:

# Using unpkg CDN (recommended)
curl -fsSL https://unpkg.com/langmart-gateway-type3/scripts/install.sh | bash -s -- \
  --marketplace-url ws://your-marketplace:8081 \
  --api-key YOUR_API_KEY \
  --full \
  --service

Windows (PowerShell)

Install on Windows using PowerShell:

# One-liner installation
$env:MARKETPLACE_URL="ws://your-marketplace:8081"; $env:GATEWAY_API_KEY="YOUR_API_KEY"; iwr -useb https://unpkg.com/langmart-gateway-type3/scripts/install.ps1 | iex

# Or download and run with parameters
iwr -OutFile install.ps1 https://unpkg.com/langmart-gateway-type3/scripts/install.ps1
.\install.ps1 -MarketplaceUrl "ws://your-marketplace:8081" -ApiKey "YOUR_API_KEY" -Full -Service

Install Options

Linux/macOS (Bash)

| Option | Description | Default | |--------|-------------|---------| | --marketplace-url URL | Marketplace WebSocket URL | ws://localhost:8081 | | --api-key KEY | Gateway API key for authentication | (none) | | --port PORT | Gateway port | 8083 | | --install-dir DIR | Installation directory | /opt/langmart-gateway | | --capabilities CAPS | Capabilities: session,llm or llm | llm | | --full | Enable full capabilities (Automation + LLM) | (disabled) | | --service | Install as systemd service | (disabled) |

Windows (PowerShell)

| Parameter | Description | Default | |-----------|-------------|---------| | -MarketplaceUrl URL | Marketplace WebSocket URL | ws://localhost:8081 | | -ApiKey KEY | Gateway API key for authentication | (none) | | -Port PORT | Gateway port | 8083 | | -InstallDir DIR | Installation directory | C:\langmart-gateway | | -Capabilities CAPS | Capabilities: session,llm or llm | llm | | -Full | Enable full capabilities (Automation + LLM) | (disabled) | | -Service | Install as Windows Service (requires nssm) | (disabled) |

Examples

Linux/macOS

# LLM routing only (default)
curl -fsSL https://unpkg.com/langmart-gateway-type3/scripts/install.sh | bash -s -- \
  --marketplace-url ws://10.0.1.117:8081 \
  --api-key sk-test-inference-key

# Full Automation mode with systemd service
curl -fsSL https://unpkg.com/langmart-gateway-type3/scripts/install.sh | bash -s -- \
  --marketplace-url ws://10.0.1.117:8081 \
  --api-key sk-test-inference-key \
  --full \
  --service

# Custom port and directory
curl -fsSL https://unpkg.com/langmart-gateway-type3/scripts/install.sh | bash -s -- \
  --marketplace-url ws://10.0.1.117:8081 \
  --api-key sk-test-inference-key \
  --port 9000 \
  --install-dir /home/user/gateway

Windows

# LLM routing only (default)
.\install.ps1 -MarketplaceUrl "ws://10.0.1.117:8081" -ApiKey "sk-test-inference-key"

# Full Automation mode with Windows Service
.\install.ps1 -MarketplaceUrl "ws://10.0.1.117:8081" -ApiKey "sk-test-inference-key" -Full -Service

# Custom port and directory
.\install.ps1 -MarketplaceUrl "ws://10.0.1.117:8081" -ApiKey "sk-test-inference-key" -Port 9000 -InstallDir "D:\gateway"

Alternative CDN

# Linux/macOS - Using jsdelivr
curl -fsSL https://cdn.jsdelivr.net/npm/langmart-gateway-type3/scripts/install.sh | bash -s -- [OPTIONS]

# Windows - Using jsdelivr
iwr -useb https://cdn.jsdelivr.net/npm/langmart-gateway-type3/scripts/install.ps1 | iex

📦 Manual Installation

Option 1: npm install

# Create installation directory
mkdir -p /opt/langmart-gateway && cd /opt/langmart-gateway

# Initialize and install
npm init -y
npm install langmart-gateway-type3

# Create .env file
cat > .env << EOF
INSTANCE_ID=gw3-$(hostname)-$(date +%s | tail -c 5)
GATEWAY_PORT=8083
MARKETPLACE_URL=ws://your-marketplace:8081
GATEWAY_API_KEY=your-api-key
GATEWAY_CAPABILITIES=llm
NODE_ENV=production
EOF

# Start the gateway
node node_modules/langmart-gateway-type3/dist/index-server.js

Option 2: Clone and Build

# Clone the repository
git clone https://github.com/YiHuangDB/LangMartDesign.git
cd LangMartDesign/gateway-type3

# Install dependencies
npm install

# Build TypeScript
npm run build

# Configure
cp .env.example .env
# Edit .env with your settings

# Start
npm start

⚙️ Configuration

Environment Variables

# Gateway identification
INSTANCE_ID=gw3-server-001           # Gateway identifier (auto-generated if not set)
GATEWAY_PORT=8083                     # Gateway HTTP/WebSocket port

# Marketplace connection
MARKETPLACE_URL=ws://localhost:8081   # Marketplace WebSocket URL
GATEWAY_API_KEY=sk-your-key          # Marketplace API key

# Capabilities
GATEWAY_CAPABILITIES=llm              # "llm" or "session,llm" for full Automation

# Local vault (for storing provider API keys)
VAULT_PATH=.vault/credentials.enc     # Encrypted vault path
VAULT_PASSWORD=random-secure-password # Vault encryption password

# Provider API Keys (seller's own keys - stored in vault)
OPENAI_API_KEY=sk-...                # OpenAI API key
ANTHROPIC_API_KEY=sk-ant-...         # Anthropic API key
GROQ_API_KEY=gsk_...                 # Groq API key

Capability Modes

| Mode | GATEWAY_CAPABILITIES | Description | |------|---------------------|-------------| | LLM Only | llm | Routes LLM requests to providers (default) | | Full Automation | session,llm | LLM routing + bash, SSH, file ops, system tools |

🏃 Running

Manual Start

cd /opt/langmart-gateway

# If installed via npm
./start.sh

# Or directly
node node_modules/langmart-gateway-type3/dist/index-server.js

Systemd Service

If installed with --service:

# Start
sudo systemctl start langmart-gateway

# Stop
sudo systemctl stop langmart-gateway

# Status
sudo systemctl status langmart-gateway

# Logs
sudo journalctl -u langmart-gateway -f

Verify Installation

# Health check
curl http://localhost:8083/health

# Expected response:
{
  "status": "healthy",
  "gatewayId": "gw3-server-12345",
  "capabilities": ["session", "llm"],
  "tools": 18
}

How It Works

1. Connection: Gateway connects to marketplace via WebSocket
2. Authentication: Authenticates using marketplace API key
3. Registration: Registers available models and capabilities
4. Requests: Receives inference requests from marketplace
5. Processing: Routes requests to appropriate provider using seller's keys
6. Response: Streams responses back to marketplace

Interactive CLI UI

Overview

Gateway Type 3 includes a powerful interactive CLI UI built with React Ink that provides a rich terminal-based interface for:

• Chat conversations with LLM models
• Model management and browsing
• Connection management
• Automatic image generation with keyword detection
• Terminal-based image previews

Running the CLI UI

Local Development

# Start the interactive UI locally
npm run ui

# Or with debug mode
npm run ui:debug

Docker Container

# Access the UI in a running Docker container
docker exec -it <container-name> npm run ui

# Example with actual container name
docker exec -it langmart-gw3-aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa npm run ui

Chat Interface Features

Basic Chat

The CLI UI provides a split-screen interface with:

• Top Panel: Chat conversation history
• Bottom Panel: Input field with autocomplete support
• Status Bar: Current model, connection status, and user info

Model Selection

Press Tab key to enter model selection mode where you can:

• Browse available models
• Filter models by typing keywords (AND-condition search)
• View model details (pricing, context window, capabilities)
• Select models from your own connections or the marketplace

Slash Commands

Type / to see available commands:

• /clear - Clear conversation history
• /connections - List your connections
• /models - Browse and filter models
• /vault - Manage API keys
• /help - Show available commands

🎨 Intelligent Image Generation

Overview

The CLI UI automatically detects when you want to generate an image and seamlessly switches from the chat completion API to the image generation API. No special commands needed!

How It Works

The image detection system analyzes your input for:

1. Action Keywords (what you want to do):

   • generate
   • create
   • draw
   • make
   • produce
   • design

2. Subject Keywords (what to generate):

   • image
   • picture
   • photo
   • illustration
   • artwork
   • visual

When both an action keyword AND a subject keyword are detected, and your selected model has image_gen capability, the CLI automatically:

1. Extracts the clean prompt
2. Calls the image generation API (/v1/images/generations)
3. Displays the image preview in the terminal
4. Saves the conversation history

Example Prompts

✅ These will trigger image generation:

generate an image of a sunset over mountains
create a picture of a futuristic city
draw an illustration of a cat wearing a hat
make a photo of abstract art
produce artwork of a forest landscape
design a visual of a cyberpunk street scene

❌ These will use regular chat:

what is a sunset?              (no subject keyword)
tell me about images           (no action keyword)
how do I create a website?     (wrong context)

Automatic Prompt Cleaning

The system automatically removes action phrases from your prompt to extract the clean description:

Input:

generate an image of a beautiful sunset over mountains

Extracted Prompt:

a beautiful sunset over mountains

This ensures the image generation API receives clean, descriptive prompts.

Model Requirements

For image generation to work:

• Your selected model MUST have image_gen capability
• The model must support the /v1/images/generations endpoint
• Compatible models include: DALL-E, Stable Diffusion, and similar image generation models

Terminal Image Preview

When an image is generated, you'll see:

1. ✅ Success confirmation message
2. The prompt used for generation
3. Terminal-based image preview (ASCII/pixel art rendering)
4. Note about terminal resolution limitations

The preview uses 50% of your terminal width/height for optimal display.

Image Generation API Call

Behind the scenes, the CLI calls:

POST /v1/images/generations
{
  "model": "your-selected-model",
  "prompt": "extracted-clean-prompt",
  "n": 1,
  "size": "1024x1024",
  "response_format": "b64_json"
}

The response is automatically:

• Decoded from base64
• Converted to terminal-friendly display format
• Rendered in the chat interface
• Added to conversation history

Advanced Features

Keyword-Based Filtering

In model selection mode, type keywords to filter:

groq llama        # Shows only Groq models with "llama" in name
openai gpt-4      # Shows only OpenAI GPT-4 models
image gen         # Shows only models with image generation

Multiple keywords use AND logic - all must match.

Autocomplete Support

Type / and the CLI will suggest commands:

• Type /c → suggests /clear
• Press Tab to accept suggestion
• Cursor automatically positioned at end

Tool Calling

The CLI supports native function calling with:

• Marketplace tools (connections, models, vault, logs)
• Web tools (search, fetch)
• Core tools (system operations)
• MCP tools (Model Context Protocol servers)

Conversation Persistence

Conversations are automatically saved to:

~/.gateway-type3/sessions/<session-id>.json

This allows resuming conversations across sessions.

Troubleshooting

Image Generation Not Working

Problem: Image generation doesn't trigger Solutions:

1. Verify your model has image_gen capability
2. Include both action and subject keywords
3. Check model is properly selected (not "No model selected")
4. Verify marketplace connection is active

Problem: Image preview not showing Solutions:

1. Check terminal supports image rendering
2. Verify terminal-image package is installed
3. Check Docker container has necessary dependencies
4. Review logs for base64 decode errors

Connection Issues

Problem: CLI shows "Not connected" Solutions:

1. Verify Gateway Type 3 server is running
2. Check MARKETPLACE_URL environment variable
3. Ensure API key is valid and authenticated
4. Review gateway logs for connection errors

Model List Empty

Problem: No models shown in model selection Solutions:

1. Wait for initial model sync to complete
2. Verify connections are configured and active
3. Check API keys are valid in vault
4. Run /connections to verify connection status

Docker-Specific Instructions

Running in Docker

When using the Docker container, you can access the CLI UI by:

# 1. Find your container name
docker ps | grep type3

# 2. Access the interactive UI
docker exec -it <container-name> npm run ui

# 3. Exit the UI
# Press Ctrl+C to exit back to shell

Environment Variables for Docker

The container needs these environment variables:

GATEWAY_API_KEY=sk-test-inference-key           # Your marketplace API key
MARKETPLACE_URL=ws://localhost:8081             # WebSocket URL for gateway connection
MARKETPLACE_API_URL=http://localhost:8081       # HTTP URL for REST API calls
USER_ID=aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa   # Your user ID

Important: The Docker container needs both URLs:

• MARKETPLACE_URL (ws://) for WebSocket gateway connection
• MARKETPLACE_API_URL (http://) for REST API calls (models, connections, etc.)

Persistent Data in Docker

The container uses volumes to persist:

• /app/data/vault - API keys and credentials
• /app/data/config - Gateway configuration
• /root/.gateway-type3/sessions - Conversation history

Make sure to mount these volumes when creating the container:

docker run -d \
  --name my-gateway-type3 \
  --network host \
  -e GATEWAY_API_KEY=sk-test-inference-key \
  -e MARKETPLACE_URL=ws://localhost:8081 \
  -e MARKETPLACE_API_URL=http://localhost:8081 \
  -v gateway-type3-data:/app/data \
  langmart-gateway-type3-ui:latest

Code Reference

Image Detection Logic

Located in: cli-split-layout.tsx (lines 1701-1746)

The detectImageGeneration() function:

1. Checks if model has image_gen capability
2. Detects action and subject keywords
3. Extracts clean prompt using regex patterns
4. Returns detection result with cleaned prompt

Image Generation Function

Located in: cli-split-layout.tsx (lines 1748-1823)

The generateImage() function:

1. Calls /v1/images/generations API
2. Receives base64-encoded image data
3. Converts to terminal display format
4. Renders in chat interface
5. Saves to conversation history

Chat Message Handler

Located in: cli-split-layout.tsx (lines 1825-1860)

The sendChatMessage() function:

1. Checks for image generation intent first
2. If detected, calls generateImage() and returns
3. Otherwise, proceeds with chat completion API

Tips and Best Practices

For Best Image Results

1. Use descriptive, detailed prompts
2. Include style keywords (e.g., "photorealistic", "watercolor", "3D render")
3. Specify composition (e.g., "close-up", "wide angle", "aerial view")
4. Add lighting details (e.g., "golden hour", "studio lighting", "neon lights")

Example High-Quality Prompts

generate an image of a photorealistic sunset over snow-capped mountains,
golden hour lighting, wide angle view

create a picture of a cyberpunk street scene at night, neon signs,
rain-soaked streets, cinematic lighting

draw an illustration of a magical forest with glowing mushrooms,
fantasy art style, ethereal atmosphere

For Better Terminal Experience

1. Use a terminal with good Unicode support
2. Increase terminal font size for better image preview
3. Use a color-capable terminal (256-color or true-color)
4. Maximize terminal window for larger previews

Related Documentation

• Gateway Server: See main README sections above for server operations
• Tool Calling: See marketplace-tools.ts for available marketplace tools
• MCP Integration: See mcp-manager.ts for Model Context Protocol support
• Local Vault: See local-vault.ts for credential management
• Web Tools: See web-tools.ts for web search and fetch capabilities

WebSocket Protocol

Connection

ws://marketplace.api/gateway
Headers: {
  'Authorization': 'Bearer sk-...',
  'X-Gateway-ID': 'gw3-seller-001'
}

Message Types

• auth: Authentication handshake
• heartbeat: Keep-alive ping/pong
• request: Inference request from marketplace
• response: Streaming response to marketplace
• error: Error notification
• update: Version/configuration updates

Monitoring

The gateway emits events for monitoring:

• connected: Successfully connected to marketplace
• disconnected: Connection lost
• error: Error occurred
• request: Processing request
• request_complete: Request completed

Security

• TLS/SSL WebSocket connection
• API key authentication
• Request signing
• Rate limiting
• IP whitelisting (optional)

Dependencies

• ws: WebSocket client
• axios: HTTP client for provider APIs
• uuid: Request ID generation

Troubleshooting

Connection Issues

• Check firewall rules for WebSocket
• Verify marketplace URL
• Ensure API key is valid

Provider Errors

• Verify provider API keys
• Check rate limits
• Monitor provider status

Performance

• Monitor latency metrics
• Check network connectivity
• Consider geographic proximity