@malv/base-app

Framework Component Type: Build Tools & Development Infrastructure

Purpose

Build tools, development scripts, and local infrastructure server for MALV applications. This package provides everything needed to develop, build, and deploy MALV apps with automatic local infrastructure setup.

Architecture

@malv/base-app
├── Development Scripts
│   ├── malv-dev.js              # Multi-app batch coordinator
│   ├── malv-dev-cloudflare.js   # Single-app Cloudflare Workers dev
│   └── malv-dev-cloudflare-multi.js  # Multi-app orchestrator
│
├── Code Generators (15+)
│   ├── generate-all.js          # Run all generators
│   ├── generate-token-types.js  # TokenPayloads.ts from tokens.json
│   ├── generate-event-handler-types.js  # Event handler types
│   ├── generate-types.js        # Type definitions
│   └── ...
│
├── Infrastructure Server (Port 59459)
│   ├── /apps      # App metadata & renderers
│   ├── /search    # Semantic search with embeddings
│   ├── /token     # Token signing & verification
│   ├── /storage   # File-based storage operations
│   └── /queue     # Tool dispatch queue with batching
│
└── Build & Deploy
    ├── malv-build.js            # Production build
    ├── malv-deploy-cloudflare.js  # Deploy to Cloudflare
    └── setup-production.js      # Prepare production bundle

Development Workflow

Quick Start

# Start a single app in development mode
cd packages/malv-apps/my-app
yarn run dev

# Or use malv-dev directly
malv-dev --port 4557

# Start multiple apps (batched automatically)
# Terminal 1:
cd packages/malv-apps/auth && malv-dev
# Terminal 2 (within 4 seconds):
cd packages/malv-apps/conversation && malv-dev
# Terminal 3 (within 4 seconds):
cd packages/malv-apps/orchestrator && malv-dev

How Multi-App Batching Works

The malv-dev command implements a batch coordinator pattern that automatically groups multiple apps started within a 4-second window:

┌─────────────────────────────────────────────────────────────────┐
│                    Multi-App Batch Coordinator                  │
├─────────────────────────────────────────────────────────────────┤
│                                                                 │
│  Terminal 1: malv-dev     ──┐                                   │
│                             │                                   │
│  Terminal 2: malv-dev     ──┼──► Batch Coordinator              │
│                             │    (4-second window)              │
│  Terminal 3: malv-dev     ──┘           │                       │
│                                         ▼                       │
│                              malv-dev-cloudflare-multi.js       │
│                                         │                       │
│                    ┌────────────────────┼────────────────────┐  │
│                    ▼                    ▼                    ▼  │
│              Wrangler #1          Wrangler #2          Wrangler #3
│              (App 1)              (App 2)              (App 3)  │
│                                                                 │
│                    Shared Infrastructure Server                 │
│                         (Port 59459)                            │
└─────────────────────────────────────────────────────────────────┘

Coordination Flow:

1. Leader Election: First app to register becomes the "leader"
2. Batch Collection: Leader waits 4 seconds for more apps to join
3. Multi-App Spawn: Leader launches malv-dev-cloudflare-multi.js with all apps
4. Non-Leaders Exit: Other processes exit after multi-app process starts
5. Fallback: If multi-app process already running, new apps start in single-app mode

State File: ~/.malv/dev-coordinator.json

{
  "apps": [
    { "path": "/path/to/app1", "port": 4557, "envPath": null, "pid": 12345 },
    { "path": "/path/to/app2", "port": 4558, "envPath": null, "pid": 12346 }
  ],
  "leaderPid": 12345,
  "processRunning": false,
  "lastUpdate": 1234567890
}

Infrastructure Server

During local development, all infrastructure services are provided by a unified Node.js HTTP server on port 59459. This server is automatically started when you run any app with malv-dev or malv-dev-cloudflare.

Singleton Pattern

Only one infrastructure server runs at a time, regardless of how many apps are started:

App 1 starts → Acquires lock → Starts server (owner)
App 2 starts → Lock exists → Connects to existing server
App 3 starts → Lock exists → Connects to existing server

Lock File: ~/.malv/infrastructure-server.lock

Available Endpoints

| Endpoint | Method | Description | |----------|--------|-------------| | /health | GET | Server health check with running app count | | /apps | GET | List all running apps | | /apps/{appName}/tools.json | GET | Tool definitions for an app | | /apps/{appName}/tools-with-output-token/{tokenName}.json | GET | Tools that output a specific token | | /apps/{appName}/objects.json | GET | All object definitions for an app | | /apps/{appName}/objects/{objectType}.json | GET | Single object definition by type | | /apps/{appName}/events.json | GET | Event definitions | | /apps/{appName}/tokens.json | GET | Token definitions | | /apps/{appName}/web-renderers/{name}.js | GET | Bundled web renderer | | /apps/{appName}/objects/{type}/icons/{name}.js | GET | Bundled icon | | /search?q=<query> | GET | Semantic search across tools/objects | | /perception | GET | List all perception files from all running apps | | /perception/{appName} | GET | Get perception files for a specific app | | /token/sign-token | POST | Sign a JWT token | | /token/get-public-keys | POST | Get public keys for verification | | /storage | POST | Storage operations (read/write/list/delete) | | /storage/upload | PUT | Direct file upload | | /queue | POST | Queue tool for batched background execution | | /queue/status | GET | Queue status (pending items per app) |

Route Details

Apps Routes (/apps/*)

Serves app metadata and bundled assets from running apps:

# List running apps
curl http://localhost:59459/apps

# Get tools for an app
curl http://localhost:59459/apps/@malv/auth/tools.json

# Get tools that output a specific token (for prerequisite discovery)
curl http://localhost:59459/apps/@malv/gmail/tools-with-output-token/gmail_access.json

# Get a bundled web renderer
curl http://localhost:59459/apps/@malv/tables/web-renderers/table.js

Prerequisite Tool Discovery: The /tools-with-output-token/{tokenName}.json endpoint returns tools that have the specified token in their output_tokens. This is used by the orchestrator to find prerequisite tools - for example, when list_emails requires gmail_access token that the user doesn't have, the orchestrator queries this endpoint to discover login_gmail which can provide that token.

Bundling: Renderers are served from dist/standalone/ if pre-built, otherwise bundled on-demand from TypeScript source using Rollup.

Search Routes (/search)

Semantic search using HuggingFace embeddings (Xenova/bge-base-en-v1.5):

# Search for tools related to "email"
curl -H "X-App-ID: @malv/orchestrator" \
     -H "X-App-Secret: <secret>" \
     "http://localhost:59459/search?q=email&types=tools&limit=10"

# Search storage data with security key filtering
curl -H "X-App-ID: @malv/orchestrator" \
     -H "X-App-Secret: <secret>" \
     "http://localhost:59459/search?q=project%20goals&types=storage&securityKeys=[\"account:user123\"]"

Response:

{
  "query": "email",
  "types": ["tools"],
  "count": 3,
  "results": [
    {
      "type": "tools",
      "appName": "@malv/gmail",
      "appUrl": "http://localhost:4560",
      "name": "list_emails",
      "description": "Lists emails from Gmail inbox",
      "similarity": 0.92
    }
  ]
}

How it works:

1. Apps generate embeddings at build time (stored in embeddings/embeddings.json)
2. Search endpoint loads embeddings from all running apps
3. Creates embedding for query and calculates cosine similarity
4. Returns results sorted by similarity

Storage Search: In addition to tools and objects, the search endpoint supports semantic search over app storage data. Embeddings are automatically generated when apps write to storage, enabling AI-powered cross-app data discovery with permission-aware filtering via security keys. See SEMANTIC_STORAGE.md for complete documentation.

Perception Routes (/perception)

Fetch perception files that define how the AI should perceive users based on their tokens:

# Get all perception files from all running apps
curl http://localhost:59459/perception

# Get perception files for a specific app
curl http://localhost:59459/perception/@malv/research
# Or using short name:
curl http://localhost:59459/perception/research

Response (all apps):

{
  "@malv/research": [
    {
      "name": "create_project",
      "tokens": { "absent": { "@malv/research": "project" } },
      "perception": "Wants to create their first research project",
      "tasks": ["Guide user into creating a project"]
    },
    {
      "name": "guide_user",
      "tokens": { "exists": { "@malv/research": "project" } },
      "perception": "Working on a specific research project",
      "tasks": ["Edit goal, objective, questions of the research project"]
    }
  ]
}

Response (single app):

{
  "app": "@malv/research",
  "perceptions": [
    {
      "name": "create_project",
      "tokens": { "absent": { "@malv/research": "project" } },
      "perception": "Wants to create their first research project",
      "tasks": ["Guide user into creating a project"]
    }
  ]
}

Apps define perception files in perception/*.json. The orchestrator evaluates conditions against available tokens and storage data to determine which perceptions apply.

Supported conditions:

• tokens.exists / tokens.absent - Check token presence
• storage - Query actual storage values (e.g., check if a field is null or empty)

Storage condition example:

{
  "tokens": { "exists": { "@malv/research": "project" } },
  "storage": {
    "@malv/research": {
      "/teams/<project.teamId>/research/projects/<project.projectId>/config.json": {
        "objective": { "operator": "equals", "value": null }
      }
    }
  },
  "perception": "Wants us to set objective for research project",
  "tasks": ["Edit objective"]
}

Storage paths support <tokenType.field> templates resolved from token payloads. Operators include: equals, notEquals, exists, notExists, isEmpty, isNotEmpty.

Token Routes (/token/*)

Sign and verify JWT tokens with Ed25519 signatures:

# Sign a token
curl -X POST http://localhost:59459/token/sign-token \
  -H "Content-Type: application/json" \
  -d '{
    "appId": "@malv/auth",
    "appSecret": "<secret>",
    "type": "account",
    "payload": { "userId": "user-123", "team": "team-456" },
    "expiresIn": 3600
  }'

Token Structure:

{
  "appId": "@malv/auth",
  "type": "account",
  "iss": "malv-local-dev",
  "sub": "user-123",
  "iat": 1234567890,
  "exp": 1234571490,
  "aud": "storage",
  "storage_permissions": [
    { "type": "read", "prefix": "/teams/team-456/" },
    { "type": "write", "prefix": "/teams/team-456/" }
  ]
}

Storage Permissions: Permissions are extracted from the app's storage.json file at token signing time. See STORAGE_LOGIC.md for the complete permission embedding and validation flow.

Storage Routes (/storage)

File-based storage with token authentication:

# Write a file
curl -X POST http://localhost:59459/storage \
  -H "Content-Type: application/json" \
  -H "X-Provided-Tokens: [\"<signed-token>\"]" \
  -d '{
    "operation": "write",
    "path": "/teams/team-456/data.json",
    "data": {"key": "value"}
  }'

# Read a file
curl -X POST http://localhost:59459/storage \
  -H "X-Provided-Tokens: [\"<signed-token>\"]" \
  -d '{"operation": "read", "path": "/teams/team-456/data.json"}'

Permission System:

1. App Secret Auth: Apps with valid secret have full access to their own storage
2. Token Permissions: Check permissions embedded in signed tokens (from storage.json)
3. Cross-App Permissions: Validate against target app's storage.json definitions

For detailed information on how permissions are embedded in tokens and validated during storage access, see STORAGE_LOGIC.md.

App Registration & Health Monitoring

Running apps are tracked in a registry for discovery and health monitoring.

Registry File: ~/.malv/running-apps.json

{
  "@malv/auth": {
    "path": "/path/to/auth",
    "url": "http://localhost:4557",
    "healthy": 1234567890,
    "pid": 12345
  },
  "@malv/orchestrator": {
    "path": "/path/to/orchestrator",
    "url": "http://localhost:4558",
    "healthy": 1234567892,
    "pid": 12346
  }
}

Health Monitoring:

• Each app sends health checks every 30 seconds
• Apps with no health update for 60 seconds are considered stale
• Stale apps are automatically removed from registry
• PID verification prevents cross-process unregistration

Code Generators

The package includes 15+ code generators that create TypeScript types from declarative JSON schemas.

Available Generators

| Script | Input | Output | Description | |--------|-------|--------|-------------| | generate-all.js | - | - | Runs all generators | | generate-token-types.js | tokens.json | TokenPayloads.ts | Token payload types | | generate-event-handler-types.js | events.json | EventHandlerTypes.ts | Event handler types | | generate-types.js | tools.json | Type definitions | Tool input/output types | | generate-environment.js | environment.json | Environment.ts | Environment config types | | generate-exports.js | package.json | Export mappings | Package export configuration | | generate-icons.js | objects.json | Icon scaffolding | Object icon templates | | generate-object-renders.js | objects.json | Renderer scaffolding | Web renderer templates | | generate-object-types.js | objects.json | Object type definitions | Object metadata types |

Example: Token Type Generation

Input (tokens.json):

{
  "account": {
    "schema": {
      "type": "object",
      "properties": {
        "userId": { "type": "string" },
        "team": { "type": "string" },
        "role": { "type": "string", "enum": ["owner", "admin", "member"] }
      },
      "required": ["userId", "team", "role"]
    }
  }
}

Output (TokenPayloads.ts):

export type TokenPayloads = {
  account: {
    userId: string;
    team: string;
    role: "owner" | "admin" | "member";
  };
};

Running Generators

# Generate all types
yarn run generate

# Generate specific types
yarn run generate:tokens
yarn run generate:events
yarn run generate:objects

Build & Deploy Scripts

Development Scripts

| Script | Description | |--------|-------------| | malv-dev.js | Multi-app batch coordinator | | malv-dev-cloudflare.js | Single-app Cloudflare Workers development | | malv-dev-cloudflare-multi.js | Multi-app orchestrator | | malv-dev-cloudflare-setup.js | Setup Cloudflare environment | | malv-dev-server.js | Local development server |

Build Scripts

| Script | Description | |--------|-------------| | malv-build.js | Production build with Rollup | | malv-build-standalone.js | Standalone bundle build | | setup-production.js | Prepare production artifacts | | setup-cloudflare.js | Setup Cloudflare deployment |

Deploy Scripts

| Script | Description | |--------|-------------| | malv-deploy-cloudflare.js | Deploy to Cloudflare Workers | | publish-github.js | Publish to GitHub Packages | | publish-npmjs.js | Publish to npm registry |

CLI Commands

Apps typically expose these commands via package.json:

{
  "scripts": {
    "dev": "malv-dev",
    "dev:cloudflare": "malv-dev-cloudflare",
    "generate": "generate-all",
    "generate:tokens": "generate-token-types",
    "generate:events": "generate-event-handler-types",
    "build": "malv-build",
    "setup-production": "setup-production",
    "deploy": "malv-deploy-cloudflare"
  }
}

Command Options

malv-dev

malv-dev [options]

Options:
  --port <port>     Specify port (default: auto-allocate from 4557)
  --env, -e <path>  Custom .env file path

malv-dev-cloudflare-multi

malv-dev-cloudflare-multi <app-args...>

App argument format: path[:port][:env-path]

Examples:
  malv-dev-cloudflare-multi ./app1 ./app2
  malv-dev-cloudflare-multi ./app1:4557 ./app2:4558
  malv-dev-cloudflare-multi ./app1:4557:/path/.env ./app2::/other/.env

File Locations

System Files

| Path | Description | |------|-------------| | ~/.malv/ | MALV configuration directory | | ~/.malv/running-apps.json | Running apps registry | | ~/.malv/dev-coordinator.json | Multi-app batch coordinator state | | ~/.malv/infrastructure-server.lock | Infrastructure server lock file | | ~/.malv/secrets/{appName} | App secrets storage | | ~/.malv/cloudflare-state/ | Cloudflare persistent state |

Per-App Files

| Path | Description | |------|-------------| | tmp/cloudflare/ | Generated Cloudflare configuration | | tmp/cloudflare/wrangler.toml | Wrangler configuration | | .dev.vars | Environment variables for wrangler | | dist/standalone/ | Pre-built standalone bundles | | embeddings/embeddings.json | Semantic search embeddings |

App Initialization Flow

When starting an app with malv-dev-cloudflare, the following happens:

1. Validate Environment
   └─ Check required env vars, custom env path

2. Run Setup Script
   └─ malv-dev-cloudflare-setup.js
      └─ Generate wrangler.toml, worker entry point

3. Run Code Generators
   └─ generate-all.js
      ├─ TokenPayloads.ts
      ├─ EventHandlerTypes.ts
      └─ Environment.ts

4. Validate TypeScript
   └─ Check for syntax errors (exit on error)

5. Allocate Port
   └─ Use specified port or find available (starting 4557)

6. Build Standalone Bundles
   └─ Rollup build for renderers and icons

7. Generate App Secret
   └─ Create or retrieve from ~/.malv/secrets/{appName}

8. Write Environment
   └─ .dev.vars with SECRET, APPS_CDN_URL, SEARCH_URL

9. Start Wrangler
   └─ wrangler dev --config tmp/cloudflare/wrangler.toml

10. Register App
    └─ Add to ~/.malv/running-apps.json

11. Start Infrastructure Server
    └─ If not already running, start on port 59459

12. Start Health Monitoring
    └─ 30-second interval health checks

13. Setup File Watchers
    └─ Watch tools.json, objects.json, .env for changes

Wrangler Integration

Apps run as Cloudflare Workers using Wrangler in development mode.

Generated Configuration

tmp/cloudflare/wrangler.toml:

name = "malv-auth-dev"
main = "worker.ts"
compatibility_date = "2024-01-01"

[vars]
ENVIRONMENT = "development"

[[d1_databases]]
binding = "DB"
database_name = "malv-dev"
database_id = "local"

Worker Entry Point

tmp/cloudflare/worker.ts:

import { createWorker } from '@malv/base-app/cloudflare-worker';
import tools from '../../dist/tools.js';

export default createWorker({ tools });

Cloudflare Worker Template

The package includes a Cloudflare Worker template that:

• Handles tool execution requests
• Manages capabilities (storage, token, event)
• Streams responses via SSE
• Captures output tokens

Location: bin/cloudflare-worker/worker.ts

Request-Scoped Logging

The worker uses AsyncLocalStorage to provide request-scoped console.log capture. This prevents cross-request I/O errors when multiple requests run concurrently in the same Workers isolate.

Key files:

• toolExecutionContext.ts - AsyncLocalStorage instance for execution context
• setupConsoleCapture.ts - Global console.log override (imported first in worker.ts)

Each tool execution runs inside toolExecutionContext.run(), ensuring logs are routed to the correct request's SSE stream.

Dependencies

Runtime Dependencies

• miniflare - Local Cloudflare Workers runtime
• wrangler - Cloudflare Workers CLI
• rollup - Module bundler
• @rollup/plugin-typescript - TypeScript support
• @rollup/plugin-commonjs - CommonJS support
• chokidar - File watching

Peer Dependencies

• @malv/types - Type definitions
• @malv/shared - Shared utilities
• @malv/base - Core framework

Best Practices

1. Use Multi-App Mode: Start apps within 4 seconds to batch them together
2. Let Infrastructure Start Automatically: Don't manually start the infrastructure server
3. Run Generators After Schema Changes: Always run yarn generate after modifying tokens.json, events.json, or objects.json
4. Pre-Build Standalone Bundles: Run malv-build-standalone for faster renderer serving
5. Check Registry on Issues: If apps aren't discovered, check ~/.malv/running-apps.json
6. Clean State on Errors: Delete ~/.malv/ directory to reset all state

Troubleshooting

Infrastructure Server Won't Start

# Check if server is running
curl http://localhost:59459/health

# Remove stale lock file
rm ~/.malv/infrastructure-server.lock

# Restart apps

Apps Not Discovered

# Check running apps registry
cat ~/.malv/running-apps.json

# Check app health
curl http://localhost:4557/health

# Clean stale entries
# Restart all apps

Port Already in Use

# Find process using port
lsof -i :4557

# Kill process
kill <pid>

# Or specify different port
malv-dev --port 4560

Generator Errors

# Validate JSON schemas
cat tokens.json | jq .

# Run specific generator with debug
node bin/generate-token-types.js --debug

Related Documentation

• SEMANTIC_STORAGE.md - Semantic storage search: automatic embedding generation, security keys, and cross-app data discovery
• SEMANTIC_STORAGE_LOCATION_FILTERING.md - Location-based filtering for precise semantic storage searches
• STORAGE_LOGIC.md - Storage permission embedding and validation flow

Related Packages

• @malv/types - TypeScript type definitions
• @malv/shared - Shared utilities
• @malv/base - Core framework implementation
• @malv/dev - Development CLI tools
• @malv/infrastructure - Production infrastructure services