💡 What is TERMES?

TERMES is the autonomous web digesting, headless automation, and synthetic API engine of the Terra Ecosystem (inspired by termite mound biology and symbiotic digestion).

It allows developers to turn ANY website without an API, e-commerce store, legacy portal, SPA, or document into a live Inverted API and Site-to-Webhook event trigger with 100% Privacy & Security and $0 Monthly Server Overhead.

[!IMPORTANT] No Git Cloning Needed! The core Terra monorepo is private. TERMES is distributed globally over the internet as an open-access NPM package (terra-termes) and live Web Console. You do NOT need to clone any repository to use TERMES in your terminal or applications.

⚡ How TERMES Works (The Inverted API Concept)

In traditional web development, APIs are published and controlled by target service providers (who often paywall or restrict them).

TERMES Inverted APIs invert this relationship:

                            🏰 TERMES CONSOLE / TERMITARIUM
                                   (Control Plane)
                                          │
                ┌─────────────────────────┼─────────────────────────┐
                ▼                         ▼                         ▼
        👃 NASUTE WORKERS        🕳️ MUD TUNNEL PROXY        🪵 CELLULOSE PROCESSOR
      (Headless Execution)      (Stealth & Anti-Bot)          (HTML/DOM Cleaning)
                │                         │                         │
                └─────────────────────────┼─────────────────────────┘
                                          ▼
                               🦠 PROTOZOA ENGINE
                      (Parser CSS/XPath & Schema Builder)
                                          │
                ┌─────────────────────────┴─────────────────────────┐
                ▼                                                   ▼
       🌐 SYNTHETIC REST API                                 🦟 ALATES SWARM
     (Termitomyces - 0ms CDN)                             (Inverted Webhooks Engine)
                │                                                   │
                └─────────────────────────┬─────────────────────────┘
                                          ▼
                                 🔄 TROPHALLAXIS
          (Feeds to Combase/Rolla & AWS/Azure/GCP Multi-Cloud Bridges)

1. Declare Spec (Termitarium 🏰): Define CSS selectors, XPath rules, or regex patterns for the target data you want to extract from any website in the world.
2. Stealth Navigation (Mud Tunnel 🕳️ & Nasute Workers 👃): TERMES navigates target sites using headless runners with anti-bot bypass, stealth headers, and proxy rotation.
3. Digest Raw Content (Cellulose Processor 🪵 & Protozoa Engine 🦠): Cleans raw HTML/DOM trees and digests unstructured cellulose into clean JSON schemas.
4. Publish Synthetic REST API (Termitomyces 🍄): Cultivates and publishes live Synthetic REST APIs on global CDN (GitHub Pages / Raw CDN) with 0ms server delay and $0 cost.
5. Site-to-Webhook (Alates Swarm 🦟): Converts passive websites into active Inverted Webhooks. TERMES monitors DOM content changes and dispatches instant POST alerts to your app, Discord, or Slack.
6. Multi-Cloud Data Feeds (Trophallaxis 🔄): Streams digested data directly to Terra Titans (Combase, Rolla, Lumina) and Cloud Providers (AWS S3, Azure Blob, GCP Storage).

🏛️ Ecosystem Core Concepts & Terminology

| Biological Metaphor | Module Name | Technical Function | | :--- | :--- | :--- | | 🏰 Termitarium | Extraction Spec Engine | Control panel & recipe manager storing target URLs and CSS selectors. | | 👃 Nasute Workers | Headless Execution | Autonomous web runners executing extraction tasks. | | 🕳️ Mud Tunnel | Stealth Layer | User-Agent rotation, stealth proxying, and Anti-Bot bypass. | | 🪵 Cellulose | DOM Content | Unstructured raw HTML, JavaScript DOM trees, or document tables. | | 🦠 Protozoa | Symbiotic Parser | Schema builder converting raw DOM nodes into structured JSON objects. | | 🍄 Termitomyces | Synthetic APIs | Live public or private REST endpoints served from global CDNs at 0ms. | | 🦟 Alates Swarm | Inverted Webhooks | Site-to-Webhook engine firing HTTP alerts when target DOM varies. | | 🔄 Trophallaxis | Multi-Cloud Bridges | Data exchange pipelines feeding Combase, Rolla, AWS, Azure, and GCP. |

📦 Global Installation & Quick Start

Installing TERMES requires only Node.js 18+ installed on your machine.

Option 1: Global CLI Installation (Recommended)

Install terra-termes globally via NPM:

# Install globally from NPM
npm install -g terra-termes

# Verify installation
termes --version

Option 2: Instant NPX Execution (No Setup)

Run TERMES CLI directly without permanent installation:

# Launch live web console directly
npx terra-termes console

# Create an extraction spec
npx terra-termes spec list

🔑 Authentication Setup

TERMES uses a GitHub Personal Access Token (PAT) with repo permissions to publish 0ms Synthetic APIs and maintain state at $0 cost.

Set your token as an environment variable in your terminal:

# On Linux / macOS
export GITHUB_TOKEN="ghp_your_github_personal_access_token"

# On Windows PowerShell
$env:GITHUB_TOKEN="ghp_your_github_personal_access_token"

💻 CLI Commands Reference

🌐 Abrir Consola Web Local (Offline en Localhost)

# Abrir consola web local en puerto por defecto (http://localhost:3720)
termes console

# O en puerto personalizado:
termes studio --port 4000

Inicia un servidor HTTP local en http://localhost:3720 (o puerto personalizado) para administrar TERMES de forma 100% privada sin depender de Internet. Si el puerto está ocupado, detecta automáticamente el siguiente disponible.

🏰 Termitarium Extraction Specs & Inverted APIs

# 1. Create a new Inverted API Spec
termes spec create --name "tracker-precios" --url "https://tienda.com/producto" --selectors '{"precio":".price-tag","titulo":"h1"}'

# 2. List all active extraction specs
termes spec list

# 3. Digest a spec and publish live Synthetic API
termes spec digest --id spec_xyz123

# 4. Delete a spec
termes spec delete --id spec_xyz123

🦟 Alates Swarm — Inverted Webhooks (Site-to-Webhook)

# 1. Create an Inverted Webhook trigger
termes webhook create --name "Alerta Cambio Precio" --url "https://mi-app.com/webhook" --condition on_change

# 2. List active webhooks
termes webhook list

# 3. Delete a webhook
termes webhook delete --id wh_xyz123

🔄 Trophallaxis — Multi-Cloud Bridges & Mapeador de Campos

# 1. Crear un puente Multi-Cloud con Mapeador de Campos JSON
termes bridge create \
  --name "Sync Combase Events" \
  --type terra_combase \
  --repo "https://github.com/amglogicalis/combase-storage" \
  --target "sandbox_events" \
  --spec "spec_xyz123" \
  --mapper '{"precio":"price_eur","titulo":"product_title"}'

# 2. Listar puentes activos y sus destinos
termes bridge list

# 3. Probar un puente mediante Simulación Dry-Run (sin enviar datos reales)
termes bridge simulate --id bridge_xyz123

# 4. Ver histórico y logs de auditoría de un puente
termes bridge logs --id bridge_xyz123

# 5. Actualizar configuración de un puente existente
termes bridge update --id bridge_xyz123 --target "eventos_v2"

# 6. Eliminar un puente
termes bridge delete --id bridge_xyz123

🛠️ Node.js & TypeScript SDK Usage

You can import terra-termes directly into any Node.js, Next.js, Express, or TypeScript project:

import { Termes } from 'terra-termes';

// Initialize TERMES SDK
const termes = new Termes({
  githubToken: process.env.GITHUB_TOKEN!
});

// Load state from Vault
await termes.init();

// 1. Create an Inverted API Spec
const spec = await termes.createSpec(
  'laptop-price-tracker',
  'https://store.com/laptops',
  {
    title: 'h1.product-title',
    price: '.product-price'
  },
  {
    description: 'E-commerce price tracker',
    cdnRepo: 'termes-repo-public',
    apiIsPrivate: false
  }
);

// 2. Create a Trophallaxis Multi-Cloud Bridge with Field Mapper
const bridge = await termes.createTrophallaxisBridge(
  'Terra Combase Storage Bridge',
  'terra_combase',
  {
    sourceSpecId: spec.specId,
    repoUrl: 'https://github.com/amglogicalis/combase-storage',
    targetName: 'laptop_prices',
    fieldMapper: {
      price: 'price_eur',
      title: 'product_name'
    }
  }
);

// 3. Test Bridge Dry-Run Simulation
const simulation = termes.simulateBridge(bridge.targetId, {
  title: 'Gaming Laptop 16"',
  price: '1299.99'
});
console.log('🧪 Simulated Mapped Data:', simulation.mappedData);

// 4. Digest target URL and publish Synthetic API to CDN + feed bridges automatically
const { result, cdnUrl } = await termes.digestSpec(spec.specId);

console.log('✔ Digested in:', result.durationMs, 'ms');
console.log('🌐 Live Inverted Synthetic API URL (0ms CDN):', cdnUrl);
console.log('📦 Extracted Data:', result.data);

// 5. Create an Inverted Webhook (Site-to-Webhook)
const webhook = await termes.createInvertedWebhook(
  'Price Change Trigger',
  'https://myapp.com/api/webhooks',
  'on_change'
);

🌐 Live Online Console

Access the official TERMES Web Console hosted on GitHub Pages: 👉 https://amglogicalis.github.io/termes-repo-public/