EspoCRM Researcher

An intelligent MCP (Model Context Protocol) server for EspoCRM research and exploration. Specializes in entity discovery, metadata analysis, and business heuristics management through Gists.

Table of Contents

• Overview
• Installation
• Configuration
• Core Features
• Available Tools
  • Entity Discovery Tools
  • Data Tools
  • Entity Tools
  • Modification Tools
  • Report Tools
  • Gist Tools
  • CRM-Specific Tools
• User Stories
• Architecture
• Development
• Publishing
• License

Overview

EspoCRM Researcher is a high-performance MCP server designed to handle massive EspoCRM instances with thousands of entities and complex metadata. It provides intelligent caching, progressive loading, and semantic search capabilities to make CRM exploration efficient and intuitive.

Key Capabilities

• Progressive Metadata Loading: Handles massive metadata files efficiently
• LRU Caching: Intelligent caching for frequently accessed entities
• Gist Management: Store and retrieve business heuristics and agent instructions
• Semantic Search: Natural language search across Gists
• ORM-Style Query Builder: Fluent interface for complex queries
• Relationship Traversal: Navigate entity relationships efficiently

Installation

Via NPX (Recommended)

npx espocrm-researcher

Global Installation

npm install -g espocrm-researcher

Configuration

Configure in your MCP client (e.g., Claude Desktop, Cline, Continue):

{
  "mcpServers": {
    "espocrm-researcher": {
      "command": "npx",
      "args": ["espocrm-researcher"],
      "env": {
        "ESPOCRM_BASE_URL": "https://crm.example.com",
        "ESPOCRM_API_KEY": "your_api_key_here",
        "ESPOCRM_USER_ID": "your_user_id_here"
      }
    }
  }
}

Optional Configuration

{
  "env": {
    "ESPOCRM_BASE_URL": "https://crm.example.com",
    "ESPOCRM_API_KEY": "your_api_key",
    "ESPOCRM_USER_ID": "your_user_id",
    "LOG_LEVEL": "info",              // debug, info, warn, error
    "CACHE_TTL": "3600",             // Cache TTL in seconds
    "CACHE_SIZE": "500",             // Max cache entries
    "METADATA_PROGRESSIVE_LIMIT": "50", // Entities per batch
    "ENABLE_SEMANTIC_SEARCH": "true",   // Enable AI search
    "MCP_DISABLE_LOGGING": "true"       // Disable all logging for MCP compatibility
  }
}

MySQL Configuration (Optional - Performance Boost)

For 10-100x performance improvement on read operations, you can enable direct MySQL access:

{
  "env": {
    // ... existing configuration ...
    "MYSQL_ENABLED": "true",
    "MYSQL_HOST": "localhost",
    "MYSQL_PORT": "3306",
    "MYSQL_USER": "espocrm_user",
    "MYSQL_PASSWORD": "your_mysql_password",
    "MYSQL_DATABASE": "espocrm_db",
    "MYSQL_SSL": "false",
    "ROUTING_MODE": "auto"  // auto, mysql, or api
  }
}

Benefits of MySQL Direct Access:

• Bypasses HTTP overhead for read operations
• Enables complex analytics queries
• Supports aggregations and time series analysis
• Automatic fallback to API on failure
• Connection pooling for optimal performance

Core Features

1. Progressive Metadata Loading

• Loads metadata in batches to handle massive CRMs
• Priority loading for frequently used entities
• Memory-efficient processing

2. Intelligent Caching

• LRU (Least Recently Used) cache for entity metadata
• Keyword indexing for fast lookups
• Automatic cache invalidation

3. Gist Management

• Store business rules and agent instructions
• Filter by Content Type and Status
• Version tracking and approval workflow

4. Semantic Search

• Natural language queries
• Relevance scoring
• Context-aware results

Available Tools

Entity Resolution Tools (New in v2.0.1)

These tools solve the common problem of finding entities when you don't know the exact name.

resolve_entity

Find the correct entity name using fuzzy matching. Perfect when you know roughly what entity you need but not the exact name.

Parameters:

• searchTerm: The entity name you're looking for (e.g., "PropertyUnit", "real estate")
• threshold (optional): Similarity threshold 0-1 (default: 0.6)
• maxResults (optional): Maximum results to return (default: 5)

Example:

// Looking for property-related entities
await resolve_entity({ searchTerm: "property unit" });
// Returns: { resolvedEntity: "RealEstateProperty", confidence: 0.85, ... }

// Looking for customer entities
await resolve_entity({ searchTerm: "customer" });
// Returns: { resolvedEntity: "Contact", confidence: 0.7, ... }

discover_all_entities

Discover all entities including custom ones, with categorization.

Parameters:

• includeFieldCount (optional): Include field count for each entity
• filter (optional): Filter entities by name pattern

Example:

await discover_all_entities({ includeFieldCount: true });
// Returns categorized entities: system, custom, extension

Entity Discovery Tools

discover_entities

Discovers available entities in the CRM with pagination support.

Parameters:

• page (optional): Page number for pagination
• limit (optional): Number of entities per page (default: 50)
• filter (optional): Filter entities by name pattern

Example:

await discover_entities({ page: 1, limit: 20, filter: "Contact" });

discover_fields

Discovers fields for a specific entity with detailed metadata.

Parameters:

• entityType: The entity type to discover fields for
• includeRelationships (optional): Include relationship fields

Example:

await discover_fields({ 
  entityType: "Contact", 
  includeRelationships: true 
});

analyze_entity_relationships

Analyzes relationships between entities.

Parameters:

• entityType: The entity type to analyze

Example:

await analyze_entity_relationships({ entityType: "Account" });

Data Tools

searchRecords

Search for records with advanced filtering and streaming support.

Parameters:

• entityType: Entity type to search
• searchTerm (optional): Search term
• filters (optional): Array of filter conditions
• orderBy (optional): Sort field
• order (optional): Sort direction (asc/desc)
• limit (optional): Maximum results
• offset (optional): Skip records
• fields (optional): Fields to return

Example:

await searchRecords({
  entityType: "Contact",
  searchTerm: "John",
  filters: [
    { field: "emailAddress", op: "isNotEmpty" },
    { field: "createdAt", op: "after", value: "2024-01-01" }
  ],
  orderBy: "createdAt",
  order: "desc",
  limit: 50
});

getRecord

Retrieve a single record by ID.

Parameters:

• entityType: Entity type
• id: Record ID

Example:

await getRecord({ 
  entityType: "Contact", 
  id: "507f1f77bcf86cd799439011" 
});

getRelated

Get related records with relationship traversal.

Parameters:

• entityType: Source entity type
• id: Source record ID
• link: Relationship name
• limit (optional): Maximum results

Example:

await getRelated({
  entityType: "Account",
  id: "507f1f77bcf86cd799439011",
  link: "contacts",
  limit: 100
});

Entity Tools

list_entities

List all available entities with basic metadata.

Example:

await list_entities();

get_entity_metadata

Get detailed metadata for a specific entity.

Parameters:

• entityType: Entity type

Example:

await get_entity_metadata({ entityType: "Opportunity" });

Modification Tools

createRecord

Create a new record.

Parameters:

• entityType: Entity type
• data: Record data

Example:

await createRecord({
  entityType: "Contact",
  data: {
    firstName: "John",
    lastName: "Doe",
    emailAddress: "[email protected]"
  }
});

updateRecord

Update an existing record.

Parameters:

• entityType: Entity type
• id: Record ID
• data: Fields to update

Example:

await updateRecord({
  entityType: "Contact",
  id: "507f1f77bcf86cd799439011",
  data: {
    status: "Active",
    lastContactedAt: new Date().toISOString()
  }
});

deleteRecord

Delete a record.

Parameters:

• entityType: Entity type
• id: Record ID

Example:

await deleteRecord({
  entityType: "Contact",
  id: "507f1f77bcf86cd799439011"
});

Report Tools

list_reports

List available reports in the CRM.

Example:

await list_reports();

execute_report

Execute a report and get results.

Parameters:

• reportId: Report ID
• format (optional): Output format (json, csv, excel)
• parameters (optional): Report parameters

Example:

await execute_report({
  reportId: "monthly-sales-report",
  format: "json",
  parameters: {
    startDate: "2024-01-01",
    endDate: "2024-01-31"
  }
});

Analytics Tools (MySQL Required)

These tools provide advanced analytics capabilities when MySQL is configured:

aggregate_analytics

Perform aggregated analytics on entity data.

Parameters:

• entity: Entity to analyze
• metrics: Array of metrics (count, sum, avg, min, max)
• groupBy (optional): Fields to group by
• filters (optional): Filter conditions
• timeRange (optional): Time range filter

Example:

await aggregate_analytics({
  entity: "Opportunity",
  metrics: [
    { type: "count", alias: "total_opportunities" },
    { type: "sum", field: "amount", alias: "total_revenue" },
    { type: "avg", field: "amount", alias: "average_deal_size" }
  ],
  groupBy: ["stage", "assignedUserId"],
  timeRange: {
    field: "createdAt",
    start: "2024-01-01",
    end: "2024-12-31"
  }
});

time_series_analytics

Analyze data over time intervals.

Parameters:

• entity: Entity to analyze
• interval: Time interval (hour, day, week, month, quarter, year)
• dateField: Date field to use
• metrics: Metrics to calculate

Example:

await time_series_analytics({
  entity: "Lead",
  interval: "month",
  dateField: "createdAt",
  metrics: [
    { type: "count", alias: "new_leads" },
    { type: "sum", field: "opportunityAmount", alias: "potential_revenue" }
  ]
});

top_performers

Identify top performers by various metrics.

Parameters:

• entity: Entity to analyze
• groupBy: Field to group by (e.g., assignedUserId)
• metric: Performance metric
• limit: Number of top performers

Example:

await top_performers({
  entity: "Opportunity",
  groupBy: "assignedUserId",
  metric: { type: "sum", field: "amount" },
  limit: 10
});

Gist Tools

search_gists

Search for Gists with filtering and relevance scoring.

Parameters:

• query (optional): Search query
• contentType (optional): Filter by content type
• status (optional): Filter by status
• operation (optional): Filter by operation type
• limit (optional): Maximum results

Example:

await search_gists({
  query: "customer onboarding",
  contentType: "Agent Gist",
  status: "Approved",
  limit: 10
});

get_gist

Retrieve a specific Gist by ID.

Parameters:

• id: Gist ID

Example:

await get_gist({ id: "6814f5d6c53bdb53d" });

create_gist

Create a new Gist for storing business heuristics.

Parameters:

• name: Gist name
• content: Gist content
• contentType: Content type (e.g., "Agent Gist")
• operation (optional): Operation type
• tags (optional): Array of tags

Example:

await create_gist({
  name: "Customer Onboarding Process",
  content: "1. Verify email\n2. Complete profile\n3. Schedule demo",
  contentType: "Agent Gist",
  operation: "onboarding",
  tags: ["customer", "onboarding", "process"]
});

semantic_search_gists

Search Gists using natural language queries.

Parameters:

• query: Natural language search query
• limit (optional): Maximum results
• threshold (optional): Minimum relevance score (0-1)

Example:

await semantic_search_gists({
  query: "How do we handle customer complaints?",
  limit: 5,
  threshold: 0.7
});

CRM-Specific Tools

get_user_info

Get current user information and permissions.

Example:

await get_user_info();

get_global_search

Perform a global search across all entities.

Parameters:

• q: Search query

Example:

await get_global_search({ q: "[email protected]" });

User Stories

Story 1: Business Analyst - Entity Discovery

As a Business Analyst
I want to discover all entities and their relationships in our CRM
So that I can understand the data model and plan integrations

// Discover all entities
const entities = await discover_entities({ limit: 100 });

// Analyze a specific entity
const contactFields = await discover_fields({ 
  entityType: "Contact",
  includeRelationships: true 
});

// Understand relationships
const relationships = await analyze_entity_relationships({ 
  entityType: "Contact" 
});

Story 2: AI Agent - Learning Business Rules

As an AI Agent
I want to access approved business heuristics
So that I can follow company-specific processes

// Find all approved agent instructions
const gists = await search_gists({
  contentType: "Agent Gist",
  status: "Approved"
});

// Search for specific process
const onboarding = await semantic_search_gists({
  query: "customer onboarding process",
  threshold: 0.8
});

Story 3: Developer - Data Migration

As a Developer
I want to efficiently query and export CRM data
So that I can migrate to a new system

// Stream all contacts with email
const contacts = await searchRecords({
  entityType: "Contact",
  filters: [{ field: "emailAddress", op: "isNotEmpty" }],
  fields: ["id", "firstName", "lastName", "emailAddress"]
});

// Get related data
for (const contact of contacts) {
  const opportunities = await getRelated({
    entityType: "Contact",
    id: contact.id,
    link: "opportunities"
  });
}

Story 4: Support Agent - Customer Research

As a Support Agent
I want to quickly find all information about a customer
So that I can provide informed support

// Global search for customer
const results = await get_global_search({ 
  q: "[email protected]" 
});

// Get detailed contact info
const contact = await getRecord({
  entityType: "Contact",
  id: results.Contact[0].id
});

// Get all related records
const accounts = await getRelated({
  entityType: "Contact",
  id: contact.id,
  link: "accounts"
});

Story 5: Manager - Report Generation

As a Manager
I want to execute and export reports
So that I can analyze business performance

// List available reports
const reports = await list_reports();

// Execute monthly sales report
const salesData = await execute_report({
  reportId: "monthly-sales-report",
  format: "json",
  parameters: {
    startDate: "2024-01-01",
    endDate: "2024-01-31"
  }
});

Architecture

Service Layer

1. MetadataIndexService

   • Progressive loading of entity metadata
   • LRU caching with configurable size
   • Priority-based entity loading

2. GistCacheService

   • In-memory caching of frequently accessed Gists
   • Keyword indexing for fast lookups
   • Automatic cache invalidation

3. GistSemanticSearchService

   • Natural language processing
   • TF-IDF relevance scoring
   • Context-aware search results

4. QueryBuilder

   • Fluent ORM-style interface
   • Complex query construction
   • Automatic pagination

5. SearchOptimizer

   • Streaming search results
   • Efficient filtering
   • Memory-conscious processing

Performance Optimizations

• Lazy Loading: Metadata loaded on-demand
• Batch Processing: Efficient API calls
• Stream Processing: Handle large datasets
• Smart Caching: Frequently used data cached
• Connection Pooling: Reused HTTP connections

Examples

See the examples/ directory for comprehensive examples:

• gist-usage-example.ts - Working with Gists
• create-record-example.ts - Creating various record types with relationships

Development

Setup

# Clone the repository
git clone https://github.com/CG-Labs/EspoCRM-Researcher.git
cd espocrm-researcher

# Install dependencies
npm install

# Run in development mode
npm run dev

Testing

# Run all tests
npm test

# Run with coverage
npm run test:coverage

Building

# Build for production
npm run build

# Lint code
npm run lint

# Format code
npm run format

Publishing

To publish to npm:

# Login to npm
npm login

# Ensure you have access to @cg-labs
npm org ls @cg-labs

# Publish
npm publish --access public

License

MIT License - see LICENSE file for details.

Repository: https://github.com/CG-Labs/EspoCRM-Researcher
NPM Package: https://www.npmjs.com/package/espocrm-researcher
Author: CG Labs