mrmd-sync

Real-time sync server for collaborative markdown editing. Connects browsers to your filesystem with bidirectional sync via Yjs CRDTs.

Browser ←──WebSocket──→ mrmd-sync ←──→ ./docs/*.md
   │                        │
   └── Real-time collab ────┘

Quick Start

# Start syncing a directory
npx mrmd-sync ./docs

That's it. Open ws://localhost:4444/readme in mrmd-editor and start editing. Changes sync to ./docs/readme.md.

Table of Contents

• Installation
• CLI Usage
• Programmatic Usage
• Examples
  • Basic File Sync
  • With Authentication
  • Express Integration
  • Multiple Directories
• Configuration
• HTTP Endpoints
• API Reference
• How It Works
• Security
• Operational Notes
• Troubleshooting

Installation

# Global install
npm install -g mrmd-sync

# Or use npx (no install)
npx mrmd-sync ./docs

# Or add to project
npm install mrmd-sync

CLI Usage

Basic

# Sync current directory's docs folder
mrmd-sync ./docs

# Custom port
mrmd-sync --port 8080 ./docs

# Short flag
mrmd-sync -p 8080 ./docs

All Options

mrmd-sync [options] [directory]

Options:
  --port, -p <port>           WebSocket port (default: 4444)
  --debounce <ms>             Write debounce delay (default: 1000)
  --max-connections <n>       Max total connections (default: 100)
  --max-per-doc <n>           Max connections per document (default: 50)
  --max-file-size <bytes>     Max file size to sync (default: 10485760)
  --ping-interval <ms>        Heartbeat interval (default: 30000)
  --cleanup-delay <ms>        Doc cleanup after disconnect (default: 60000)
  --i-know-what-i-am-doing    Allow syncing system paths (/, /etc, /home)
  --help, -h                  Show help

Example: Production Settings

mrmd-sync \
  --port 443 \
  --max-connections 500 \
  --max-per-doc 100 \
  --debounce 2000 \
  ./production-docs

Programmatic Usage

import { createServer } from 'mrmd-sync';

const server = createServer({
  dir: './docs',
  port: 4444,
});

// Server is now running
console.log('Sync server started');

// Graceful shutdown
process.on('SIGINT', () => {
  server.close();
});

Examples

Basic File Sync

The simplest setup - sync a folder:

import { createServer } from 'mrmd-sync';

// Start server
const server = createServer({
  dir: './my-notes',
  port: 4444,
});

// That's it! Connect from browser:
// ws://localhost:4444/meeting-notes  →  ./my-notes/meeting-notes.md
// ws://localhost:4444/todo           →  ./my-notes/todo.md
// ws://localhost:4444/journal/2024   →  ./my-notes/journal/2024.md

Browser side (with mrmd-editor):

import mrmd from 'mrmd-editor';

// Connect to sync server
const drive = mrmd.drive('ws://localhost:4444');

// Open a document
const editor = drive.open('meeting-notes', '#editor');

// Everything typed syncs automatically!

With Authentication

Protect documents with custom auth:

import { createServer } from 'mrmd-sync';
import jwt from 'jsonwebtoken';

const server = createServer({
  dir: './docs',
  port: 4444,

  // Auth receives the HTTP request and document name
  auth: async (req, docName) => {
    // Get token from query string: ws://localhost:4444/doc?token=xxx
    const url = new URL(req.url, 'http://localhost');
    const token = url.searchParams.get('token');

    if (!token) {
      return false; // Reject connection
    }

    try {
      const payload = jwt.verify(token, process.env.JWT_SECRET);

      // Check document access
      if (docName.startsWith('private/') && !payload.isAdmin) {
        return false;
      }

      return true; // Allow connection
    } catch {
      return false; // Invalid token
    }
  },
});

Browser side:

const token = await getAuthToken();
const drive = mrmd.drive(`ws://localhost:4444?token=${token}`);

Express Integration

Run alongside an Express server:

import express from 'express';
import { createServer } from 'mrmd-sync';

const app = express();

// Your REST API
app.get('/api/documents', (req, res) => {
  res.json({ documents: ['readme', 'notes', 'todo'] });
});

// Start Express
const httpServer = app.listen(3000);

// Start sync server on different port
const syncServer = createServer({
  dir: './docs',
  port: 4444,
  auth: async (req, docName) => {
    // Validate session cookie
    const sessionId = parseCookie(req.headers.cookie)?.session;
    return await validateSession(sessionId);
  },
});

console.log('API: http://localhost:3000');
console.log('Sync: ws://localhost:4444');

Multiple Directories

Run multiple sync servers for different purposes:

import { createServer } from 'mrmd-sync';

// Public docs - no auth, read-heavy
const publicDocs = createServer({
  dir: './public-docs',
  port: 4444,
  maxConnections: 200,
  maxConnectionsPerDoc: 100,
});

// Private workspace - auth required, smaller limits
const privateDocs = createServer({
  dir: './private-docs',
  port: 4445,
  maxConnections: 20,
  maxConnectionsPerDoc: 5,
  auth: async (req, docName) => {
    return checkAuth(req);
  },
});

// Team collaboration - balanced settings
const teamDocs = createServer({
  dir: './team-docs',
  port: 4446,
  maxConnections: 50,
  maxConnectionsPerDoc: 20,
  debounceMs: 500, // Faster saves for active collab
});

Monitoring & Stats

Get real-time statistics:

import { createServer } from 'mrmd-sync';

const server = createServer({ dir: './docs', port: 4444 });

// Check stats periodically
setInterval(() => {
  const stats = server.getStats();
  console.log(`Connections: ${stats.totalConnections}`);
  console.log(`Active docs: ${stats.totalDocs}`);

  // Per-document breakdown
  stats.docs.forEach(doc => {
    console.log(`  ${doc.name}: ${doc.connections} clients`);
  });
}, 10000);

Example output:

Connections: 12
Active docs: 3
  readme: 5 clients
  meeting-notes: 4 clients
  todo: 3 clients

Custom File Handling

Access the underlying Yjs documents:

import { createServer } from 'mrmd-sync';

const server = createServer({ dir: './docs', port: 4444 });

// Get a specific document
const doc = server.getDoc('readme');

// Access Yjs Y.Text
const content = doc.ytext.toString();
console.log('Current content:', content);

// Watch for changes
doc.ydoc.on('update', () => {
  console.log('Document updated!');
  console.log('New content:', doc.ytext.toString());
});

// Programmatically edit (syncs to all clients!)
doc.ytext.insert(0, '# Hello\n\n');

Subdirectory Support

Organize documents in folders:

const server = createServer({ dir: './docs', port: 4444 });

// These all work:
// ws://localhost:4444/readme        →  ./docs/readme.md
// ws://localhost:4444/notes/daily   →  ./docs/notes/daily.md
// ws://localhost:4444/2024/jan/01   →  ./docs/2024/jan/01.md

Browser side:

const drive = mrmd.drive('ws://localhost:4444');

// Open nested documents
drive.open('notes/daily', '#editor1');
drive.open('2024/jan/01', '#editor2');

Configuration

Full Options Reference

createServer({
  // === Directory & Port ===
  dir: './docs',           // Base directory for .md files
  port: 4444,              // WebSocket server port

  // === Authentication ===
  auth: async (req, docName) => {
    // req: HTTP upgrade request (has headers, url, etc.)
    // docName: requested document name
    // Return true to allow, false to reject
    return true;
  },

  // === Performance ===
  debounceMs: 1000,        // Delay before writing to disk (ms)
                           // Lower = faster saves, more disk I/O
                           // Higher = batched writes, less I/O

  // === Limits ===
  maxConnections: 100,     // Total WebSocket connections allowed
  maxConnectionsPerDoc: 50,// Connections per document
  maxMessageSize: 1048576, // Max WebSocket message (1MB)
  maxFileSize: 10485760,   // Max file size to sync (10MB)

  // === Timeouts ===
  pingIntervalMs: 30000,   // Heartbeat ping interval (30s)
                           // Detects dead connections
  docCleanupDelayMs: 60000,// Cleanup delay after last disconnect (60s)
                           // Keeps doc in memory briefly for reconnects

  // === Security ===
  dangerouslyAllowSystemPaths: false,  // Must be true for /, /etc, /home, etc.
});

Recommended Settings by Use Case

Local Development:

{
  dir: './docs',
  port: 4444,
  debounceMs: 500,        // Fast feedback
  docCleanupDelayMs: 5000, // Quick cleanup
}

Team Collaboration:

{
  dir: './team-docs',
  port: 4444,
  maxConnections: 50,
  maxConnectionsPerDoc: 20,
  debounceMs: 1000,
  auth: validateTeamMember,
}

Public Documentation:

{
  dir: './public-docs',
  port: 4444,
  maxConnections: 500,
  maxConnectionsPerDoc: 200,
  debounceMs: 2000,        // Reduce write load
  maxFileSize: 1048576,    // 1MB limit
}

HTTP Endpoints

The sync server exposes HTTP endpoints for monitoring and health checks.

Note: These endpoints are unauthenticated by design - they're intended for internal monitoring (load balancers, Kubernetes probes, Prometheus, etc.). If you need to protect them, put a reverse proxy in front.

GET /health or GET /healthz

Health check for load balancers and orchestrators.

curl http://localhost:4444/health

{
  "status": "healthy",
  "shutting_down": false
}

Returns 200 when healthy, 503 when shutting down.

GET /metrics

Server metrics in JSON format.

curl http://localhost:4444/metrics

{
  "uptime": 3600,
  "connections": {
    "total": 150,
    "active": 12
  },
  "messages": {
    "total": 45230,
    "bytesIn": 1048576,
    "bytesOut": 2097152
  },
  "files": {
    "saves": 89,
    "loads": 23
  },
  "errors": 0,
  "lastActivity": "2024-01-15T10:30:00.000Z"
}

GET /stats

Detailed statistics including per-document breakdown.

curl http://localhost:4444/stats

{
  "uptime": 3600,
  "connections": { "total": 150, "active": 12 },
  "documents": [
    { "name": "readme", "connections": 5, "path": "/docs/readme.md" },
    { "name": "notes/daily", "connections": 7, "path": "/docs/notes/daily.md" }
  ],
  "config": {
    "port": 4444,
    "dir": "/docs",
    "debounceMs": 1000,
    "maxConnections": 100,
    "maxConnectionsPerDoc": 50
  }
}

API Reference

createServer(options)

Creates and starts a sync server.

Parameters:

| Option | Type | Default | Description | |--------|------|---------|-------------| | dir | string | './docs' | Directory to sync | | port | number | 4444 | WebSocket port | | auth | function | null | Auth handler (req, docName) => boolean \| Promise<boolean> | | debounceMs | number | 1000 | Write debounce delay | | maxConnections | number | 100 | Max total connections | | maxConnectionsPerDoc | number | 50 | Max connections per doc | | maxMessageSize | number | 1048576 | Max message size (bytes) | | maxFileSize | number | 10485760 | Max file size (bytes) | | pingIntervalMs | number | 30000 | Heartbeat interval | | docCleanupDelayMs | number | 60000 | Cleanup delay | | dangerouslyAllowSystemPaths | boolean | false | Allow syncing /, /etc, /home, etc. |

Returns: Server object

Server Object

const server = createServer(options);

server.getDoc(name)

Get or create a document by name.

const doc = server.getDoc('readme');
// doc.ydoc - Y.Doc instance
// doc.ytext - Y.Text instance
// doc.awareness - Awareness instance
// doc.conns - Set of WebSocket connections
// doc.filePath - Path to .md file

server.getStats()

Get current statistics.

const stats = server.getStats();
// {
//   totalConnections: 12,
//   totalDocs: 3,
//   docs: [
//     { name: 'readme', connections: 5 },
//     { name: 'notes', connections: 7 },
//   ]
// }

server.close()

Gracefully shutdown the server.

server.close();
// - Closes all WebSocket connections
// - Stops file watcher
// - Cleans up Y.Doc instances

server.config

Access the resolved configuration.

console.log(server.config);
// { dir: './docs', port: 4444, debounceMs: 1000, ... }

server.wss

Access the underlying WebSocket server (ws library).

server.wss.clients.forEach(client => {
  console.log('Client connected');
});

server.docs

Access the Map of loaded documents.

server.docs.forEach((doc, name) => {
  console.log(`${name}: ${doc.ytext.length} chars`);
});

How It Works

Architecture

┌─────────────────────────────────────────────────────────┐
│                      mrmd-sync                          │
├─────────────────────────────────────────────────────────┤
│                                                         │
│   Browser A ──┐                                         │
│               │     ┌──────────┐     ┌──────────────┐  │
│   Browser B ──┼────►│  Y.Doc   │◄───►│  readme.md   │  │
│               │     │  (CRDT)  │     │  (on disk)   │  │
│   Browser C ──┘     └──────────┘     └──────────────┘  │
│                           │                             │
│                     Yjs Protocol                        │
│                     - Sync updates                      │
│                     - Awareness (cursors)               │
│                                                         │
└─────────────────────────────────────────────────────────┘

Sync Flow

Browser → File:

1. User types in browser
2. mrmd-editor sends Yjs update via WebSocket
3. Server applies update to Y.Doc
4. Server broadcasts to other clients
5. Debounced write to .md file

File → Browser:

1. External edit (VS Code, git pull, etc.)
2. Chokidar detects file change
3. Server reads new content
4. Character-level diff computed
5. Diff applied to Y.Doc
6. Update broadcast to all browsers

Why Yjs?

Yjs is a CRDT (Conflict-free Replicated Data Type) library that:

• Never loses data - Concurrent edits merge automatically
• Works offline - Changes sync when reconnected
• Character-level sync - Only changed characters transmitted
• Proven - Used by Notion, Linear, and others

Security

System Path Protection

By default, mrmd-sync refuses to sync system directories like /, /etc, /home, /var, etc. This prevents accidentally exposing sensitive system files over WebSocket.

# This will be REJECTED:
mrmd-sync /
mrmd-sync /home
mrmd-sync /etc/myapp

Error: Refusing to sync dangerous system path: "/"

Syncing system directories (/, /etc, /home, etc.) can expose sensitive files
and allow remote file creation anywhere on your system.

When You Actually Need System Access

If you're building a personal file server, NAS interface, or system admin tool, you can explicitly opt-in:

CLI:

# The flag name makes you think twice
mrmd-sync --i-know-what-i-am-doing /

# Or the longer version
mrmd-sync --dangerous-allow-system-paths /home

Programmatic:

import { createServer } from 'mrmd-sync';

const server = createServer({
  dir: '/',
  dangerouslyAllowSystemPaths: true,  // Required for /, /etc, /home, etc.

  // STRONGLY RECOMMENDED: Add authentication!
  auth: async (req, docName) => {
    const token = getTokenFromRequest(req);
    return validateToken(token);
  },
});

What's Considered Dangerous?

These paths require explicit opt-in:

| Path | Why It's Dangerous | |------|-------------------| | / | Access to entire filesystem | | /etc | System configuration files | | /home | All users' home directories | | /var | System logs, databases | | /usr | System binaries | | /root | Root user's home | | /bin, /sbin | System executables |

Safe by default (no flag needed):

• /home/youruser/docs - User-specific subdirectory
• ./docs - Relative paths
• /srv/myapp/data - Application-specific paths

Best Practices for System Access

If you enable system path access:

1. Always use authentication

   createServer({
     dir: '/',
     dangerouslyAllowSystemPaths: true,
     auth: (req, docName) => validateUser(req),
   })

2. Run as unprivileged user

   # Don't run as root!
   sudo -u www-data mrmd-sync --i-know-what-i-am-doing /srv/files

3. Use a reverse proxy with TLS

   location /sync {
     proxy_pass http://localhost:4444;
     proxy_http_version 1.1;
     proxy_set_header Upgrade $http_upgrade;
     proxy_set_header Connection "upgrade";
   }

4. Consider read-only for sensitive paths

   auth: async (req, docName) => {
     // Block writes to sensitive directories
     if (docName.startsWith('etc/') || docName.startsWith('var/')) {
       return false;
     }
     return validateUser(req);
   }

Operational Notes

Instance Locking

mrmd-sync uses a PID file (.mrmd-sync/server.pid) to prevent multiple instances from running on the same directory. If you try to start a second instance:

Error: Another mrmd-sync instance is already running on this directory.

  PID: 12345
  Port: 4444
  Started: 2024-01-15T10:30:00.000Z

Stop the other instance first, or use a different directory.

If a previous instance crashed without cleanup, the stale PID file is automatically detected and removed.

Yjs Document Growth

Yjs stores operation history for undo/redo and conflict resolution. Long-running sessions with many edits will accumulate history, increasing memory usage. This is expected behavior - documents are cleaned up after clients disconnect (controlled by docCleanupDelayMs).

For high-traffic scenarios, consider:

• Lower docCleanupDelayMs to free memory faster
• Restart periodically during maintenance windows

External File Edits

When files are edited externally (VS Code, git pull, etc.):

1. Chokidar detects the change
2. A character-level diff is computed
3. Changes are applied to the Yjs document
4. All connected clients receive the update

This is intentional - the .md file is the source of truth. However, simultaneous external edits and browser edits may result in merged content.

Crash Recovery

If persistYjsState: true (default), Yjs snapshots are saved to .mrmd-sync/*.yjs. On restart:

1. Snapshot is loaded (contains recent edits)
2. File is read
3. If they differ, the file wins (it's the source of truth)

This protects against corrupt snapshots but means edits between the last file write and a crash may be lost. The debounceMs setting controls how quickly edits are persisted to the file.

Troubleshooting

Connection Rejected

"Max connections reached"

// Increase limit
createServer({ maxConnections: 200 })

"Invalid document name"

// Document names must:
// - Not contain ".."
// - Not start with "/" or "\"
// - Only contain: letters, numbers, dashes, underscores, dots, slashes
//
// Valid: readme, notes/daily, 2024-01-01, my_doc
// Invalid: ../secret, /etc/passwd, doc<script>

"Unauthorized"

// Check your auth function
auth: async (req, docName) => {
  console.log('Auth attempt:', docName, req.headers);
  // Make sure you return true for valid requests
  return true;
}

File Not Syncing

Check file path:

const doc = server.getDoc('readme');
console.log('File path:', doc.filePath);
// Should be: ./docs/readme.md

Check file permissions:

ls -la ./docs/
# Ensure writable

Check file size:

// Default max is 10MB
createServer({ maxFileSize: 50 * 1024 * 1024 }) // 50MB

High Memory Usage

Reduce cleanup delay:

createServer({
  docCleanupDelayMs: 10000, // 10s instead of 60s
})

Check for leaked connections:

setInterval(() => {
  const stats = server.getStats();
  console.log('Active docs:', stats.totalDocs);
  console.log('Connections:', stats.totalConnections);
}, 5000);

Slow Performance

Increase debounce for write-heavy workloads:

createServer({
  debounceMs: 2000, // 2 seconds
})

Reduce max file size:

createServer({
  maxFileSize: 1024 * 1024, // 1MB
})

License

MIT