@withmarfa/sync

Headless engine that keeps one or more local folders two-way synced with a Marfa server. Watches the disk, listens to the server's SSE stream, reconciles both directions with conflict detection.

The engine is UI-agnostic — a CLI ships in this package and drives the engine over a loopback HTTP control surface; menu-bar and desktop UIs use the same surface.

Install

npm install -g @withmarfa/sync

Quick start

marfa-sync start <folder> --url <api-url> --key <api-key>

Subsequent roots can be added at runtime:

marfa-sync roots add <path>
marfa-sync status

--daemon writes a launchd plist (macOS) so the engine survives terminal sessions. Run marfa-sync --help for the full command list.

Mapping rules

<root>/.marfa/mappings.json (optional) decides what type each file becomes when synced. Resolution order, first match wins:

1. Frontmatter type: if present (text files only)
2. User rules from mappings.json
3. Built-in mime defaults (image/* → core.file.image, audio, video)
4. Fallback: core.file

Example — promote any .md file in Notes/ to core.note, populating title from the first H1, body from everything after the frontmatter, and tags from the frontmatter:

{
  "rules": [
    {
      "id": "notes",
      "match": { "glob": "Notes/**/*.md" },
      "transform": {
        "type": "core.note",
        "tier": "library",
        "propertyExtractors": [
          { "kind": "first-h1", "into": "title" },
          { "kind": "body-after-frontmatter", "into": "body" },
          { "kind": "frontmatter-key", "key": "tags", "into": "tags" }
        ]
      }
    }
  ]
}

Match predicates: glob, mime, frontmatter (key/value match, or true for presence). All present predicates AND together.

Configuration

The agent merges configuration from three sources (highest priority first):

1. CLI flags — --url, --key, --types
2. Environment variables — MARFA_API_URL, MARFA_API_KEY
3. Config file — ~/.marfa/sync.json

The Marfa kit the agent talks through is pinned in package.json; a released build always ships against the kit version its lockfile resolves.

First run

On start, the engine reconciles the watched folder against the server in both directions:

• Push. Every pre-existing local file that resolves to a mapping (or carries a frontmatter type:) is enqueued and pushed, at any depth — the startup scan recurses the whole tree, matching the live watcher. Pushed items carry source_id = <root-relative path> (e.g. Notes/a.md), giving each a stable natural key, so a later push of the same file upserts rather than duplicating, and folder queries such as source_id starts_with "Notes/" resolve.

• Pull (default on). For a single-root engine, the server's items of the mapped type(s) are pulled down to disk. This is the right behavior for setting up a new device — "mirror my library down" — but it means pointing at a small folder against a populated space fills that folder with the server's whole note set. Pass --no-pull for a push-only first run: existing local files are still pushed, but nothing is pulled down. (Multi-root engines never pull server-only items, because per-root ownership of an untracked server item is ambiguous.)

# Back up an existing folder of notes without pulling the server library down:
marfa-sync start ./my-notes --no-pull

--no-pull is a per-invocation choice — it is not persisted to ~/.marfa/sync.json, so a --daemon respawn reverts to the pull-on default.

Upgrading

Notable releases ship migration notes in CHANGELOG.md. The most consequential ones to read before upgrading:

• 1.1.0 — makes delete and rename intent durable across crashes and outages, and makes all engine-written local files atomic. The SQLite schema upgrade is automatic; no operator migration is required.
• 0.11.0 — drops the legacy sync-agent extension namespace. Run the server-side namespace migration before installing this version if your data carries extensions['sync-agent'].*. A direct 0.9.x → 0.11.0 jump without the migration silently loses extension data. Newer Marfa servers emit a boot-time WARN when un-migrated rows are detected.
• 0.9.0 — package renamed from @withmarfa/sync-agent to @withmarfa/sync. Rename your package.json dependency.

More

Full per-root layout, mapping internals, the loopback HTTP control surface, and conflict-resolution semantics live in the repository.