Mnemo Migrate

Move existing documents and memories into Mnemo from files or another memory provider. Imports are resumable, failed records are isolated, and source systems never need to share credentials with Mnemo.

Install

npm install -g getmnemo-migrate

Quickstart

export MNEMO_API_KEY=mn_...
export MNEMO_WORKSPACE_ID=...

# Preview the source. This does not write anything.
getmnemo-migrate plan from raw-csv \
  --file ./contacts.csv \
  --map 'content=*,id=Record ID'

# Import it into one customer container.
getmnemo-migrate from raw-csv \
  --file ./contacts.csv \
  --map 'content=*,id=Record ID' \
  --container pilot:customer-name

That is the full path: preview, import, then keep the printed job ID for status and retries.

What the CLI Handles

• Checkpoints each import so it can resume without replaying successful records.
• Sends documents in batches of up to 50 and memories in batches of up to 100.
• Divides a rejected memory batch until the exact malformed records are isolated.
• Records malformed source rows and failed writes without stopping valid records.
• Preserves source IDs and provenance on every imported record.
• Runs retrieval checks from a JSON question file after the import.

The content=* mapping serializes each complete CSV or JSON record. For cleaner source data, map a specific field instead, such as content=notes,id=record_id,company=company.

Import Lanes

| Lane | Default sources | Behavior | | --- | --- | --- | | document | CSV, JSON, JSONL, Markdown, text, directories | Sends source material through Mnemo document ingestion so facts, provenance, entities, and time can be extracted. | | memory | Mem0, Zep, Supermemory, Letta | Preserves already-extracted memory text as atomic memories. |

Override the default only when the source semantics require it with --lane memory or --lane document.

File Sources

| Source | Required option | Notes | | --- | --- | --- | | raw-csv | --file export.csv | RFC-compatible CSV parsing, including quoted multiline fields. | | raw-json | --file export.json | Top-level arrays up to 50 MB. Convert larger files to JSONL. | | raw-jsonl | --file export.jsonl | Streams one JSON object per line. | | text-file | --file notes.md | Imports one Markdown or text document. | | directory | --file ./export | Recursively imports supported files in deterministic order; ignores symlinks and unsupported files. |

Mappings use target=source pairs and support dotted source paths:

getmnemo-migrate from raw-json \
  --file ./history.json \
  --map 'content=payload.body,id=record.id,customer=account.id' \
  --container pilot:customer-name

Map container=<field> when one approved export contains multiple customers. Each record is then written to its own isolated container; --container remains the fallback.

Provider Sources

| Source | Source credentials | Useful options | | --- | --- | --- | | mem0 | MEM0_API_KEY | --user, --base-url | | zep | ZEP_API_KEY | --user is required, --base-url | | supermemory | SUPERMEMORY_API_KEY | --source-container is required, --base-url | | letta | LETTA_API_KEY, LETTA_BASE_URL | Imports core and archival memory |

Recommended Flow

1. Export data from the source system.
2. Run plan from ... and inspect the count and sample. Planning makes no Mnemo API calls.
3. Import into a customer-specific container with conservative concurrency.
4. Keep the printed migration job ID.
5. Run status <jobId> or reconcile <jobId> to obtain the final completion report.
6. Retry isolated write failures with retry-failed <jobId>.
7. Validate retrieval against a small set of customer-approved questions before connecting an agent.

getmnemo-migrate status
getmnemo-migrate cancel mig_...
getmnemo-migrate resume mig_...
getmnemo-migrate reconcile mig_...
getmnemo-migrate retry-failed mig_...

Use --json with planning, imports, resume, retry, reconciliation, status, and cancellation when another system needs to consume the report.

Create a verification file after agreeing expected answers with the customer:

[
  {
    "question": "What is the current support response time?",
    "expect": ["two business hours"],
    "searchMode": "hybrid",
    "limit": 10
  }
]

Then run retrieval acceptance against the same isolated container:

getmnemo-migrate verify mig_... --questions ./acceptance-questions.json --json

The command exits non-zero when an expected phrase is missing. Its JSON report includes the result IDs that supported each successful check. For an import that mapped records to several containers, pass --container to verify one customer boundary at a time.

Cancellation pauses after the current bounded batch. Resume skips successful source records and retries unresolved ones. Mnemo custom IDs and idempotency keys also protect against duplicate writes if local state is lost.

Each record outcome is durably journaled together with its source ID, reuse status, remote ingestion job IDs, and safe failure classification. Resume and retry-failed use the latest outcome for each source record, so an old failure cannot replay a record that later succeeded. State is stored under ~/.getmnemo/migrations with owner-only permissions. Set MNEMO_MIGRATION_DIR to use another state directory. Jobs created by earlier package versions keep their existing progress when they first adopt the outcome journal.

Reliability Limits

• Concurrency is capped at 25 even if a larger value is requested.
• HTTP 429 and server/network failures retry up to five times with backoff.
• Successful memory responses are accepted only when every input has one unique created or deduplicated searchable receipt.
• Document batch totals and item indexes must agree before any item is checkpointed.
• Malformed JSONL rows are journaled while valid rows continue.
• Only the latest unresolved retryable writes can be retried in a new job.
• Bulk requests contain at most 50 documents and stay below a conservative request-size ceiling.
• Documents larger than the API limit are split deterministically near line boundaries.
• Atomic memories longer than the memory endpoint limit are split with part metadata.
• JSON arrays are capped at 50 MB to avoid loading unbounded files into memory; JSONL and CSV stream.
• The completion report distinguishes accepted writes from completed and failed ingestion jobs.

Environment

| Variable | Purpose | | --- | --- | | MNEMO_API_KEY | Destination API key. Required for writes and reconciliation. | | MNEMO_WORKSPACE_ID | Destination workspace. Required for writes and reconciliation. | | MNEMO_API_URL | API base URL. Defaults to https://api.mnemohq.com. | | MNEMO_CONTAINER_TAG | Optional default destination container. | | MNEMO_MIGRATION_DIR | Optional local checkpoint directory. |

The older GETMNEMO_* variable names remain supported for existing scripts. When both are set, MNEMO_* wins.

See Pilot integration contract for the boundary between source exports, Mnemo, governance, and agents.

License

MIT