@ixo/matrix-sled-migration
v1.1.1
Published
CLI tool to migrate Matrix bot encryption keys from Sled to SQLite via server-side key backup
Downloads
33
Readme
Matrix Sled Migration Tool
A CLI tool to migrate Matrix bot encryption keys from Sled storage to SQLite via server-side key backup.
What This Tool Does
Matrix bots using the matrix-bot-sdk with end-to-end encryption store their encryption keys in a local database. Older versions used Sled (a Rust-based embedded database), while newer versions use SQLite.
This tool helps you migrate your bot's encryption keys from Sled to SQLite without losing the ability to decrypt old messages. It works by:
- Extracting encryption keys from your Sled database
- Uploading them to the Matrix server's key backup system
- Allowing your bot (with the new SQLite storage) to recover the keys on first startup
Quick Start
npx @ixo/matrix-sled-migration --helpPrerequisites
- Node.js 18+ - Required for running the tool
- Rust toolchain - Required for the
extractstep (compiles a Rust binary to read Sled)# Install Rust curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh source ~/.cargo/env - Bot access token - Your bot's Matrix access token
- Bot storage directory - Path to where your bot stores its data
Installation
You can run the tool directly with npx (no installation required):
npx @ixo/matrix-sled-migration <command>Or install globally:
npm install -g @ixo/matrix-sled-migration
matrix-sled-migration <command>Environment Variables
Required
| Variable | Description | Example |
|----------|-------------|---------|
| HOMESERVER_URL | Matrix homeserver URL | https://matrix.example.com |
| ACCESS_TOKEN | Bot's access token | syt_xxx... |
| STORAGE_PATH | Path to bot's storage directory | /app/storage |
Optional
| Variable | Description | Default |
|----------|-------------|---------|
| CRYPTO_STORE_PATH | Path to crypto store | STORAGE_PATH/encrypted |
| MIGRATION_DIR | Directory for migration files | Current directory |
| OLD_DEVICE_ID | Device ID to delete | Auto-detected |
| MIGRATION_PASSWORD | Account password (non-interactive) | - |
| MIGRATION_CONFIRM | Confirm device deletion (non-interactive) | - |
| FORCE_NEW_BACKUP | Skip prompt when existing backup found | - |
| RECOVERY_PHRASE | Oracle recovery phrase for SSSS commands (extract-backup-key, store-backup-key-in-ssss, oracle-all) | - |
Commands
Step-by-Step Migration
Run these commands in order:
1. Backup (Optional but Recommended)
Create a backup of your crypto store before making any changes.
STORAGE_PATH=/app/storage \
npx @ixo/matrix-sled-migration backup2. Extract Keys
Extract encryption keys from the Sled database. Requires Rust toolchain.
STORAGE_PATH=/app/storage \
npx @ixo/matrix-sled-migration extractThis compiles a Rust binary and extracts keys to extracted-keys.json.
Fault-Tolerant Extraction (for corrupted databases)
If extraction fails with deserialization errors (e.g., "leading sigil is incorrect"), use the Rust extractor directly with fault-tolerant mode:
cd rust-key-extractor
./target/release/sled-key-extractor \
--sled-path $STORAGE_PATH/encrypted/matrix-sdk-crypto \
--output extracted-keys.json \
--skip-errors \
--failed-output failed-sessions.json \
--verboseThis mode skips corrupted entries and extracts as many valid keys as possible.
3. Enable Server Backup
Create a server-side key backup and generate a recovery key.
HOMESERVER_URL=https://matrix.example.com \
ACCESS_TOKEN=syt_xxx \
STORAGE_PATH=/app/storage \
npx @ixo/matrix-sled-migration enableIMPORTANT: This step outputs a recovery key. Save it securely! It's the only way to recover your encryption keys.
4. Upload Keys
Upload the extracted keys to the server backup.
HOMESERVER_URL=https://matrix.example.com \
ACCESS_TOKEN=syt_xxx \
STORAGE_PATH=/app/storage \
npx @ixo/matrix-sled-migration upload5. Verify Backup
Verify that all keys were uploaded successfully.
HOMESERVER_URL=https://matrix.example.com \
ACCESS_TOKEN=syt_xxx \
STORAGE_PATH=/app/storage \
npx @ixo/matrix-sled-migration verify6. Delete Old Device (Optional)
Delete the old device from the Matrix server. Requires your account password.
HOMESERVER_URL=https://matrix.example.com \
ACCESS_TOKEN=syt_xxx \
STORAGE_PATH=/app/storage \
npx @ixo/matrix-sled-migration deleteAutomated Migration
Run steps 3-5 in one command:
HOMESERVER_URL=https://matrix.example.com \
ACCESS_TOKEN=syt_xxx \
STORAGE_PATH=/app/storage \
npx @ixo/matrix-sled-migration allOracle Migration (Existing SSSS Backup)
Oracles that already have SSSS (Secret Storage) set up via MATRIX_RECOVERY_PHRASE and an existing server-side key backup don't need the enable step. Instead, the backup key is extracted from SSSS.
Extract Backup Key from SSSS
First try extracting the backup key directly from SSSS:
HOMESERVER_URL=https://matrix.example.com \
ACCESS_TOKEN=syt_xxx \
STORAGE_PATH=/app/storage \
RECOVERY_PHRASE="your_recovery_phrase" \
npx @ixo/matrix-sled-migration extract-backup-keyStore Backup Key in SSSS (if extract-backup-key fails)
If extract-backup-key fails with "No backup key found in SSSS", the backup key was never stored in SSSS (common when resetKeyBackup() created the backup). Extract it from sled and store it in SSSS:
# First, extract the backup key from sled (add --backup-key-output to your extract step):
key-extractor --sled-path $STORAGE_PATH/encrypted/matrix-sdk-crypto \
--output extracted-keys.json \
--backup-key-output backup-key.json --verbose
# Then store it in SSSS:
HOMESERVER_URL=https://matrix.example.com \
ACCESS_TOKEN=syt_xxx \
STORAGE_PATH=/app/storage \
RECOVERY_PHRASE="your_recovery_phrase" \
npx @ixo/matrix-sled-migration store-backup-key-in-ssssOracle Automated Migration
Run store-backup-key-in-ssss -> upload -> verify in one command (requires backup-key.json from the extract step):
HOMESERVER_URL=https://matrix.example.com \
ACCESS_TOKEN=syt_xxx \
STORAGE_PATH=/app/storage \
RECOVERY_PHRASE="your_recovery_phrase" \
npx @ixo/matrix-sled-migration oracle-allGenerate Recovery Key Only
For new deployments, generate a recovery key without migration:
npx @ixo/matrix-sled-migration generate-keyComplete Migration Workflow
┌─────────────────────────────────────────────────────────────────┐
│ MIGRATION WORKFLOW │
├─────────────────────────────────────────────────────────────────┤
│ │
│ 1. STOP YOUR BOT │
│ └── Ensure no processes are accessing the storage │
│ │
│ 2. BACKUP (optional) │
│ └── npx @ixo/matrix-sled-migration backup │
│ │
│ 3. EXTRACT │
│ └── npx @ixo/matrix-sled-migration extract │
│ └── Requires: Rust toolchain │
│ └── Output: extracted-keys.json │
│ │
│ 4. ENABLE + UPLOAD + VERIFY (or run "all") │
│ └── npx @ixo/matrix-sled-migration all │
│ └── Output: recovery-key.txt (SAVE THIS!) │
│ │
│ 5. SAVE RECOVERY KEY │
│ └── Store securely (password manager, encrypted backup) │
│ │
│ 6. DELETE OLD DEVICE (optional) │
│ └── npx @ixo/matrix-sled-migration delete │
│ │
│ 7. UPDATE YOUR BOT │
│ └── Deploy new version with SQLite crypto store │
│ └── Configure with recovery key for key restoration │
│ │
│ 8. START YOUR BOT │
│ └── Bot will restore keys from server backup │
│ │
└─────────────────────────────────────────────────────────────────┘Oracle Migration Workflow
For oracles with SSSS already set up (via MATRIX_RECOVERY_PHRASE):
┌─────────────────────────────────────────────────────────────────┐
│ ORACLE MIGRATION WORKFLOW │
├─────────────────────────────────────────────────────────────────┤
│ │
│ 1. STOP YOUR ORACLE │
│ └── Ensure no processes are accessing the storage │
│ │
│ 2. BACKUP (optional) │
│ └── npx @ixo/matrix-sled-migration backup │
│ │
│ 3. EXTRACT (with --backup-key-output) │
│ └── key-extractor --sled-path ... --output extracted-keys │
│ --backup-key-output backup-key.json │
│ └── Output: extracted-keys.json + backup-key.json │
│ │
│ 4a. TRY extract-backup-key (backup key may already be in SSSS) │
│ └── npx @ixo/matrix-sled-migration extract-backup-key │
│ │
│ 4b. IF 4a FAILS: store-backup-key-in-ssss │
│ └── npx @ixo/matrix-sled-migration store-backup-key-in-ssss │
│ └── Reads backup-key.json, stores key in SSSS │
│ │
│ 5. UPLOAD + VERIFY │
│ └── npx @ixo/matrix-sled-migration upload │
│ └── npx @ixo/matrix-sled-migration verify │
│ │
│ 6. CLEAR OLD STORAGE │
│ └── rm -rf /bot/storage/* │
│ │
│ 7. DEPLOY UPDATED ORACLE │
│ └── Deploy new version with SQLite crypto store │
│ └── Oracle auto-extracts backup key from SSSS on startup │
│ │
│ 8. START YOUR ORACLE │
│ └── Oracle restores keys from server backup automatically │
│ │
└─────────────────────────────────────────────────────────────────┘Key difference from bot migration: Oracles use SSSS instead of enable. First try extract-backup-key to get the key from SSSS. If the key isn't there (common when resetKeyBackup() created the backup without storing it), use store-backup-key-in-ssss to extract it from sled and store it in SSSS. The oracle's updated code auto-extracts the backup key from SSSS on every startup using MATRIX_RECOVERY_PHRASE.
Docker Usage
# Build
docker build -t matrix-sled-migration .
# Run
docker run -it --rm \
-v /path/to/bot/storage:/data \
-e HOMESERVER_URL=https://matrix.example.com \
-e ACCESS_TOKEN=syt_xxx \
-e STORAGE_PATH=/data \
matrix-sled-migration allKubernetes Usage
# Scale down the bot
kubectl scale deployment my-bot --replicas=0
# Run migration pod
kubectl run migration \
--image=ghcr.io/ixoworld/matrix-sled-migration-tool:latest \
--env="HOMESERVER_URL=https://matrix.example.com" \
--env="ACCESS_TOKEN=$BOT_TOKEN" \
--env="STORAGE_PATH=/data" \
--overrides='{
"spec": {
"volumes": [{"name": "storage", "persistentVolumeClaim": {"claimName": "my-bot-pvc"}}],
"containers": [{"name": "migration", "volumeMounts": [{"name": "storage", "mountPath": "/data"}]}]
}
}' \
--rm -it -- all
# SAVE THE RECOVERY KEY!
# Then deploy updated bot with SQLite supportRust Key Extractor CLI
The rust-key-extractor binary can be run directly for more control:
./target/release/sled-key-extractor [OPTIONS] --sled-path <PATH> --output <FILE>| Option | Description |
|--------|-------------|
| -s, --sled-path <PATH> | Path to the Sled crypto store directory |
| -o, --output <FILE> | Output file for extracted keys JSON |
| -p, --passphrase <PASS> | Store passphrase (default: empty string) |
| -v, --verbose | Enable verbose output |
| --skip-errors | Fault-tolerant mode - skip corrupted entries |
| --failed-output <FILE> | Output file for failed session details |
| --backup-key-output <FILE> | Extract backup decryption key from sled account tree |
Files Generated
| File | Description |
|------|-------------|
| recovery-key.txt | Recovery key - SAVE THIS SECURELY! |
| backup-private-key.bin | Private key for backup encryption |
| backup-public-key.txt | Public key for reference |
| extracted-keys.json | Keys extracted from Sled |
| backup-key.json | Backup decryption key from sled (when using --backup-key-output) |
| failed-sessions.json | Failed sessions (when using --skip-errors) |
| migration-state.json | Migration progress tracking |
Security
Recovery Key
The recovery key is critical:
- It's the only way to recover your encryption keys
- Store it in a password manager or encrypted backup
- Never commit it to version control
- Never share it with anyone
After Migration
After successful migration:
- Delete
extracted-keys.json(contains unencrypted keys) - Delete
backup-private-key.bin - Keep
recovery-key.txtin secure storage - Delete
migration-state.json
Troubleshooting
"Rust/Cargo not found"
Install the Rust toolchain:
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
source ~/.cargo/env"Failed to open Sled crypto store"
- Verify
STORAGE_PATHpoints to the correct directory - Check that the crypto store exists at
STORAGE_PATH/encrypted - Ensure the bot is stopped (not holding database locks)
"No keys were extracted"
- The crypto store may be empty (new bot with no E2EE history)
- The Sled store may be corrupted
"leading sigil is incorrect or missing" / "Failed to retrieve inbound group sessions"
The database has corrupted session entries. Use fault-tolerant extraction:
cd rust-key-extractor
./target/release/sled-key-extractor \
--sled-path $STORAGE_PATH/encrypted/matrix-sdk-crypto \
--output extracted-keys.json \
--skip-errors \
--verboseThis skips corrupted entries and extracts all valid keys. See MIGRATION_STEPS.md for details.
"Authentication failed"
- Verify
ACCESS_TOKENis correct and not expired - Check that
HOMESERVER_URLis correct
"Module not found" errors
Ensure you have Node.js 18+ installed:
node --version # Should be v18.x.x or higherLicense
Apache-2.0