💾 @backtest-kit/mongo

MongoDB + Redis persistence for backtest-kit. Swaps the default file storage for a production backend — durable, queryable, atomic, with O(1) cached reads — in one setup() call and zero strategy-code changes.

📚 Docs · 🌟 Reference implementation · 🐙 GitHub

npm install @backtest-kit/mongo backtest-kit mongoose ioredis

import { setup } from '@backtest-kit/mongo';
setup(); // reads connection settings from env; call once before any trading operation

That single call reimplements all 16 of backtest-kit's IPersist*Instance contracts against MongoDB (source of truth) with a Redis O(1) read cache. Your strategy code does not change.

Why

File storage is perfect on day one and a bottleneck the day you're doing thousands of context-keyed reads per second across parallel symbols. This package moves persistence to MongoDB without touching strategy logic: every read goes Redis-first for the Mongo _id (two O(1) hops), every write is one atomic findOneAndUpdate upsert (read-after-write guaranteed, concurrent duplicates rejected by the unique index), and adapters whose data drives decisions store the simulation timestamp so look-ahead protection is enforceable even inside the database.

• 🗄️ MongoDB backend — all 16 IPersist*Instance contracts implemented with Mongoose.
• ⚡ O(1) reads via Redis — one GET + one findById, no B-tree scans on the hot path.
• 🔒 Atomic writes — findOneAndUpdate({ upsert:true, new:true }) guarantees read-after-write with no race.
• 🛡️ Look-ahead protection — decision-affecting adapters store the simulation when.
• 🪦 Soft delete — Measure / Interval / Memory carry a removed flag instead of being deleted (audit trail).
• 🔌 Zero strategy changes — drop setup() into your entry point; everything else stays the same.

Configuration

import { setup } from '@backtest-kit/mongo';
setup({
  CC_MONGO_CONNECTION_STRING: 'mongodb://mongo:27017/mydb',
  CC_REDIS_HOST: 'redis', CC_REDIS_PORT: 6379, CC_REDIS_PASSWORD: 'secret',
});

| Variable | Default | Description | |----------|---------|-------------| | CC_MONGO_CONNECTION_STRING | mongodb://localhost:27017/backtest-kit?wtimeoutMS=15000 | MongoDB connection string | | CC_REDIS_HOST | 127.0.0.1 | Redis host | | CC_REDIS_PORT | 6379 | Redis port | | CC_REDIS_USER | (empty) | Redis username | | CC_REDIS_PASSWORD | (empty) | Redis password |

Values passed to setup() / setConfig() always take precedence over env vars. Within the CLI, put setup() in config/setup.config.ts — when present, the CLI skips its default file-adapter registration and your config owns persistence.

API reference

| Export | Description | |--------|-------------| | setup(config?) | Configure and register all 16 adapters in one call. Reads env when config omitted. | | install() | Register adapters only — when config was already applied via setConfig/env. | | setConfig(config) | Override individual connection parameters at runtime. | | getConfig() | The current merged configuration (env + any setConfig overrides). | | setLogger(logger) | Replace the internal logger with your own implementation. | | getMongo() | The connected Mongoose instance (lazy singleton). | | getRedis() | The connected ioredis instance (lazy singleton). |

The 16 adapters

Each adapter covers one persistence slot in backtest-kit. The unique index is the compound key MongoDB enforces at the storage engine.

| Adapter | Collection | Unique index | |---------|------------|--------------| | Candle | candle-items | symbol + interval + timestamp | | Signal | signal-items | symbol + strategyName + exchangeName | | Strategy | strategy-items | symbol + strategyName + exchangeName | | Schedule | schedule-items | symbol + strategyName + exchangeName | | Risk | risk-items | riskName + exchangeName | | Partial | partial-items | symbol + strategyName + exchangeName + signalId | | Breakeven | breakeven-items | symbol + strategyName + exchangeName + signalId | | Storage | storage-items | backtest + signalId | | Notification | notification-items | backtest + notificationId | | Log | log-items | entryId | | Measure | measure-items | bucket + entryKey | | Interval | interval-items | bucket + entryKey | | Memory | memory-items | signalId + bucketName + memoryId | | Recent | recent-items | symbol + strategyName + exchangeName + frameName + backtest | | State | state-items | signalId + bucketName | | Session | session-items | strategyName + exchangeName + frameName |

• Candle is immutable — first write wins; subsequent writes to the same (symbol, interval, timestamp) are silently ignored via $setOnInsert (historical OHLCV never changes).
• All others use $set — each write replaces the previous value.
• Measure / Interval / Memory soft-delete — removeMeasureData / removeIntervalData / removeMemoryData set removed: true rather than deleting; listings filter on removed: false, keeping a full audit trail.

How it works

Every domain is two layers: a DbService (MongoDB) and a CacheService (Redis). Reading state for a context key asks Redis for the Mongo _id first; a hit is two O(1) ops, a miss falls back to an indexed findOne and backfills Redis.

read signal for (BTCUSDT, my_strategy, binance)
  ├─ Redis GET → hit  → Mongo findById(_id)        ← O(1) + O(1)
  └─ Redis GET → miss → Mongo findOne(filter) → Redis SET → return

After every write the Redis entry is refreshed in the same call, so write-then-read always hits the cache.

backtest-kit requires that once write*Data() returns, the next read*Data() sees the new value. Every write is one findOneAndUpdate round-trip:

const document = await SignalModel.findOneAndUpdate(
  { symbol, strategyName, exchangeName },
  { $set: { payload } },
  { upsert: true, new: true, setDefaultsOnInsert: true },
);
await signalCacheService.setSignalId(readTransform(document.toJSON()));

The filter matches the unique compound index, so MongoDB rejects any concurrent duplicate insert at the storage engine; the returned document is written straight to Redis, making the next read O(1) on fresh data.

Adapters whose data influences decisions (Risk, Partial, Breakeven, Recent, State, Session, Memory, Interval) store when: Number — the simulation timestamp in ms — alongside the payload, so backtest-kit can verify no read returns data written at a future simulation time. Measure is exempt because it caches LLM / external-API responses, where look-ahead bias is not meaningful.

Internal architecture (complete source map)

Public surface — functions/setup.ts (setup/install/setConfig/getConfig/setLogger), index.ts re-exports + getMongo/getRedis.

Adapter classes (classes/Persist*Instance.ts, 16) — each implements one backtest-kit IPersist*Instance contract and delegates to its domain DbService: PersistCandleInstance, PersistSignalInstance, PersistStrategyInstance, PersistScheduleInstance, PersistRiskInstance, PersistPartialInstance, PersistBreakevenInstance, PersistStorageInstance, PersistNotificationInstance, PersistLogInstance, PersistMeasureInstance, PersistIntervalInstance, PersistMemoryInstance, PersistRecentInstance, PersistStateInstance, PersistSessionInstance.

Service layer (lib/services/):

• base/ — MongoService (lazy Mongoose connection), RedisService (lazy ioredis), LoggerService.
• db/ — one *DbService per domain: the Mongoose models, schemas, unique compound indexes, and findOneAndUpdate upsert logic.
• cache/ — one *CacheService per domain (CandleCacheService, SignalCacheService, BreakevenCacheService, IntervalCacheService, LogCacheService, MeasureCacheService, MemoryCacheService, NotificationCacheService, PartialCacheService, RecentCacheService, …): Redis _id mapping for O(1) lookups.

Shared primitives (lib/common/) — BaseCRUD (the upsert/read/remove pattern every DbService reuses) and BaseMap (the Redis key-mapping pattern every CacheService reuses).

DI & config — lib/core/{di,provide,types}.ts (IoC container wiring Db/Cache/base services), lib/index.ts (container bootstrap), config/{mongo,redis,params}.ts (connection builders + merged params), interfaces/Logger.interface.ts.

🤝 Contribute

Fork / PR on GitHub.

📜 License

MIT © tripolskypetr