mcpscraper-memory-db

Shared data-access layer for the mcpscraper memory product: Postgres db/*.ts modules, the lib/*.ts utilities they depend on (auth, secret encryption, vault contracts, cost rates, the already-HTTP-only credit-debit client), and the rag/*.ts embedding/indexing layer (Jina embeddings over @mastra/pg's PgVector).

This package exists because mcp-memory and mcp-scraper-scheduler both need this exact logic — before this package, it was hand-copied between the two repos, which is exactly the kind of drift this package is meant to prevent. Each consumer's own src/mastra/db/*.ts, lib/*.ts, and rag/*.ts files are one-line re-export shims pointing here; the real implementation lives only in this package.

Usage

import { query, ensureSchema } from 'mcpscraper-memory-db/db/pool'
import { resolveToolAuth } from 'mcpscraper-memory-db/lib/auth'

Every subpath under db/, lib/, and rag/ is exported individually — there is no single barrel export, to keep tree-shaking simple and match how the original files were imported.

Requires

DATABASE_URL (Postgres, used by db/pool.ts), JINA_API_KEY/JINA_EMBED_MODEL/JINA_EMBED_DIM (used by rag/embedder.ts), SCHEDULE_SECRET_ENC_KEY (used by lib/secret-crypto.ts), MCP_SCRAPER_INTERNAL_URL/INTERNAL_MEMORY_DEBIT_SECRET (used by lib/credit-debit.ts's HTTP call into mcp-scraper's real ledger) — same env vars either consuming repo already needs.

Publishing

npm run build
npm publish

Bump the version in package.json first. Both consumers pin an exact or caret version and need a npm install mcpscraper-memory-db@latest + redeploy to pick up a new release — there is no auto-sync.