@galdor/memory-s3vectors

An Amazon S3 Vectors long-term memory store for galdor. It implements the core memory.Store interface, so it drops in behind a Retriever exactly like the bundled InMemoryStore — but persists vectors in an S3 Vectors index.

Credentials

Resolved by the AWS SDK's default provider chain (environment variables → shared ~/.aws config → ECS container credentials → EC2 IMDS / task role). No static keys are accepted here; configure AWS the standard way.

Usage

import { openS3Vectors } from "@galdor/memory-s3vectors";
import { Retriever } from "@galdor/core/memory";

// Probes the index and creates it if missing.
const store = await openS3Vectors({
  bucket: "my-vectors",     // an existing S3 Vectors bucket
  index: "galdor-chunks",   // optional; default "galdor-chunks", created if absent
  region: "us-east-1",      // optional; default from the AWS chain
  dim: 1024,                // embedding dimensionality
  // distance: "cosine",    // optional; default cosine (ignored if the index exists)
});

await store.add([
  { id: "c1", documentId: "d1", index: 0, text: "…", embedding: vec, metadata: { lang: "es" } },
]);

const hits = await store.retrieve({ embedding: queryVec, k: 5, filter: { lang: "es" } });
// hits: Result[] in descending relevance (higher score = more relevant)

await store.delete("d1"); // removes every chunk of document d1
await store.close();

Compose it with an embedder via Retriever for text queries:

const retriever = new Retriever({ store, embedder, defaultK: 5 });
const hits = await retriever.retrieve({ text: "capital of Ecuador" });

Behavior

• Open — validates bucket/dim/index name, then probes the index; if missing, creates it as float32, the configured dim and distance metric, with __text declared non-filterable.
• add — upserts each chunk keyed by its id (idempotent). Stores the embedding plus documentId, index, text and any chunk metadata. Rejects empty ids, dimension mismatches, and metadata keys using the reserved __ prefix. Batched at 500 vectors per call.
• retrieve — top-K nearest by the query embedding; filter is an exact-match AND over metadata (single key bare, multiple keys via $and). Cosine distance becomes 1 - distance (anti-correlated hits dropped); Euclidean becomes 1 / (1 + distance).
• delete — scans the index and removes every vector whose documentId metadata matches; batched at 500 keys per call.
• close — destroys the underlying client.

Notes

• Vector-only: retrieve requires query.embedding.
• The metadata keys __document_id, __index, __text are reserved.