🚀 Star-Vault

Unleash the future of data management with the ultimate platform for secure, scalable, and dynamic data operations. Power the next generation of applications by combining advanced encryption, revolutionary real-time querying, and seamless synchronization to create an ecosystem where data transforms into action.

📘 Table of Contents

• 🌌 Features
• ⚙️ System Requirements

API Overview

• 📜 API Specifications
• 🔧 Constructor
• 🔐 authOptions (StarAuth Configuration)
• 🔒 Lockout Configuration
• Timing Heuristics
• Password Management
• 🌍 Star Locator Configuration
• Stellar Login
• Hooks & Extensions
• Guest Session Configurations
• Cleanup & Locking

Core Engine

• 📦 Core Methods
• 📥 Import & Snapshot Methods
• 🧾 StarTransactions (WAL + Idempotency)

Query System

• 🔍 Query Engine
• ⚙️ Return Modes
• Execution Modes
• Methods
• Examples
• ⚙️ Advanced Controls
• Performance & Behavior
• Developer Note

Hyper Normalization

• ⚡️ Hyper-Normalization™
• 🧬 Origin of Hyper-Normalization™
• 🌌 Why Hyper-Normalization Matters
• 🧩 User Display Example
• Benchmark & Reproducibility
• 🚀 Performance Implications

Authentication

• 🛡️ Authentication
• 👤 User Registration & Authentication
• 🛰️ Session Management
• 🔑 Password & Magic Link Recovery
• 👥 Guest Account Management
• 🔁 User Lifecycle Management
• 📊 User & Session Querying
• 🧩 Internal Validation & Utility Methods

History + Logging

• 🧭 Record History & Timeline
• 🧾 Auditing
• 🪶 Debug & Developer Logging Options

Logger

• 🛰️ StarLogger
• 🔧 Core Logger Methods
• Write, System, and Audit Events
• Recovery, Query & Maintenance
• Rotation & Retention

Storage Layer

• 📂 Starchive (Storage Layer)
• ⚙️ Starchive Configuration Modes
• ⚙️ Starchive Configurations
• 📦 Starchive Core Methods
• ✍️ Starchive Convenience Methods
• 🛰️ Remote Management Helpers
• 🔏 Starchive Signing & Presigning
• 🔏 Starchive Downloading
• 📜 Starchive Records

Event System

• 🎧 Event Listening
• 🌌 Collection & Path Matching
• 🌌 StarVault Events

Vacuum System

• 🧽 Vacuum & Auto-Maintenance System
• ⚙️ Automatic Vacuum Policy
• 🪄 Methods
• 🪶 Example
• ⚡ Auto-Vacuum Triggers
• 📊 Metrics Example

Getting Started

• ✨ Getting Started

• 📦 Installation

• Basic Usage

• 📈 Querying

• 🎧 Listening to Changes

• 🌐 Wildcard Collection Queries

• 👥 Registering Users

• 📋 Listing Users and Sessions

• 🔍 Quick Starchive Examples

• ✨ License

• 🚀 Transform Data into Action

🌌 Features

• 🔐 Optional Encryption – Secure sensitive data at rest
• ⚙️ Sharded Storage Engine – Efficiently scales writes across shards
• 🧠 In-Memory Caching – High-speed read layer with StarCache
• 📜 Write-Ahead Logging – Resilient logs with rotation and retention policies
• 🔍 Advanced Query Engine – Chainable and expressive queries with filtering, search, sorting, and spatial support
• 🚀 Real-Time Event Emission – Listen to data changes with fine-grained control
• 🛡️ Authentication Layer – Optional handler to authorize every operation
• 🌠 Collection Wildcards – Seamlessly operate across multiple collections

⚙️ System Requirements

| Requirement | Version | |----------------|--------------------| | Node.js | ≥ 19.x | | npm | ≥ 9.x (recommended)| | OS | Windows, macOS, Linux |

📜 API Specifications

🔧 Constructor

Use these parameters when initializing StarVault.

new StarVault(dbPath, logPath, shardCount, maxLogSize, logRetention, options)

Core Parameters

| Key | Description | Default | |---------------|------------------------------------------------------------------------------|--------------| | dbPath | Root directory for all collection data; each collection is stored in sharded files. | Required | | logPath | Directory where write-ahead logs (WAL) are stored for durability and replay. | Required | | shardCount | Number of shards used per collection. More shards improve concurrency. | 4 | | maxLogSize | Max size of each WAL file before rotation. Accepts values like "100MB". | "869MB" | | logRetention| Duration to retain old WAL files. Accepts values like "1w" or "30d". | "1w" |

options Object

| Key | Description | Default | |-------------------|-----------------------------------------------------------------------------|------------------------| | enableEncryption| Enables encryption for data at rest. | false | | vaultPath | Path to the vault metadata file used for encryption. | "./star-vault.json" | | masterKey | Master encryption key for vault encryption. Must securely generate/store.| null | | authHandler | Custom function that receives clientAuth and returns true/false. | null | | authOptions | Configuration object passed to internal StarAuth for authentication. | {} | | auditHistory | Turn on default auditing for history/timeline reads (writes entries to audit.log). Also toggle per call. | false | | enableMesh | Turn on StarMesh sync layer. | false | | meshOptions | Options passed to StarMesh when enableMesh = true. | {} | | starchive | Configuration for the Starchive subsystem (local and remote file management). | {} | | debugOptions | Fine-grained developer logging controls for internal tracing (see below). | { all flags default to true } | | dirMode | UNIX permission mode for directories (e.g., 0o700). | 0o700 | | fileMode | UNIX permission mode for files (e.g., 0o600). | 0o600 |

🔐 authOptions (StarAuth Configuration)

Pass this object under the options.authOptions key when creating your StarVault instance.

Collection Settings

| Key | Description | Default | |----------------------|--------------------------------------------|---------------------| | starAuthEnabled | Global switch that controls whether the StarAuth subsystem activates. Set to false to disable authentication features entirely while keeping vault access open. | true | | collection | User collection name | "auth-users" | | sessionCollection | Session tracking collection | "auth-sessions" | | resetCollection | Password reset token collection | "auth-resets" | | lockCollection | Lockout state tracking collection (per account/IP/UA). | "auth-locks" | | stellarCollection | Magic link / code token collection | "stellar-auths" |

Session Behavior

These options control the creation, validation, and protection of sessions against brute-force attacks.

| Key | Description | Default | |----------------------------|------------------------------------------------------------------------------|--------------------------| | tokenExpiry | Lifetime of each session token in seconds. | 3600 (1 hour) | | sessionValidationFields | Fields compared on every request to detect hijacking (e.g. IP, fingerprint). | ["ip", "fingerprint"] | | strictSessionValidation | If true, any mismatch in validation fields invalidates the session. | true | | enableSuspiciousCheck | If true, compares geo/IP against recent sessions to flag anomalies. | true | | sessionPolicy | Session strategy: "default" (allow many), "strict" (1 active max), "replace" (replace old). | "default" |

🔒 Lockout Configuration

Controls the tracking, weighting, and escalation behavior of failed login attempts. All options go inside options.authOptions.lockout.

| Key | Description | Default | |--------------------|-------------------------------------------------------------------------------------------------|------------------------| | strategy | Lockout algorithm: "fixed", "slidingWindow", or "exponential". | "fixed" | | scopes | Dimensions to track failures against: ["account", "ip", "ipua"]. | ["account", "ipua"] | | maxAttempts | Failures allowed before triggering lock. | 5 | | baseDuration | Base lockout duration in ms. | 15 * 60 * 1000 | | maxDuration | Max lockout duration in ms (for exponential strategy). | 24 * 60 * 60 * 1000 | | minDuration | Absolute minimum lock duration in ms (applies to all strategies; jitter won’t go below this). | 30 * 1000 | | jitterDuration | Random jitter ± ms added to lock duration. | 15 * 1000 | | decayDuration | Auto-decay failures if no attempts within this duration. | 30 * 60 * 1000 | | windowDuration | For slidingWindow, the rolling time window in ms. | 10 * 60 * 1000 | | windowThreshold | For slidingWindow, number of failures in the window that triggers lock. | 7 | | windowGrowth | For slidingWindow, how lock duration grows when over threshold: "none" \| "linear" \| "exponential". | "none" | | windowGrowthFactor | For slidingWindow: "exponential" — multiplier per extra failure over threshold. | 2 | | windowGrowthStep | For slidingWindow: "linear" — extra ms per extra failure over threshold. | baseDuration / 2 | | captchaAfter | Number of failures after which a CAPTCHA challenge is required. | null | | otpAfter | Number of failures after which an OTP challenge is required. | null | | captchaVerifier | Function (req) => boolean to validate CAPTCHA. | null | | otpVerifier | Function ({ email, code }) => boolean to validate OTP. | null | | onLock | Callback (info) => void fired when a lock is applied. | null | | onUnlock | Callback (info) => void fired when a lock is cleared. | null | | onChallenge | Callback (info) => void fired when a CAPTCHA/OTP challenge is required. | null | | notifyLock | Optional notifier (info) => void for alerts/logging. | null | | bypassed | Function (req) => boolean to completely bypass lockout logic for trusted requests. | null |

Timing Heuristics

options.authOptions.lockout.timingHeuristics adds intelligence by classifying failures as bursts, normal, or human-like delays.

| Key | Description | Default | |-----------------------|-----------------------------------------------------------|----------| | enabled | Enable timing heuristics. | true | | burstDetectionDuration | Attempts within this ms count as a burst (weighted more). | 3000 | | humanDetectionDuration | Attempts slower than this ms count as human/grace. | 45000 | | maxConsiderDuration | Ignore intervals longer than this ms. | 600000 | | burstWeight | Weight applied to burst failures. | 2.0 | | normalWeight | Weight applied to normal failures. | 1.0 | | graceWeight | Weight applied to grace (slow) failures. | 0.0 | | emaAlpha | Alpha for exponential moving average of attempt intervals. | 0.3 | | decayOnGrace | Whether to decay counters when a grace attempt is detected.| true | | decayStep | How many failures to subtract during a grace decay. | 1 | | minFailures | Clamp: minimum number of failures tracked. | 0 | | maxFailures | Clamp: maximum number of failures tracked. | 9999 |

Example Fixed Lockout Config

authOptions: {
  lockout: {
    strategy: "fixed",
    scopes: ["account", "ipua"],
    maxAttempts: 5,                     // lock after N total failures
    baseDuration: 15 * 60 * 1000,       // fixed lock duration (with jitter)
    minDuration: 30 * 1000,
          jitterDuration: 15 * 1000,          // desync scripted retries
    decayDuration: 30 * 60 * 1000,      // clear counters after idle period
  
    // Optional human gates
    captchaAfter: 3,
    otpAfter: null,
    captchaVerifier: async (req) => verifyCaptcha(req.body.token),
  
    //  Heuristics amplify bursts, forgive human pace
    timingHeuristics: {
      enabled: true,
      burstDetectionDuration: 3000,
      humanDetectionDuration: 45000,
      burstWeight: 2.0,
      normalWeight: 1.0,
      graceWeight: 0.0,
      emaAlpha: 0.3,
      decayOnGrace: true,
      decayStep: 1,
      minFailures: 0,
      maxFailures: 9999
    },
  
    onLock: (info) => console.log("LOCKED", info),
    onUnlock: (info) => console.log("UNLOCKED", info),
    onChallenge: (info) => console.log("CHALLENGE", info),
    bypassed: (req) => false
  }
}

👉 When to use: Simple setups where predictability matters. Good for smaller apps, prototypes, or admin dashboards where you want consistent enforcement.

Example Sliding Window Lockout Config

authOptions: {
  lockout: {
    strategy: "slidingWindow",
    scopes: ["account", "ipua"],
    
    windowDuration: 10 * 60 * 1000,
          windowThreshold: 7,
    
          baseDuration: 60 * 1000,
          minDuration: 30 * 1000,
          maxDuration: 24 * 60 * 60 * 1000,
          jitterDuration: 15 * 1000,
    
          windowGrowth: "exponential",
          windowGrowthFactor: 2,
          // windowGrowthStep: 30 * 1000, // if using "linear"

    // Optional Human gates
    captchaAfter: 3,
    otpAfter: null,
    captchaVerifier: async (req) => verifyCaptcha(req.body.token),

    // Heuristics amplify bursts, forgive human pace
    timingHeuristics: {
      enabled: true,
      burstDetectionDuration: 2000,
      humanDetectionDuration: 45000,
      burstWeight: 2.5,
      normalWeight: 1.0,
      graceWeight: 0.0,
      emaAlpha: 0.35,
      decayOnGrace: true,
      decayStep: 1,
      minFailures: 0,
      maxFailures: 9999
    },

    onLock: (info) => console.log("LOCKED", info),
    onUnlock: (info) => console.log("UNLOCKED", info),
    onChallenge: (info) => console.log("CHALLENGE", info),
    bypassed: (req) => false
  }
}

👉 When to use: Ideal for games and consumer apps. It’s forgiving to legit players (counters decay naturally) but still stops brute-force bursts quickly.

Example Exponential Lockout Config

authOptions: {
  lockout: {
    strategy: "exponential",
    scopes: ["account", "ipua"],
    maxAttempts: 5,
    baseDuration: 30 * 1000,
          minDuration: 30 * 1000,
          maxDuration: 24 * 60 * 60 * 1000,
          jitterDuration: 10 * 1000,
    captchaAfter: 3,
    otpAfter: 5,
    captchaVerifier: async (req) => verifyCaptcha(req.body.token),
    otpVerifier: async ({ email, code }) => verifyOtp(email, code),
    onLock: (info) => console.log("LOCKED", info),
    onUnlock: (info) => console.log("UNLOCKED", info),
    timingHeuristics: {
      enabled: true,
      burstWeight: 3.0,
      graceWeight: 0.0,
      emaAlpha: 0.4
    }
  }
}

👉 When to use: Best for high-security apps, admin portals, or payment flows. Makes repeated brute-forcing impractical, but can be harsh for end-users if tuned too aggressively.

🧭 Legacy Lockout

These highlight the original “simple” lockout knobs. They remain in place for backward compatibility and apply in addition to the modern authOptions.lockout logic.

| Key | Description | Default | |---------------------|----------------------------------------------------------------------------------------------------------|----------------------------| | lockoutDuration | (Legacy) Fixed lock duration (ms) after too many failed logins. | 900000 (15 minutes) | | maxLoginAttempts | (Legacy) Number of failed attempts before the legacy lockout triggers. | 5 |

How they interact:

• On every failed login, the modern lockout mechanism (if configured) evaluates first.
• The legacy counters also increment and, if failedAttempts >= maxLoginAttempts, a fixed lock of lockoutDuration is applied.
• This ensures older deployments keep their original behavior while you migrate to the new model.

Recommendation:

• Prefer the new authOptions.lockout settings for fine-grained control.
• If you want the legacy lockout to rarely affect users, set a very large maxLoginAttempts (e.g., 1_000_000) and leave modern lockout to do the real work.

Example (modern lockout active, legacy effectively neutralized):

authOptions: {
  // Modern lockout (recommended)
  lockout: {
    strategy: "exponential",
    scopes: ["account", "ipua"],
    maxAttempts: 5,
    baseDuration: 15 * 60 * 1000,    // 15 minutes
    maxDuration: 24 * 60 * 60 * 1000 // 24 hours
    // ...captcha/otp/timingHeuristics, etc.
  },

 maxLoginAttempts: 5,
  lockoutDuration: 15 * 60 * 1000
}

Password Management

| Key | Description | Default | |----------------------|----------------------------------------------------------------------------------------------|--------------------------------------------| | passwordRequirements | Object defining the password policy | { minLength: 8, requireLetter: true, requireNumber: true, requireSymbol: false } | | └ minLength | Minimum number of characters required | 8 | | └ requireLetter | Requires at least one letter (a-z or A-Z) | true | | └ requireNumber | Requires at least one numeric digit (0-9) | true | | └ requireSymbol | Requires at least one symbol (e.g., !@#$%^&*) | false | | └ customValidator | Optional custom function (password) => boolean for advanced password checks | null | | lockingCombinations | Password hash complexity | 10 |

🌍 Star Locator Configuration

| Key | Description | Default | |---|---|---| | enableGeo | Enable geo lookups. | false | | enableReverseGeo | Reverse geocode coordinates. | false | | enableGeoDebug | Log provider results. | false | | geoProviders | Extra providers (ip, {signal}) => result. | [] | | ipgeolocationKey | ipgeolocation.io key. | null | | ipinfoToken | ipinfo.io token. | null | | nominatimUserAgent | UA for Nominatim reverse geo. | "StarAuth/1.0 ([email protected])" | | googleMapsKey | Google Geocoding key. | null | | mapboxToken | Mapbox token. | null | | geoCacheTtlMs | Cache TTL. | 600000 | | geoDeadlineMs | Deadline per lookup. | 3000 | | geoTimeoutMs | Timeout per provider. | 2500 | | geoScoreOk | Early-accept score threshold. | 5 | | geoMaxConcurrency | Max concurrent calls. | 8 |

Example geo Object in Sessions

geo: {
  requestIP: "203.0.113.45",
  ip: "203.0.113.45",
  city: "City",
  region: "Region",
  country: "Country",
  continent: "Continent",
  org: "Org / ASN",
  isp: "ISP",
  loc: "12.3456,-98.7654",
  timezone: "Area/City",
  postal: "ZIP",
  flag: "🌍",
  confidence: 0.83,
  geoAddress: "123 Main St, Example City, Country"
}

Custom Provider Registration

const vault = new StarVault(dataDir, logDir, 4, "869MB", "1w", {
  authOptions: {
    enableGeo: true,
    enableReverseGeo: true,
    ipinfoToken: process.env.IPINFO_TOKEN,
    geoProviders: [
      async function customProvider(ip, { signal }) {
        return { city: "Custom City", country: "Customland" };
      }
    ]
  }
});

Stellar Login

| Key | Description | Default | |---------------------------|-------------------------------------------------|-----------------| | stellarRequestCooldown| Minimum delay between stellar requests | 60000 (1 min) | | generateStellarCode | Function to generate a stellar numeric code | 6-digit default |

Hooks & Extensions

| Key | Description | |----------------------|------------------------------------------------------| | onSuspiciousSession| (currentSession, pastSession) => void | | handleHijack | (session, field, expected, actual) => void | | onCleanup | ({ result, timestamp, vaultID }) => void | | tagSession | (session, userData) => string[] |

Guest Session Configurations

| Key | Description | Default | |-----------------------------|-----------------------------------------------------------------------------|----------------------------------| | allowGuestSessions | Enables or disables guest session support. | true | | guestInactivityThreshold | Duration of inactivity (in ms) before considering a guest account stale. | 7 * 24 * 60 * 60 * 1000 (1w) | | cleanupGuestInterval | Interval (in ms) to check for inactive guest accounts. | 5 * 60 * 1000 (5m) | | guestActivityTrackers | Array of custom functions to execute on guest activity updates. | [] | | generateGuestID | Function to generate guest user IDs. | () => "guest-" + UUID |

Cleanup & Locking

| Key | Description | Default | |--------------------------------|------------------------------------------------------|-------------| | vaultID | ID for this vault instance | null | | autoCleanupInterval | How often to auto-clean expired tokens | null | | expiredSessionCleanupInterval | Interval for cleaning expired sessions | null | | detachedTimers | Controls whether background cleanup timers remain linked to the process. Set to false to keep timers active until completion (useful for long-running or graceful shutdown scenarios). | true | | cleanupExpiredTokensActionInfo | Metadata for cleanup activity | {} | | cleanupExpiredTokensClientAuth | Auth context used for cleanup calls | null | | cleanupExpiredSessionsActionInfo | Metadata for session cleanup | {} | | cleanupExpiredSessionsClientAuth | Auth context used during session cleanup | null |

📦 Core Methods

The primary database interaction methods.

| Method | Description | Sync/Async | |--------|-------------|------------| | create(collection, data, actionInfo = {}, clientAuth = null) | Creates a new record in the specified collection. Returns the created record object. | ✅ Sync | | update(collection, id, updates, actionInfo = {}, clientAuth = null) | Updates an existing record by ID. Setting record data properties to undefined removes them respectively. Returns the updated record. Throws error if not found. | ✅ Sync | | unset(collection, id, dotPaths = [], actionInfo = {}, clientAuth = null) | Removes specific nested fields using dot notation (e.g., "profile.stats.xp"). Returns updated record. | ✅ Sync | | deleteCollection(collection, actionInfo = {}, clientAuth = null) | Deletes an entire collection. Returns { wholeCollection: true, deleted: true/false }. | ✅ Sync | | deleteRecord(collection, id, actionInfo = {}, clientAuth = null) | Deletes a specific record by ID. Returns { id, deleted: true/false }. | ✅ Sync | | delete(collection, id, actionInfo = {}, clientAuth = null) | Soft-deletes a record by filtering it out and overwriting the collection. Returns { id, deleted: true }. | ✅ Sync | | softDelete(collection, id, { reason = "soft-delete", deletedBy = null } = {}, actionInfo = {}, clientAuth = null) | Mark a record as deleted without removing it. Sets flags: deleted: true, deletedAt, deletedReason, deletedBy. Emits update. Returns { id, softDeleted: true, at, reason, deletedBy }. | ✅ Sync | | restore(collection, id, { restoredBy = null } = {}, actionInfo = {}, clientAuth = null) | Reverse a prior soft delete. Clears deleted* flags and sets restoredAt, restoredBy. Emits update. Returns { id, restored: true, at, restoredBy }. | ✅ Sync |

🧾 StarTransactions (WAL + Idempotency)

Star-Vault handles multi-record transactions with staged commits, WAL logging, and per-step idempotency.

Core Traits

• Logs each TX-BEGIN and TX-COMMIT before writing data.
• Replays committed groups safely after crash or restart.
• Keeps uncommitted changes isolated until commit.

Transaction Methods

// High-level helper
const { transactionID, result } = await vault.transact({ type: "transfer" }, async (t) => {
     await t.update("accounts", "A", { balance: v => v - 100 });
     await t.update("accounts", "B", { balance: v => v + 100 });
     await t.create("ledger", { from: "A", to: "B", amount: 100, ts: Date.now() });
     return "ok";
});

// Manual flow
const tx = await vault.begin({ type: "batch-import" });
await tx.create("users", { id: "u1", name: "Nova" });
await tx.softDelete("posts", "p-123", { reason: "policy", deletedBy: "mod-9" });
await tx.restore("posts", "p-123", { restoredBy: "mod-9" });
await tx.commit(); // or await tx.abort("reason");

| Context | Method | Purpose | Sync/Async | |----------|---------|----------|-------------| | vault | transact(actionInfo, fn) | Open txn, run fn(tx), commit on success, abort on failure. Returns { transactionID, result }. | ⚙️ Async | | vault | begin(actionInfo?, clientAuth?) | Start txn and return handle { id, create, update, unset, deleteRecord, softDelete, restore, commit, abort }. | ⚙️ Async | | tx | create(collection, data, actionInfo?, clientAuth?) | Stage record creation. | ⚙️ Async | | tx | update(collection, id, updates, actionInfo?, clientAuth?) | Stage record update; supports function deltas v => v - 50. | ⚙️ Async | | tx | unset(collection, id, dotPaths = [], actionInfo?, clientAuth?) | Stage removal of nested fields. | ⚙️ Async | | tx | deleteRecord(collection, id, actionInfo?, clientAuth?) | Stage hard deletion. | ⚙️ Async | | tx | softDelete(collection, id, { reason?, deletedBy? } = {}, actionInfo?, clientAuth?) | Stage soft deletion (flags only). | ⚙️ Async | | tx | restore(collection, id, { restoredBy? } = {}, actionInfo?, clientAuth?) | Stage record restoration. | ⚙️ Async | | tx | commit(actionInfo?) | Apply staged ops, refresh cache, emit events, seal WAL. | ⚙️ Async | | tx | abort(reason = "manual") | Discard staging and log TX-ABORT. | ⚙️ Async |

Idempotency and WAL

• Each step logs internal transactionID and deterministic idempotencyKey.
• Steps replay safely on crash recovery; duplicates ignored.

Example Flow

await vault.transact({ actor: "u42" }, async (tx) => {
     await tx.create("users", { id: "42", name: "Orion" });
     await tx.update("profiles", "42", { visits: v => v + 1 });
     await tx.softDelete("sessions", "s-23", { reason: "expired", deletedBy: "system" });
});

🔍 Query Engine

The Query Engine powers Star-Vault’s adaptive data traversal and filtering system — providing a fully chainable, composable query builder with deterministic caching and automatic mode optimization.

Overview

vault.query("collection")
     .whereRecord("id", "value")                    // Filter by root record metadata
     .where({ key : "value" })                      // Filter by record data fields
     .search("field", "text")                       // Text search within a field
     .recent("timestamp", "7d")                     // Filter records from last 7 days
     .near("location", { lat : 0, lng : 0 }, 50)    // Geospatial radius filter
     .sort({ name : 1 })                            // Sort ascending (1) or descending (-1)
     .select(["id", "name"])                        // Return specific fields only
     .limit(10)                                     // Limit number of results
     .offset(0)                                     // Skip first N results
     .filterBy(record => record.active)             // Apply custom JS-level filter
     .callback(row => { row._scanned = true })      // Optional pre-execution side effect
     .return("data")                                                                              //  (Optional) Return only the record data
     .execute(false, "auto");                       // Execute with collection matching and mode control

⚙️ Return Modes

| Method | Description | Default | |---------|-------------|----------| | return("record") | Return the full record object including metadata ({ id, data, timestamp, ... }). | ✅ Default | | return("data") | Return only the user data (record.data). | | | returnFull() | Shorthand for .return("record"). | ✅ |

Execution Modes

| Mode | Description | Ideal Use Case | |------|--------------|----------------| | auto | Automatically selects the best execution path depending on query shape. Defaults to hyper when possible. | General queries | | hyper | Direct single-record retrieval path. Bypasses iteration and uses direct cache lookup for deterministic key-value equality filters. | Exact key lookups (id, userID, etc.) | | default | Standard streaming scan with limit/offset early cutoffs. Supports progressive paging and partial reductions. | Sequential or paginated scans |

⚙️ Star-Vault automatically promotes auto queries to hyper when the filter is deterministic (single key-value pair + limit ≤ 1).

Methods

| Method | Description | Sync/Async | |--------|--------------|------------| | query(collection, actionInfo?) | Begin a query on the specified collection. Returns a chainable QueryBuilder. | ✅ Sync | | whereRecord(criteria) | Filter by root record metadata (record.id, etc.). | ✅ Sync | | where(criteria) | Filter by record data key-value pairs. | ✅ Sync | | search(field, text) | Text search (substring or tokenized) in a specific field. | ✅ Sync | | recent(field, duration) | Filter by recency, e.g. "7d", "30m". | ✅ Sync | | near(field, center, radius) | Filter spatially near a { lat, lng } point. | ✅ Sync | | sort(criteria) | Sort results ascending (1) or descending (-1). | ✅ Sync | | select(fields) | Choose fields to include in the output. | ✅ Sync | | limit(number) | Restrict result count. | ✅ Sync | | offset(number) | Skip first N results for pagination. | ✅ Sync | | filterBy(fn) | Apply a custom filter function to each record. | ✅ Sync | | callback(fn) | Add a hook or side effect before result finalization. | ✅ Sync | | configureAutoIndex(options) | Enables or tunes adaptive auto-indexing for queries that repeatedly miss fast paths. | ✅ Sync | | return(mode) | Control whether queries return "record" (full record) or "data" (just data). | ✅ Sync | | returnFull() | Explicitly reset return mode to "record". | ✅ Sync | | execute(exactCollection = false, executionMode = "auto") | Execute query. If exactCollection = true, restrict to the specified collection only. | ✅ Sync | | getByID(collection, id, actionInfo?) | Retrieve a record directly by primary key. | ✅ Sync | | getManyByID(collection, ids, options?) | Retrieve multiple records by an array of primary keys, preserving input order. | ✅ Sync | | range(min, max) | Static helper to produce inclusive numeric ranges. | ✅ Sync |

Examples

🔸 Full Records (Default)

const orders = vault.query("orders") .where({ status : "shipped" }) .execute(); // Returns [{ id, data, timestamp, ... }]

🔸 Data-Only View

const orders = vault.query("orders") .where({ status : "shipped" }) .return("data") .execute(); // Returns only record.data objects

🔸 Revert to Full Records

const orders = vault.query("orders") .where({ status : "shipped" }) .returnFull() .execute();

🔸 Auto Mode (Adaptive)

const user = vault.query("users")
     .where({ id : "u-1" })
     .select(["name", "email"])
     .execute(false, "auto")[0];

Automatically selects hyper mode due to deterministic equality filter and small result limit.

🔸 Hyper Mode (Accelerated)

const user = vault.query("users")
     .where({ id : "u-1" })
     .limit(1)
     .execute(false, "hyper")[0];

Uses a direct reference lookup path through StarCache for O(1) deterministic retrieval.

🔸 Default Mode (Streaming Scan)

const users = vault.query("users")
     .where({ active : true })
     .offset(100)
     .limit(50)
     .sort({ createdAt : -1 })
     .execute(false, "default");

Uses a sequential scan with early cutoff logic for predictable pagination over large datasets.

🔸 Many-by-ID (order-preserving)

const [ u1, u5, u3 ] = vault.getManyByID("users", ["u-1", "u-5", "u-3"]);

🔸 Advanced Example (Custom Logic & Hooks)

vault.query("orders")
     .where({ status : "pending" })
     .filterBy(order => order.total > 100)
     .callback(order => { order.checked = true })
     .select(["id", "total", "status"])
     .limit(5)
     .execute(false, "auto");

The callback() hook works best for small, customized post-processing operations or live UI effects before final emission.

⚙️ Advanced Controls

configureAutoIndex(options)

Enable or tune adaptive auto-indexing for queries that repeatedly miss fast paths.

Options

• enabled : boolean — Turn auto-indexing on/off.
• missThreshold : number — How many cache/scan “misses” trigger an index build.
• windowMs : number — Rolling time window for counting misses.
• maxConcurrentBuilds : number — Cap background index builds.
• lazyMaxScan : number — Upper bound for a “lazy” scan before promoting to an index.

Example

vault.query("orders")
     .configureAutoIndex({
          enabled : true,
          missThreshold : 8,
          windowMs : 60_000,
          maxConcurrentBuilds : 2,
          lazyMaxScan : 50_000
     })
     .where({ status : "pending" })
     .sort({ createdAt : -1 })
     .limit(100)
     .execute(false, "auto");

Notes

• configureAutoIndex() affects only the current query builder chain.
• The adaptive indexer monitors repetitive scans and builds background indexes once thresholds reach their limits.
• Star-Vault manages concurrency and I/O impact internally—no manual worker orchestration required.
• Auto-indexing integrates with StarCache and never blocks live queries.
• You can disable it in lightweight or memory-constrained deployments without affecting correctness.

Performance & Behavior

• Adaptive Execution Pathing — chooses optimal mode based on filter complexity and limit count.
• Cache-Aware Traversal — leverages in-memory record handles for active collections.
• Early-Cutoff Scanning — truncates scan loops when limits are reached to reduce latency.
• Geospatial and Temporal Filtering — supports proximity, duration, and time-based conditions natively.
• Lightweight Chaining — each query stage mutates minimal state, maintaining sub-millisecond builder overhead.

Developer Note

Query Engine provides direct low-level access to StarVault's compositional access model.
When combined with Hyper-Normalization™, queries can achieve sub-millisecond lookups even across multi-collection normalized schemas.

⚡️ Hyper-Normalization™

Star-Vault introduces Hyper-Normalization™, a revolutionary data-modeling paradigm that unlocks the highest logical purity of data — with near-zero performance penalties.

Definition

Hyper-Normalization™ (noun)  /ˈhaɪ.pər ˌnɔr.mə.laɪˈzeɪ.ʃən/

A data-modeling paradigm introduced by Steven Compton in 2025 (Star-Vault) that implements physical-path normalization, eliminating runtime join planning through deterministic reference resolution and achieving near-constant-time cross-collection traversal under defined consistency and locality models.

First published: 2025-11-04 Coined and defined by Steven Compton, creator of Star-Vault.

🧬 Origin of Hyper-Normalization™

Before Star-Vault, no database truly achieved runtime-level normalization without sacrificing speed.   Relational databases (PostgreSQL, MySQL, Oracle) implemented normalization theoretically through 3NF to 5NF schemas—but still required runtime joins and query planners to reconstruct relationships. Each query carried planner latency, driver overhead, and redundant I/O.

NoSQL systems (MongoDB, Couchbase) took the opposite path—denormalizing data for speed at the cost of consistency and integrity. Developers duplicated or embedded related data, breaking the very principles of normalization.

Columnar and graph databases (ClickHouse, Neo4j, ArangoDB) came close by optimizing joins or reference traversals, yet still relied on runtime evaluation paths or bulk vectorized execution. They never fused structural purity with constant-time resolution.

Star-Vault extends that evolution by redefining how normalization operates in real time.

Instead of chasing faster joins or heavier optimizers, Star-Vault redefined the storage and access model itself. This engine makes normalization a native physical characteristic of data organization, embedding relational integrity into the storage layer itself. It forms a unified data graph that resolves relationships instantly with zero planner cost. Every relationship resolves in near-constant time through StarCache, Star-Vault's in-memory reference engine optimized for deterministic cross-collection access.

This design yields a system where perfect normalization and constant-time performance coexist, eliminating the trade-off between relational purity and real-time scalability.

⚙️ In short: Star-Vault did not just optimize normalized data—it turned normalization itself into the fastest query path in real workloads.

🌌 Why Hyper-Normalization Matters

• 🔗 No denormalization trade-offs — each concept lives in its own collection.
• ⚡ Near-zero-cost references — cross-collection lookups resolve deterministically without runtime joins.
• 🧠 Near constant-time traversal — each relationship resolves through deterministic reference pathways optimized for direct in-memory access, yielding O(1) access.
• 🔐 MVCC-safe snapshots — all reads operate on immutable snapshots for perfect consistency.
• 💾 Physically normalized storage — data and indexes align for locality at the shard and log levels, keeping access ultra-fast.

🧩 Example — The “User Display” Case

Traditional SQL joins:

SELECT users.id, first_names.name, last_names.name, emails.email, usernames.username
FROM users
JOIN first_names ON users.first_name_id = first_names.id
JOIN last_names ON users.last_name_id = last_names.id
JOIN emails ON users.email_id = emails.id
JOIN usernames ON users.username_id = usernames.id
WHERE users.id = 7865789;

Star-Vault’s Hyper-Normalized equivalent:

const user = vault.query("users").where({ userID : 7865789 }).first();

const first  = vault.query("first-names").where({ firstNameID : user.firstNameID }).select(["firstName"]).first();
const last   = vault.query("last-names").where({ lastNameID : user.lastNameID }).select(["lastName"]).first();
const email  = vault.query("emails").where({ emailID : user.emailID }).select(["email"]).first();
const uname  = vault.query("usernames").where({ usernameID : user.usernameID }).select(["username"]).first();

Each lookup executes in microseconds - directly from in-memory StarCache. No planner. No join reordering. No network round-trips in embedded setups. Just raw, deterministic access.

Benchmark & Reproducibility

Test Configuration

To verify Star-Vault’s retrieval performance under normalized, encrypted conditions, a reproducible benchmark evaluated an 8-collection user display composition scenario. Each iteration reconstructs a normalized user profile composed of users, first-names, last-names, emails, usernames, locales, permission-sets, and profiles — all under identical encryption, indexing, and cache settings.

| Parameter | Configuration | |------------|---------------| | Runtime | Node.js 22.0.0 (Windows 11) | | Dataset | 8 facet collections (User composite) | | Access Method | Star-Vault compositional queries and direct retrievals | | Runs | 20 warm runs after 5 warm-ups | | Environment | NVMe SSD, 32 GB RAM | | Encryption | Enabled | | Audit History | Disabled | | Execution Mode | Hyper (automatic optimization) |

Table 1. Test configuration used for Star-Vault performance benchmarks.

Reproducible Benchmark Code

const { performance } = require("node:perf_hooks");

const StarVault = require("@trap_stevo/star-vault");











async function buildUserDisplay_query(vault, userID)
{
     const t0 = performance.now();
     
     const user = vault.query("users").where({ userID }).select([
          "firstNameID","lastNameID","emailID","usernameID",
          "avatarFileID","localeID","permissionSetID","profileID"
     ]).execute(true)[0];
     
     if (!user) { return null; }
     
     const [
          first,last,email,uname,
          avatar,locale,perms,profile
     ] = await Promise.all([
          vault.query("first-names").where({ id : user.firstNameID }).select(["firstName"]).execute(true)[0],
          vault.query("last-names").where({ id : user.lastNameID }).select(["lastName"]).execute(true)[0],
          vault.query("emails").where({ id : user.emailID }).select(["email"]).execute(true)[0],
          vault.query("usernames").where({ id : user.usernameID }).select(["username"]).execute(true)[0],
          vault.starchive ? vault.starchive.records.getByID(user.avatarFileID) : null,
          vault.query("locales").where({ id : user.localeID }).select(["code"]).execute(true)[0],
          vault.query("permission-sets").where({ id : user.permissionSetID }).select(["roles"]).execute(true)[0],
          vault.query("profiles").where({ id : user.profileID }).select(["bio","links"]).execute(true)[0]
     ]);
     
     const display = {
          name : `${first?.firstName || ""} ${last?.lastName || ""}`.trim(),
          email : email?.email || null,
          username : uname?.username || null,
          avatarFile : avatar || null,
          locale : locale?.code || "en",
          roles : perms?.roles || [],
          profile : profile || {}
     };
     
     display._ms = (performance.now() - t0).toFixed(3);
     
     return display;
}

async function buildUserDisplay_getByID(vault, userID)
{
     const t0 = performance.now();
     
     const user = vault.query("users").where({ userID }).select([
          "firstNameID","lastNameID","emailID","usernameID",
          "avatarFileID","localeID","permissionSetID","profileID"
     ]).execute(true)[0];
     
     if (!user) { return null; }
     
     const [
          first,last,email,uname,
          avatar,locale,perms,profile
     ] = await Promise.all([
          vault.getByID("first-names", user.firstNameID),
          vault.getByID("last-names", user.lastNameID),
          vault.getByID("emails", user.emailID),
          vault.getByID("usernames", user.usernameID),
          vault.starchive ? vault.starchive.records.getByID(user.avatarFileID) : null,
          vault.getByID("locales", user.localeID),
          vault.getByID("permission-sets", user.permissionSetID),
          vault.getByID("profiles", user.profileID)
     ]);
     
     const display = {
          name : `${first?.data?.firstName || ""} ${last?.data?.lastName || ""}`.trim(),
          email : email?.data?.email || null,
          username : uname?.data?.username || null,
          avatarFile : avatar || null,
          locale : locale?.data?.code || "en",
          roles : perms?.data?.roles || [],
          profile : profile?.data || {}
     };
     
     display._ms = (performance.now() - t0).toFixed(3);
     
     return display;
}

(async function main()
{
     const sV = new StarVault("./data","./logs",4,"869MB","1w",{
          enableEncryption : true,
          masterKey : "starvault-demo",
          auditHistory : false,
          debugOptions : {
               displayEventEmissions : false,
               displayRecordQueryErrors : false,
               displayCollectionQueryErrors : false
          }
     });
     
     const first = sV.create("first-names",{ id : "fn-1", firstName : "Nova" });
     
     const last  = sV.create("last-names",{ id : "ln-1", lastName  : "Orion" });
     
     const email = sV.create("emails",{ id : "em-1", email : "[email protected]" });
     
     const uname = sV.create("usernames",{ id : "un-1", username : "nova" });
     
     const locale = sV.create("locales",{ id : "lc-1", code : "en" });
     
     const perms = sV.create("permission-sets",{ id : "ps-1", roles : ["user","beta"] });
     
     const profile = sV.create("profiles",{ id : "pr-1", bio : "Explorer", links : ["https://example.com/nova"] });
     
     sV.create("users",{
          userID : "u-1",
          firstNameID : first.id,
          lastNameID : last.id,
          emailID : email.id,
          usernameID : uname.id,
          avatarFileID : "file-001",
          localeID : locale.id,
          permissionSetID : perms.id,
          profileID : profile.id
     });
     
     
     
     // Warm-up
     await buildUserDisplay_query(sV,"u-1");
     
     await buildUserDisplay_getByID(sV,"u-1");
     
     
     
     // Benchmarks
     const runBench = async (label, fn) => {
          const runs = 20;
          
          let best = Infinity, worst = 0, total = 0;
          
          for (let i=0;i<runs;i++)
          {
               const r = await fn();
               const t = Number(r?._ms || "0");
               best = Math.min(best,t);
               worst = Math.max(worst,t);
               total += t;
          }
          
          const avg = (total/runs).toFixed(3);
          
          console.log(`${label}:`,{ runs, best_ms:best.toFixed(3), avg_ms:avg, worst_ms:worst.toFixed(3) });
     };
     
     await runBench("Warm compose (query)",()=>buildUserDisplay_query(sV,"u-1"));
     
     await runBench("Warm compose (getByID)",()=>buildUserDisplay_getByID(sV,"u-1"));
})();

Results

| Method | Best (ms) | Avg (ms) | Worst (ms) | |---------|-----------|-----------|------------| | Warm compose (query) | 3.285 | 4.384 | 5.922 | | Warm compose (getByID) | 0.094 | 0.124 | 0.202 |

Table 2. Warm benchmark results comparing compositional queries and direct primary-key retrievals in Star-Vault.

Analysis

Star-Vault demonstrates sub-5 ms average performance for compositional (multi-facet) reads and sub-0.1 ms primary-key retrievals — both fully encrypted and cache-synchronized.
These results confirm the effectiveness of Star-Vault’s Hyper Normalization™ model, achieving consistent, near-instant retrieval performance across normalized datasets.

Interpretation

Even in a cold state, Star-Vault composes a fully normalized, eight-collection user display in milliseconds.
Once warmed, the deterministic cache layer maintains micro-scale latency across repeated traversals — a direct realization of Hyper Normalization™ in practice.

Cross-Hardware Consistency

Star-Vault’s traversal and caching models operate deterministically, scale linearly with CPU frequency, and maintain architecture neutrality across device classes.
Performance remains stable on laptops, servers, and embedded systems, with variance driven primarily by traversal depth rather than hardware architecture.

Result Variability Notice

Actual timings may vary slightly depending on:

• CPU frequency and thermal scaling
• Storage device performance and I/O queue depth
• Background processes and concurrent file access

However, Star-Vault’s normalized, cache-aware execution model ensures that relative performance ratios remain consistent across hardware configurations — preserving the same latency hierarchy between compositional and direct retrieval paths.

Key Takeaway

Star-Vault achieves ≈ 4.4 ms multi-facet reads and ≈ 0.094 ms direct retrievals on standard NVMe hardware — establishing a new benchmark for encrypted, normalized, cache-optimized persistence systems.

Benchmark Notes & Scope

The benchmark evaluates deterministic multi-record composition and retrieval under encrypted, normalized, and cache-warmed conditions - demonstrating Star-Vault's optimized read-path architecture. The test focuses on real-world access behavior rather than synthetic concurrency stress, emphasizing traversal consistency, normalization efficiency, and encryption overhead neutrality.

🚀 Performance Implications

• ✅ Achieves O(1) cross-collection resolution for defined reference paths; depth and topology influence total latency.
• ✅ Near-zero integrity overhead - deterministic reference synchronization without relational cost.
• ✅ Linear scaling with CPU, memory, and shard count.
• ✅ Deterministic read and write paths for predictable real-time performance.

💡 In essence:
Hyper-Normalization transforms normalization from a theoretical ideal into a physical performance advantage - proving that data integrity and speed can coexist with minimal compromise for the targeted access patterns.

🛡️ Authentication

All methods below require authOptions configuration and are used for session/user management.

👤 User Registration & Authentication

| Method | Description | Sync/Async | |--------|-------------|------------| | registerUser(email, password, actionInfo, clientAuth) | Registers a new user. . | ⚙️ Async | | registerGuestUser(req, metadata, actionInfo, clientAuth)| Registers a temporary guest user.. | ⚙️ Async | | loginGuestUser(accountID, req, actionInfo, clientAuth) | Logs in a guest user and creates a session. . | ⚙️ Async | | loginUser(email, password, req, actionInfo, clientAuth)| Logs in a user and creates a session.. | ⚙️ Async | | resumeUserSession(accountID, req, actionInfo, clientAuth) | Resumes session from account ID (used for guest linking or session carryover). | ⚙️ Async | | upgradeGuestUser(accountID, email, password, actionInfo, clientAuth) | Converts a guest account to a full registered user. | ⚙️ Async |

🛰️ Session Management

| Method | Description | Sync/Async | |--------|-------------|------------| | logoutUser(token, actionInfo, clientAuth) | Logs out a user by invalidating their session token. | ✅ Sync | | extendUserSession(token, ms, actionInfo, clientAuth) | Extends an active session's duration. | ✅ Sync | | cleanupExpiredAuthTokens(actionInfo, clientAuth) | Cleans up expired tokens and invalid sessions. | ✅ Sync | | cleanupInactiveGuests(actionInfo, clientAuth) | Removes or deactivates inactive guest accounts. | ⚙️ Async |

🔑 Password & Magic Link Recovery

| Method | Description | Sync/Async | |--------|-------------|------------| | updateUserPassword(userID, newPassword, actionInfo, clientAuth) | Updates a user's password. | ⚙️ Async | | requestUserPasswordReset(email, actionInfo, clientAuth) | Sends a password reset token. | ⚙️ Async | | resetUserPassword(token, newPassword, actionInfo, clientAuth) | Resets password using a valid token. | ⚙️ Async | | requestUserStellarLink(email, type, actionInfo, clientAuth) | Sends a Stellar link (magic link or code) for login or verification. | ⚙️ Async | | consumeUserStellarToken(token, type, req, actionInfo, clientAuth) | Consumes a Stellar link/token to verify user and return session. | ⚙️ Async |

👥 Guest Account Management

| Method | Description | Sync/Async | |--------|-------------|------------| | registerGuestUser(req, metadata, actionInfo, clientAuth) | Registers a guest session with optional request/metadata. | ⚙️ Async | | trackGuestActivity(userID, timestamp, actionInfo, clientAuth) | Updates guest user's lastActiveAt timestamp. | ⚙️ Async | | upgradeGuestUser(accountID, email, password, actionInfo, clientAuth) | Promotes a guest to a full registered account. | ⚙️ Async | | cleanupInactiveGuests(actionInfo, clientAuth) | Cleans up inactive or stale guest users based on threshold. | ⚙️ Async |

🔁 User Lifecycle Management

| Method | Description | Sync/Async | |--------|-------------|------------| | updateUser(accountID, updatedMetadata, accountType, actionInfo, clientAuth) | Updates a user by Account ID. Does not allow for password updates. | ⚙️ Async | | deleteUser(userID, reason, actionInfo, clientAuth) | Soft-deletes a user by ID. | ⚙️ Async | | deleteUserByAccountID(accountID, reason, actionInfo, clientAuth) | Soft-deletes a user using their account ID. | ⚙️ Async | | reactivateUser(userID, actionInfo, clientAuth) | Reactivates a previously deleted user. | ⚙️ Async | | reactivateUserByAccountID(accountID, actionInfo, clientAuth) | Reactivates a deleted user by account ID. | ⚙️ Async |

📊 User & Session Querying

| Method | Description | Sync/Async | |--------|-------------|------------| | getUserByAccountID(accountID, filter, accountType, actionInfo) | Retrieves a user by account id. If a certain account type (default ~ guest). | ⚙️ Async | | getUser(userID, filter, accountType, actionInfo) | Retrieves a user by id. If a certain account type (default ~ guest). | ⚙️ Async | | getUserSession(sessionID, actionInfo) | Retrieves a session by id. | ⚙️ Async | | listSessions(userID, filters, includeInactive, actionInfo) | Lists sessions for a specific user. | ⚙️ Async | | listAllSessions(filters, includeInactive, actionInfo) | Lists all sessions across users. | ⚙️ Async | | listUsers(filters, options, actionInfo) | Lists users based on filters and options (pagination, includes). | ⚙️ Async | | listUserActivity(options, actionInfo) | Lists recent activity, including filtering by active, inactive, or recently seen users. | ⚙️ Async |

🧩 Internal Validation & Utility Methods

The following utility methods get utilized internally across authentication-related flows. You may call them directly for validation, diagnostics, or extension purposes.

| Method | Description | Sync/Async | |--------------------------------|-----------------------------------------------------------------------------------------------|-------------------------------------------------------------------------| | validEmail(email) | Determines if an input email follows proper email address format. | ✅ Sync | | satisfiesPasswordStrength(password) | Validates whether the password meets the defined passwordRequirements. | ✅ Sync | | extractSessionMetadata(req) | Extracts metadata (IP, fingerprint, user agent, etc.) from the request object . | ✅ Sync | | lookupUserGeo(ip) | Performs geo-IP lookup using built-in and custom geo services . | ⚙️ Async |

🧭 Record History & Timeline

Star-Vault reconstructs record history and a human-readable timeline from write-ahead logs (WAL). These calls support filtering, integrity checks, and encryption-aware output.

Methods

| Method | Description | Sync/Async | |---|---|---| | getRecordHistory(collection, id, options = {}, actionInfo = {}, clientAuth = null) | Returns detailed event entries for a single record in time order. | ✅ Sync | | getRecordTimeline(collection, id, options = {}, actionInfo = {}, clientAuth = null) | Returns a compact timeline view (action, timestamp, brief summary). | ✅ Sync | | getCollectionHistory(collection, options = {}, actionInfo = {}, clientAuth = null) | Returns all WAL entries for a collection, including multi-record activity. | ✅ Sync | | getCollectionTimeline(collection, options = {}, actionInfo = {}, clientAuth = null) | Returns a summarized timeline for collection-level changes. | ✅ Sync | | getVaultHistory(options = {}, actionInfo = {}, clientAuth = null) | Returns all WAL entries across the vault, with filters for collections and actions. | ✅ Sync | | syncHistory(actionInfo = {}, clientAuth = null) | Flushes WAL streams and closes active handles to support backups, exports, and shutdown flows. | ⚙️ Async |

All history/timeline methods share the same options contract below, including collection-level and vault-level calls where applicable.

options (history/timeline)

| Key | Type | Default | Notes | |---|---|---|---| | includeArchived | boolean | true | Include rotated *.archived files when present. | | verifyChecksums | boolean | true | Verify each entry’s SHA-256 checksum of data; skip failures. | | includeSystem | boolean | true | Merge related system.log entries for the same collection/id (e.g., DELETE, TX events). | | actions | string | string[] | undefined | Allow-list of actions to include (e.g., ["WRITE","UPDATE"]). Case-insensitive. | | excludeActions | string | string[] | undefined | Block-list of actions to exclude (e.g., ["DELETE"]). Case-insensitive. | | since | number | string | Date | undefined | Include entries at/after this time (ms epoch, ISO string, or Date). | | until | number | string | Date | undefined | Include entries at/before this time. | | match | (row) => boolean | undefined | Custom predicate that keeps/discards rows after other filters. | | includeDecrypted | boolean | (defaults to true when a decryptor exists; otherwise false) | When true and encryption enabled, decrypt rows and populate dataDecrypted. | | audit | boolean | undefined | Per-call audit switch; when true, writes an audit entry (see Auditing). |

ℹ️ History entries include source, collection, action, id, timestamp, client, data, checksum, plus eventID, transactionID, idempotencyKey when present. With includeDecrypted and a decryptor, output also includes dataEncrypted and dataDecrypted.

Audit Examples

// Full history (archived segments + checksum verification)
const history = vault.getRecordHistory("users/region1", "1760233994625", {
  includeArchived : true,
  verifyChecksums : true
}, { actor : "admin-1" }, { token : "valid" });

// Windowed, updates only
const updates = vault.getRecordHistory("users/region1", "1760233994625", {
  actions : "UPDATE",
  since : Date.now() - 60_000,
  until : Date.now()
});

// Compact timeline, no deletions, explicit audit
const