@cmdoss/memwal-sdk

v1.0.0

Published

25 days ago

TypeScript SDK for Personal Data Wallet - Decentralized memory system with AI embeddings, HNSW vector search, SEAL encryption and Walrus storage

@cmdoss/memwal-sdk

MemWal (Memory + Walrus) - TypeScript SDK for decentralized memory storage on Sui blockchain with Walrus.

Contract Addresses

| Network | Package ID | |---------|------------| | Testnet | 0xa5d7d98ea41620c9aaf9f13afa6512455d4d10ca06ccea3f8cd5b2b9568e3a9e | | Mainnet | Coming soon |

Features

| Feature | Description | |---------|-------------| | Memory CRUD | Create, read, search, delete memories on Sui + Walrus | | Unified Search | pdw.memory.search() - auto-embeds query and searches | | SEAL Encryption | Identity-based encryption v2.2 (enabled by default) | | Vector Search | Fast HNSW search with IndexedDB persistence (browser) | | AI Classification | Auto-categorize memories (fact, event, preference, etc.) | | Batch Upload | ~90% gas savings with Quilt batching | | Knowledge Graph | Entity and relationship extraction | | IndexedDB Storage | Persistent local vector index in browser |

Installation

npm install @cmdoss/memwal-sdk @mysten/sui @mysten/dapp-kit

Quick Start

Minimal Example (Node.js Server)

import { SimplePDWClient } from '@cmdoss/memwal-sdk';
import { Ed25519Keypair } from '@mysten/sui/keypairs/ed25519';

const PACKAGE_ID = '0xa5d7d98ea41620c9aaf9f13afa6512455d4d10ca06ccea3f8cd5b2b9568e3a9e';

// Create keypair (for server-side automation)
const keypair = Ed25519Keypair.generate();
console.log('Address:', keypair.toSuiAddress());

const pdw = new SimplePDWClient({
  signer: keypair,
  network: 'testnet',
  sui: { packageId: PACKAGE_ID },
  embedding: {
    provider: 'openrouter',
    apiKey: process.env.OPENROUTER_API_KEY!,
  }
});

await pdw.ready();

// Create & Search
const memory = await pdw.memory.create('I love programming');
const results = await pdw.memory.search('coding', { limit: 5 });

React/Next.js (with Sui Wallet)

import { SimplePDWClient, DappKitSigner } from '@cmdoss/memwal-sdk/browser';
import { useCurrentAccount, useSignAndExecuteTransaction, useSuiClient } from '@mysten/dapp-kit';

const PACKAGE_ID = '0xa5d7d98ea41620c9aaf9f13afa6512455d4d10ca06ccea3f8cd5b2b9568e3a9e';

function MemoryComponent() {
  const account = useCurrentAccount();
  const suiClient = useSuiClient();
  const { mutateAsync: signAndExecute } = useSignAndExecuteTransaction();

  const saveMemory = async (content: string) => {
    // Create signer - pass signAndExecute directly
    const signer = new DappKitSigner({
      address: account!.address,
      client: suiClient,
      signAndExecuteTransaction: signAndExecute,
    });

    const pdw = new SimplePDWClient({
      signer,
      network: 'testnet',
      userAddress: account!.address,
      sui: { packageId: PACKAGE_ID },
      features: {
        enableLocalIndexing: false, // Browser: use server-side indexing
        enableEncryption: true,
      },
    });

    await pdw.ready();
    return await pdw.memory.create(content);
  };

  return <button onClick={() => saveMemory('Hello World!')}>Save Memory</button>;
}

Note: Browser SDK doesn't include embedding service. For AI features (embeddings, classification), call your server API which uses the full SDK.

Environment Variables

# .env (Browser/Next.js)
NEXT_PUBLIC_OPENROUTER_API_KEY=sk-or-v1-...  # For AI embeddings

# .env (Node.js server - optional)
OPENROUTER_API_KEY=sk-or-v1-...              # For server-side embeddings

API Reference

Core Namespaces

The SDK provides 5 core namespaces for common operations:

pdw.memory   // Create, get, search, list, delete memories
pdw.ai       // Embed, classify, extract commands
pdw.index    // HNSW index management
pdw.wallet   // Wallet address and balance info
pdw.advanced // Power user features (graph, analytics, etc.)

`pdw.memory` - Memory Operations

// CREATE - handles everything internally (embed, encrypt, upload, register, index)
const memory = await pdw.memory.create(content, {
  category?: string,      // Auto-classify if not provided
  importance?: number,    // Auto-score if not provided (1-10)
  embedding?: number[],   // Auto-generate if not provided
});

// CREATE BATCH - upload multiple memories efficiently
const memories = await pdw.memory.createBatch([
  { content: 'Memory 1', category: 'fact' },
  { content: 'Memory 2', category: 'preference' }
]);

// SEARCH - unified semantic search (auto-embeds query)
const results = await pdw.memory.search(query, {
  limit?: number,         // Max results (default: 10)
  threshold?: number,     // Min similarity 0-1 (default: 0.7)
  category?: string,      // Filter by category
  includeContent?: boolean // Include decrypted content (default: true)
});

// GET - retrieve by ID (auto-decrypts)
const memory = await pdw.memory.get(memoryId);

// LIST - list all memories with filters
const memories = await pdw.memory.list({
  category?: string,
  limit?: number,
  offset?: number
});

// DELETE
await pdw.memory.delete(memoryId);
await pdw.memory.deleteBatch([id1, id2, id3]);

`pdw.ai` - AI Operations

// Generate embeddings
const embedding = await pdw.ai.embed(text);
const embeddings = await pdw.ai.embedBatch([text1, text2]);

// Classify content
const { category, importance } = await pdw.ai.classify(content);

// Extract memory commands from user input
const memories = pdw.ai.extractMultipleMemories(userMessage);
// Input: "remember I like pizza and my name is John"
// Output: ["I like pizza", "my name is John"]

// Check if content should be saved as memory
const shouldSave = await pdw.ai.shouldSave(content);

`pdw.index` - HNSW Index Management

// Add vector to index
await pdw.index.add(spaceId, vectorId, vector, {
  content: string,
  blobId: string,
  category: string,
  importance: number,
  isEncrypted: boolean,
  forceStoreContent: boolean  // Store content even when encrypted (for RAG)
});

// Search vectors
const results = await pdw.index.search(spaceId, queryVector, {
  k: 10,
  threshold: 0.7
});

// Get index stats
const stats = pdw.index.getStats(spaceId);

// Rebuild index from blockchain
await pdw.index.rebuild(userAddress);

// Clear index
await pdw.index.clear(spaceId);

// Flush to disk
await pdw.index.flush(spaceId);

`pdw.wallet` - Wallet Information

// Get wallet address
const address = pdw.wallet.address;

// Get SUI balance
const balance = await pdw.wallet.balance();

// Get memory count
const count = await pdw.wallet.memoryCount();

`pdw.advanced` - Power User Features

// Knowledge Graph
const graph = await pdw.advanced.graph.extract(content);
// Returns: { entities: [...], relationships: [...] }

// Analytics
const insights = await pdw.advanced.analytics.getInsights();

// Manual encryption/decryption
const encrypted = await pdw.advanced.encryption.encrypt(data);
const decrypted = await pdw.advanced.encryption.decrypt(encrypted);

// Permissions
await pdw.advanced.permissions.grant(memoryId, targetAddress);
await pdw.advanced.permissions.revoke(memoryId, targetAddress);

// Transaction building (low-level)
const tx = pdw.advanced.blockchain.buildCreateMemoryTx(params);

Legacy Search API (Deprecated)

// Still works but deprecated - use pdw.memory.search() instead
await pdw.search.vector(query, { limit });      // -> pdw.memory.search()
await pdw.search.byCategory('fact');            // -> pdw.memory.list({ category: 'fact' })
await pdw.search.hybrid(query, { category });   // -> pdw.memory.search(query, { category })

Configuration

Full Configuration

const pdw = new SimplePDWClient({
  // Required
  signer: keypair,                // Sui keypair or wallet adapter
  network: 'testnet',             // 'testnet' | 'mainnet' | 'devnet'

  // Embedding configuration (required for AI features)
  embedding: {
    provider: 'openrouter',       // 'google' | 'openai' | 'openrouter' | 'cohere'
    apiKey: process.env.OPENROUTER_API_KEY,
    modelName: 'google/text-embedding-004',  // Optional: defaults per provider
    dimensions: 768               // Optional: 768 (default), 1536, or 3072
  },

  // Optional: Sui configuration
  sui: {
    packageId: '0x...',           // Default: from env PACKAGE_ID
    rpcUrl: 'https://...',        // Default: from network
  },

  // Optional: Walrus configuration
  walrus: {
    aggregatorUrl: 'https://aggregator.walrus-testnet.walrus.space',
    publisherUrl: 'https://publisher.walrus-testnet.walrus.space',
  },

  // Optional: AI configuration
  ai: {
    apiKey: process.env.OPENROUTER_API_KEY,
    chatModel: 'google/gemini-2.5-flash',
  },

  // Optional: Feature flags
  features: {
    enableEncryption: true,       // Default: true (SEAL encryption)
    enableLocalIndexing: true,    // Default: true (HNSW vector search)
    enableKnowledgeGraph: true    // Default: true (entity extraction)
  },

  // Optional: Encryption configuration
  encryption: {
    enabled: true,
    keyServers: ['0x...', '0x...'],  // Default: testnet key servers
    threshold: 2,                    // M of N key servers required
    accessRegistryId: '0x...'
  },

  // Optional: Index backup to Walrus (cloud sync)
  indexBackup: {
    enabled: true,
    aggregatorUrl: 'https://...',
    publisherUrl: 'https://...',
    autoSync: false,
    epochs: 3
  }
});

All Environment Variables Reference

# Package ID (Testnet)
PACKAGE_ID=0xa5d7d98ea41620c9aaf9f13afa6512455d4d10ca06ccea3f8cd5b2b9568e3a9e

# Embedding Configuration
EMBEDDING_PROVIDER=openrouter    # google, openai, openrouter, cohere
EMBEDDING_API_KEY=               # Falls back to provider-specific keys
EMBEDDING_MODEL=                 # Optional: override default model
EMBEDDING_DIMENSIONS=768         # 768 (default), 1536, 3072

# Provider-specific API keys (fallback)
OPENROUTER_API_KEY=sk-or-v1-...
GEMINI_API_KEY=...
OPENAI_API_KEY=sk-...
COHERE_API_KEY=...

# AI Chat Model (for RAG responses, knowledge graph extraction)
AI_CHAT_MODEL=google/gemini-2.5-flash

# Walrus (optional - defaults to testnet)
WALRUS_NETWORK=testnet
WALRUS_AGGREGATOR=https://aggregator.walrus-testnet.walrus.space
WALRUS_PUBLISHER=https://publisher.walrus-testnet.walrus.space

# SEAL Encryption (optional - defaults configured)
ENABLE_ENCRYPTION=true
SEAL_KEY_SERVERS=0xKEY1,0xKEY2
SEAL_THRESHOLD=2
ACCESS_REGISTRY_ID=0x...

Embedding Providers

| Provider | Model Default | Dimensions | API Key | |----------|---------------|------------|---------| | openrouter | google/text-embedding-004 | 768 | OPENROUTER_API_KEY | | google | text-embedding-004 | 768 | GEMINI_API_KEY | | openai | text-embedding-3-small | 1536 | OPENAI_API_KEY | | cohere | embed-english-v3.0 | 1024 | COHERE_API_KEY |

Dimensions Trade-offs:

768 (default): Fast embedding + small storage + fast search
1536: Balance of quality and speed
3072: Highest quality, slowest

Recommendation: Use openrouter with 768 dimensions for best performance.

AI Model Configuration

The SDK uses centralized defaults that can be customized via environment variables or code.

Configuration Priority

User Config (code) → Environment Variable → SDK Default

SDK Defaults

import { MODEL_DEFAULTS, getChatModel } from '@cmdoss/memwal-sdk';

// View default values
console.log(MODEL_DEFAULTS);
// {
//   EMBEDDING_OPENROUTER: 'google/gemini-embedding-001',
//   EMBEDDING_GOOGLE: 'text-embedding-004',
//   EMBEDDING_OPENAI: 'text-embedding-3-small',
//   EMBEDDING_COHERE: 'embed-english-v3.0',
//   CHAT_MODEL: 'google/gemini-2.5-flash',
//   EMBEDDING_DIMENSIONS: 768,
// }

// Get current chat model (checks env first, then default)
const model = getChatModel();  // 'google/gemini-2.5-flash'

Method 1: Environment Variables (Recommended)

# .env
# Embedding
EMBEDDING_PROVIDER=openrouter
EMBEDDING_MODEL=google/gemini-embedding-001
EMBEDDING_DIMENSIONS=768

# Chat/Analysis (RAG, knowledge graph)
AI_CHAT_MODEL=google/gemini-2.5-flash

Method 2: Code Configuration

const pdw = new SimplePDWClient({
  signer: keypair,
  network: 'testnet',

  // Embedding config
  embedding: {
    provider: 'openrouter',
    apiKey: process.env.OPENROUTER_API_KEY!,
    modelName: 'google/gemini-embedding-001',  // Override default
    dimensions: 1536,  // Override default 768
  },

  // AI chat config
  ai: {
    apiKey: process.env.OPENROUTER_API_KEY!,
    chatModel: 'google/gemini-2.5-pro',  // Override default
  },
});

Available Models

| Purpose | Environment Variable | Default | Alternatives | |---------|---------------------|---------|--------------| | Embedding | EMBEDDING_MODEL | google/gemini-embedding-001 | text-embedding-3-small, embed-english-v3.0 | | Chat/RAG | AI_CHAT_MODEL | google/gemini-2.5-flash | google/gemini-2.5-pro, openai/gpt-4o, anthropic/claude-3.5-sonnet |

Model Selection Tips

| Model | Speed | Quality | Cost | Best For | |-------|-------|---------|------|----------| | google/gemini-2.5-flash | ⚡ Fast | Good | $ | Default, most tasks | | google/gemini-2.5-pro | Medium | ⬆️ Better | $$ | Complex reasoning | | openai/gpt-4o-mini | ⚡ Fast | Good | $ | OpenAI alternative | | openai/gpt-4o | Slow | ⬆️ Best | $$$ | Highest quality | | anthropic/claude-3.5-sonnet | Medium | ⬆️ Better | $$ | Balanced |

SEAL Encryption

Encryption is enabled by default in v1.0.0. All memory content and embeddings are automatically encrypted using SEAL v2.2 (Secure Encrypted Access Layer with capability-based access).

How It Works

Identity-based: Uses Sui address as encryption identity
Threshold security: Requires M of N key servers (default: 2 of 2)
Zero gas for decryption: No blockchain transactions required
Privacy-preserving: Content never exposed on-chain

Default Encryption (Recommended)

const pdw = new SimplePDWClient({
  signer: keypair,
  network: 'testnet',
  embedding: { provider: 'openrouter', apiKey: 'key' }
  // Encryption automatically enabled!
});

// Memories encrypted on upload, decrypted on retrieval
await pdw.memory.create('My private data');

Disabling Encryption (Development Only)

const pdw = new SimplePDWClient({
  // ...
  features: {
    enableEncryption: false  // Stores content in plaintext!
  }
});

Server-Side RAG

For server-side applications with RAG (Retrieval-Augmented Generation), content needs to be stored in the local index even when encrypted.

Configuration

When indexing memories server-side, use forceStoreContent: true:

await pdw.index.add(walletAddress, vectorId, embedding, {
  content: plaintextContent,
  blobId: blobId,
  isEncrypted: true,
  forceStoreContent: true  // Store content for RAG even when encrypted
});

API Route Example

// /api/memory/index/route.ts
export async function POST(req: Request) {
  const { walletAddress, memoryId, content, embedding, blobId } = await req.json();

  const pdw = await getReadOnlyPDWClient(walletAddress);

  await pdw.index.add(walletAddress, vectorId, embedding, {
    content,
    blobId,
    isEncrypted: true,
    forceStoreContent: true  // Enable RAG for encrypted memories
  });
}

Chat API with Memory Search

// /api/chat/route.ts
export async function POST(req: Request) {
  const { messages, walletAddress } = await req.json();
  const pdw = await getReadOnlyPDWClient(walletAddress);

  // Search memories for context
  const memories = await pdw.memory.search(userMessage, {
    limit: 10,
    threshold: 0.3,
    includeContent: true
  });

  // Build prompt with memory context
  const systemPrompt = `
    User's memories:
    ${memories.map(m => m.content).join('\n')}
  `;

  // Call LLM with context
  return streamText({ model, messages, system: systemPrompt });
}

Vector Search (HNSW)

The SDK uses HNSW for fast approximate nearest neighbor search:

| Implementation | Environment | Performance | Storage | |----------------|-------------|-------------|---------| | hnswlib-node | Node.js | Fastest (native C++) | File system | | hnswlib-wasm | Browser + Node.js | Good (fallback) | IndexedDB |

The SDK auto-detects and uses the best available implementation.

Browser IndexedDB Persistence

In browsers, the vector index is automatically persisted to IndexedDB:

Database: pdw-hnsw-index
Store: vectors (per user wallet address)
Auto-save: After each memory creation
Auto-load: On SDK initialization

This means users don't lose their local search index when refreshing the page or closing the browser.

For Best Performance in Node.js

# Requires C++ build tools
npm install hnswlib-node

| Platform | Command | |----------|---------| | Windows | Install VS Build Tools with C++ workload | | macOS | xcode-select --install | | Linux | sudo apt-get install build-essential python3 |

Index Rebuild

When users log in on a new device or the local index is lost, rebuild from blockchain:

import { rebuildIndexNode, hasExistingIndexNode } from '@cmdoss/memwal-sdk';
import { SuiClient, getFullnodeUrl } from '@mysten/sui/client';

const client = new SuiClient({ url: getFullnodeUrl('testnet') });

// Check if index exists
const hasIndex = await hasExistingIndexNode(userAddress);

if (!hasIndex) {
  // Rebuild from blockchain + Walrus
  const result = await rebuildIndexNode({
    userAddress,
    client,
    packageId: process.env.PACKAGE_ID!,
    network: 'testnet',
    force: false,  // Set true to force rebuild
    fetchConcurrency: 10,  // Parallel blob fetches
    onProgress: (current, total, status) => {
      console.log(`${current}/${total}: ${status}`);
    }
  });

  console.log(`Indexed ${result.indexedMemories}/${result.totalMemories} memories`);
}

Exports

// Main exports
import { SimplePDWClient } from '@cmdoss/memwal-sdk';

// Browser-safe exports
import { SimplePDWClient, DappKitSigner } from '@cmdoss/memwal-sdk/browser';

// React hooks
import { usePDWClient, useMemory } from '@cmdoss/memwal-sdk/hooks';

// Services (advanced)
import { EmbeddingService, MemoryIndexService } from '@cmdoss/memwal-sdk/services';

// LangChain integration
import { PDWVectorStore } from '@cmdoss/memwal-sdk/langchain';

// Vercel AI SDK integration
import { createPDWTool } from '@cmdoss/memwal-sdk/ai-sdk';

// Node.js utilities
import { rebuildIndexNode, hasExistingIndexNode, clearIndexNode } from '@cmdoss/memwal-sdk';

Benchmarks

Tested on localhost with encryption enabled:

| Operation | Time | Description | |-----------|------|-------------| | Memory Search | ~0.5s | Semantic search (local HNSW) | | Create Memory | ~2.5s | Classify + embed + encrypt + upload + index | | AI Classification | ~0.8s | Categorize content | | Batch Upload (5 items) | ~3s | Quilt batching with encryption | | Blockchain Query | ~0.3s | List memories from Sui |

Documentation

ARCHITECTURE.md - System design and data flow
BENCHMARKS.md - Detailed performance metrics
CHANGELOG.md - Version history

License

MIT

Published

Vulnerabilities

Links

Maintainers

Keywords

Readme

@cmdoss/memwal-sdk

Contract Addresses

Features

Installation

Quick Start

Minimal Example (Node.js Server)

React/Next.js (with Sui Wallet)

Environment Variables

API Reference

Core Namespaces

pdw.memory - Memory Operations

pdw.ai - AI Operations

pdw.index - HNSW Index Management

pdw.wallet - Wallet Information

pdw.advanced - Power User Features

Legacy Search API (Deprecated)

Configuration

Full Configuration

All Environment Variables Reference

Embedding Providers

AI Model Configuration

Configuration Priority

SDK Defaults

Method 1: Environment Variables (Recommended)

Method 2: Code Configuration

Available Models

Model Selection Tips

SEAL Encryption

How It Works

Default Encryption (Recommended)

Disabling Encryption (Development Only)

Server-Side RAG

Configuration

API Route Example

Chat API with Memory Search

Vector Search (HNSW)

Browser IndexedDB Persistence

For Best Performance in Node.js

Index Rebuild

Exports

Benchmarks

Documentation

License

`pdw.memory` - Memory Operations

`pdw.ai` - AI Operations

`pdw.index` - HNSW Index Management

`pdw.wallet` - Wallet Information

`pdw.advanced` - Power User Features