@asaidimu/utils-workspace

Content-addressed workspace and conversation management for AI applications.

📚 Table of Contents

• Overview & Features
• Installation & Setup
• Usage Documentation
• Project Architecture
• Development & Contributing
• Additional Information

Overview & Features

@asaidimu/utils-workspace is a TypeScript library that powers AI‑assisted workspaces. It provides a complete solution for managing content‑addressed binary blobs, conversation transcripts with branching and editing, workspace state (roles, preferences, context), and token‑aware prompt assembly. The library is built to be backend‑agnostic (supports both in‑memory and IndexedDB persistence) and offline‑first by default.

Why this library? Modern AI applications need to handle large binary attachments (images, documents) efficiently, support non‑linear conversations (branching, editing), and respect token budgets when sending prompts to LLMs. @asaidimu/utils-workspace solves these challenges with a content‑addressed storage model, a turn DAG, and a pluggable prompt builder.

Key Features

• Content‑Addressed Blob Storage – Store blobs by SHA‑256 hash; automatic deduplication and reference counting.
• Blob Lifecycle Management – Ref counting with lazy eviction and explicit garbage collection. Record remote file IDs after upload to providers.
• Turn Tree & Session Management – Maintain conversation trees with parent/child relationships, edits, and branching. Auto‑link parent turns when appending.
• Dirty Buffer & Auto‑Flush – Buffer unsaved turns in memory and flush when a size threshold is reached or on session close.
• Token‑Aware Prompt Building – Build prompts with a pluggable token planner, context retriever (Jaccard similarity + freshness), and summarizer. Allocate budget across instructions, persona, preferences, transcript, and context.
• Pluggable Components – Swap context retriever, token planner, summarizer, and assembler to fit your use case.
• Multi‑Backend Support – MemoryStorage / MemoryBlobStorage for development/testing, IndexedDBStorage / IndexedDBBlobStorage for persistent browser storage.
• Workspace Commands & Reducer – Manage index state via a pure reducer; side effects are handled by WorkspaceManager.

Installation & Setup

Prerequisites

• Node.js 18+ or a modern browser with IndexedDB support.
• TypeScript 5.0+ (if using in a TS project).

Installation

npm install @asaidimu/utils-workspace

Configuration

The library works out of the box with in‑memory storage. For persistence, set up IndexedDB backends:

import { IndexedDBStorage, IndexedDBBlobStorage, ContentStore, BlobStore, WorkspaceManager } from '@asaidimu/utils-workspace';

// Persistent content storage
const contentBackend = new IndexedDBStorage({ dbName: 'my-workspace' });
await contentBackend.open();

// Persistent blob storage
const blobBackend = new IndexedDBBlobStorage({ dbName: 'my-workspace-blobs' });
await blobBackend.open();

// Create stores with custom config
const contentStore = new ContentStore(contentBackend, {
  cache: { roles: 10, preferences: 50, contextEntries: 20 },
  flush: { maxBufferSize: 20, flushIntervalMs: 30000 },
});

const blobStore = new BlobStore(blobBackend, { eagerEviction: false });
await blobStore.init();

const manager = new WorkspaceManager(contentStore);

Verification

console.log(manager instanceof WorkspaceManager); // true

Usage Documentation

Basic Usage

Create a workspace, add a role, create a session, and append a turn.

import { emptyIndexState, merge, WorkspaceManager, MemoryStorage, ContentStore } from '@asaidimu/utils-workspace';

let workspace = {
  id: 'ws-1',
  settings: { language: 'en', defaultRole: 'assistant' },
  project: { name: 'My Project', owner: 'me' },
  index: emptyIndexState(),
};

const backend = new MemoryStorage();
const contentStore = new ContentStore(backend);
const manager = new WorkspaceManager(contentStore);

// Add a role
const roleCmd = {
  type: 'role:add',
  timestamp: new Date().toISOString(),
  payload: {
    id: 'r1',
    name: 'assistant',
    label: 'Assistant',
    persona: 'You are a helpful assistant.',
    preferences: [],
  },
};
const roleResult = await manager.dispatch(workspace, roleCmd);
if (roleResult.ok) workspace = merge(workspace, roleResult.value);

// Create a session
const sessionCmd = {
  type: 'session:create',
  timestamp: new Date().toISOString(),
  payload: { id: 's1', label: 'First Chat', role: 'assistant', topics: [] },
};
const sessionResult = await manager.dispatch(workspace, sessionCmd);
if (sessionResult.ok) workspace = merge(workspace, sessionResult.value);

Session Management (with SessionManager)

SessionManager provides a live Session object with turn tree operations.

import { SessionManager } from '@asaidimu/utils-workspace';

const sessionManager = new SessionManager(manager, contentStore);
const openResult = await sessionManager.open(workspace, 's1');
if (!openResult.ok) throw new Error('Failed to open session');

const { session } = openResult.value;

// Add a turn (auto‑links parent to current head)
const turn = {
  id: crypto.randomUUID(),
  version: 0,
  role: 'user',
  blocks: [{ type: 'text', text: 'Hello, world!' }],
  timestamp: new Date().toISOString(),
  parent: null,
};
const addResult = await session.addTurn(workspace, turn);
if (addResult.ok) workspace = merge(workspace, addResult.value);

// Flush buffered turns to storage
await session.flush();

// Close session when done
await sessionManager.close(session);

Blob Registration & Resolution

import { BlobStore, MemoryBlobStorage } from '@asaidimu/utils-workspace';

const blobStorage = new MemoryBlobStorage();
const blobStore = new BlobStore(blobStorage);
await blobStore.init();

const data = new TextEncoder().encode('Content of a document');
const registerResult = await blobStore.register(data, 'text/plain', 'readme.txt');
if (registerResult.ok) {
  const blobRef = registerResult.value;
  // Use blobRef in a turn or context block
}

// Later, resolve the blob for a prompt
const resolveResult = await blobStore.resolveRef(blobRef, null);
if (resolveResult.ok) {
  const resolved = resolveResult.value;
  if (resolved.kind === 'inline') {
    console.log(new TextDecoder().decode(resolved.data));
  }
}

Building a Prompt

PromptBuilder assembles a token‑aware prompt from an EffectiveSession.

import { PromptBuilder } from '@asaidimu/utils-workspace';

const builder = new PromptBuilder({
  blobResolver: blobStore.resolveRefs.bind(blobStore),
});

const effective = await manager.resolveSession(workspace, 's1');
if (!effective.ok) throw new Error('Failed to resolve session');

const prompt = await builder.build(effective.value, {
  tokenBudget: { total: 4000 },
  relevanceConfig: { recentMessageWindow: 5, minScore: 0.2 },
  providerId: 'anthropic', // for remote blobs
});

console.log(prompt.system.instructions);
console.log(prompt.transcript.turns);
console.log(prompt.warnings);

Common Use Cases

1. Offline‑First Chat Application

• Use IndexedDBStorage and IndexedDBBlobStorage for persistence.
• Set eagerEviction: false to keep blobs until manually cleaned.
• Let ContentStore auto‑flush when maxBufferSize is reached.
• Call blobStore.gc() occasionally to reclaim space.

2. Multi‑Session Workspace with Topic‑Based Context

• Create context entries with topics.
• Use session:topics:add command to associate topics with a session.
• resolveSession automatically pulls in context entries that match those topics.

3. Branching Conversations

• Use session.branchFrom() to fork a conversation tree.
• Each branch maintains its own head pointer.
• Fork an entire session with the session:fork command.

4. Integrating with AI Providers

• After uploading a blob to a provider (e.g., Anthropic), record the remote ID:

  await blobStore.recordRemoteId(sha256, 'anthropic', 'file_abc123');

• In PromptBuilder.build, pass the providerId to get remote references instead of inline bytes.

Project Architecture

Core Components

• WorkspaceManager – Facade that coordinates the pure reducer, content side effects, and session resolution.
• ContentStore – Manages roles, preferences, context, and transcript windows. Handles LRU caching, dirty buffers, and session activation.
• BlobStore – Owns the blob registry (ref counts, remote IDs) and SHA‑256 computation. Interacts with a BlobStorage backend.
• TurnTree – Pure logic for turn chains, subtree deletion, and head management. Used by ContentStore and Session.
• Session – Live object for an open session. Holds the turn DAG and dirty buffer, provides mutation methods (addTurn, editTurn, branchFrom, deleteTurn, switchLeft/Right).
• SessionManager – Entry point for opening and closing sessions; loads the turn DAG from storage.
• PromptBuilder – Orchestrates context retrieval, token planning, blob resolution, and final assembly.
• Storage backends – MemoryStorage, IndexedDBStorage (content), MemoryBlobStorage, IndexedDBBlobStorage (blobs).

Data Flow

1. User action → WorkspaceManager.dispatch(command).
2. Reducer produces a DeepPartial<Workspace> patch and schedules side effects.
3. Side effects write to ContentStore (roles, preferences, context, turns via TurnTree).
4. Session activation loads the turn DAG via TurnTree.buildNodeGraph().
5. Turn mutations update the in‑memory Session.nodes and dirty buffer; auto‑flush writes to storage when buffer size exceeds threshold.
6. Prompt building retrieves an EffectiveSession (via Session.resolve or ContentStore.resolveSession), ranks context, plans token usage, resolves blobs, and assembles the final Prompt.

Extension Points

• ContextRetriever – Replace the default Jaccard‑based retriever with a vector or hybrid approach.
• TokenPlanner – Implement custom token estimation (e.g., using tiktoken) and priority strategies.
• Summarizer – Hook into an LLM to compress transcripts.
• BlobStorage / ContentStorage – Add new backends (e.g., S3, SQLite) by implementing the interface.

Development & Contributing

Development Setup

1. Clone the repository:

   git clone https://github.com/asaidimu/erp-utils.git
   cd erp-utils/src/workspace

2. Install dependencies:

   npm install

3. Build the project:

   npm run build

Scripts

| Script | Description | |----------------------|-------------------------------------------| | npm test | Run tests once (Vitest) | | npm run test:watch | Run tests in watch mode | | npm run test:browser | Run tests in a browser environment |

Testing

The test suite uses vitest and fake-indexeddb/auto to simulate IndexedDB. Run the tests with:

npm test

All core functionality is covered. When contributing, please add tests for new features or bug fixes.

Contributing Guidelines

• Branching: Use feature branches off main.
• Commit messages: Follow conventional commits (e.g., feat: add new retriever).
• Pull requests: Ensure tests pass and add new tests for changes.
• Code style: The project uses Prettier and ESLint. Run formatting before committing.

Issue Reporting

Please open an issue on GitHub with:

• A clear description of the problem.
• Steps to reproduce.
• Expected and actual behaviour.
• Version of the library and environment (Node.js / browser).

Additional Information

Troubleshooting

| Issue | Solution | |--------------------------------------------|--------------------------------------------------------------------------------------------| | Database not open error | Call open() on the storage backend before using it. | | Blob not found in prompt | Ensure the blob is still referenced (refCount > 0) and not evicted. Run gc() only when safe. | | Turns missing after reload | Check that flush() was called before deactivation, or that maxBufferSize was reached. | | Prompt builder returns warnings for blobs | Verify that the blob is registered and, if remote, that recordRemoteId was called. |

FAQ

Q: How does blob deduplication work?
A: BlobStore computes SHA‑256 of the data. If a blob with the same hash already exists, only the ref count is incremented – bytes are never duplicated.

Q: Can I use this library in a Node.js environment without IndexedDB?
A: Yes. Use MemoryStorage and MemoryBlobStorage for in‑memory persistence. For long‑term storage, you would need to implement a custom backend (e.g., file system or SQLite).

Q: How do I manage token budgets for large transcripts?
A: Use the summarizer option in PromptBuilder to compress older turns. The default TokenPlanner trims from the oldest turns when the budget is exceeded.

Q: What happens if I delete a role that is used by a session?
A: The reducer will reject the command with INVALID_COMMAND. You must first switch sessions to a different role or delete the sessions.

License

This project is licensed under the MIT License.

Acknowledgments

Built with inspiration from content‑addressed storage systems (IPFS, git) and conversation tree models. Special thanks to the open‑source community for tools like TypeScript, Vitest, and fake-indexeddb.