@asaidimu/utils-workspace

The Architectural Foundation for Production-Grade AI Workspaces.

📖 Table of Contents

1. 1. Introduction: The Evolution of Conversation State
2. 2. Mental Model: Conversation as a Directed Acyclic Graph (DAG)
3. 3. Core Design Philosophy: The Four Pillars of AI State
4. 4. System Architecture: The "Engine Under the Hood"
   • 4.1. CQRS: Decoupling Write and Read Paths
   • 4.2. The Index: Reactive Read Projections
   • 4.3. The Sequential Serializer: Race Condition Prevention
   • 4.4. Event Bus and Reactivity
5. 5. Domain Entities: Deep Dive into the State
   • 5.1. Sessions: The Contextual Roots
   • 5.2. Turns: The Atoms of Hyper-History
   • 5.3. Blobs: Deduplicated, Content-Addressed Asset Management
   • 5.4. Roles: Persistent Personas and AI Identity
   • 5.5. Preferences & Context: The Multi-Tier Memory Layer
6. 6. The Prompt Pipeline: Anatomy of a Request
   • 6.1. Stage 1: DAG Snapshotting and Ancestry Walking
   • 6.2. Stage 2: Semantic Retrieval and Topic Alignment
   • 6.3. Stage 3: Conflict Resolution in Instructions
   • 6.4. Stage 4: Multi-Modal Blob Resolution Protocols
   • 6.5. Stage 5: Provider-Specific Adapter Mapping
7. 7. The Content Block Specification: Standardized Interoperability
   • 7.1. Basic Blocks: Text and Thinking
   • 7.2. Asset Blocks: Image and Document
   • 7.3. Functional Blocks: ToolUse and ToolResult
   • 7.4. Structural Blocks: Summary and RoleTransition
8. 8. Command Reference: The Mutation API Manual
   • 8.1. Workspace Domain Commands
   • 8.2. Session Domain Commands
   • 8.3. Turn & DAG Domain Commands
   • 8.4. Role & Persona Domain Commands
   • 8.5. Blob & Asset Domain Commands
   • 8.6. Resource (Context/Preference) Domain Commands
9. 9. Extension Framework: Building the Plugin Ecosystem
   • 9.1. WorkspaceExtensions: Packaging Domain Logic
   • 9.2. Middleware: The Request Interceptor Pipeline
   • 9.3. TurnProcessors: Giving the AI Agency
   • 9.4. Custom Indexers: Expanding the Read Model
10. 10. Persistence Layer: Schema and Collections
11. 11. Implementation Guide: Building Your Workspace
    • 11.1. Implementing a Database Boundary
    • 11.2. Implementing a BlobStorage Boundary
    • 11.3. Bootstrapping the Manager
12. 12. Design Decisions: The "Why" Behind the "What"
13. 13. Comparison with Industry Standards
14. 14. Advanced Recipes & Design Patterns
15. 15. Technical Specifications: Protocols & Formats
16. 16. Troubleshooting & Common Failure Modes
17. 17. Testing Strategy: Ensuring Workspace Integrity
18. 18. Glossary of Terms
19. 19. API Reference Appendix
20. 20. License

1. Introduction: The Evolution of Conversation State

In the first wave of LLM integration, "conversation" was modeled as a disposable buffer—a linear array of messages passed to an endpoint. While sufficient for simple chatbots, this model is fundamentally incompatible with Professional AI Workspaces.

A workspace isn't just a chat; it's a collaborative environment where AI agents act on artifacts, remember user preferences across months, navigate complex branching decision trees, and handle multi-gigabyte document libraries without breaking a sweat.

@asaidimu/utils-workspace is the state engine designed for this complexity. It provides a high-integrity, content-addressed foundation that treats:

• Conversation as Graph Architecture: Enabling non-destructive editing and infinite branching.
• Data as Content-Addressed Assets: Eliminating redundancy and ensuring asset integrity.
• State as Observable Commands: Providing a perfect audit trail and reactive UI projections.

If you are building an IDE, a research lab, or an autonomous agent platform, this is the engine you've been looking for.

2. Mental Model: Conversation as a Directed Acyclic Graph (DAG)

Most developers think of chat history as an Array<Message>. In this framework, we discard that model in favor of a Directed Acyclic Graph (DAG).

2.1. The Hyper-History Concept

In a standard chat, each message points to the one before it. In our TurnTree model, we introduce a layer of abstraction between the "Sequence" and the "Content."

• TurnNode: Represents a logical "moment" or "slot" in the conversation timeline. It is a stable identifier that does not change.
• Turn: Represents the actual data (text, images, tool results) at that moment.

By separating these, we enable Non-Destructive Editing. When a user "edits" a prompt at TurnNode X, we don't update the database. We create Turn (Version 2) under TurnNode X. The graph now has two possible paths forward from X's parent.

2.2. Forking and Divergent Timelines

Sessions are merely pointers to a "Head" in the global graph of TurnNodes. This means "Forking" a session is an $O(1)$ metadata operation. You can create 1,000 different sessions that all share the same first 50 turns. This is critical for:

• A/B Testing: Testing how different model instructions respond to the same conversation.
• Collaborative Branching: Allowing multiple team members to explore different paths from a shared starting point.

3. Core Design Philosophy: The Four Pillars of AI State

3.1. Immutability and Versioning

We believe that AI history should be a permanent record. Every AI response and user prompt is immutable. To "change" the past, you must create a new branch. This ensures that the "reasoning path" of an agent is always recoverable.

3.2. Content-Addressability (SHA-256)

Assets (images, PDFs, source code) are identified by the cryptographic hash of their contents. This solves the "State Bloat" problem. If 100 sessions all reference the same 5MB system manual, the workspace only stores one copy.

3.3. Command-Driven Mutations (CQRS)

We strictly follow the Command Query Responsibility Segregation (CQRS) pattern.

• Commands: Describe intent (e.g., session:fork).
• Reducers: Pure-ish functions that execute the intent against the persistent stores.
• Projections: In-memory read models (The Index) that allow for sub-millisecond retrieval of workspace state.

3.4. Portability and LLM Agnosticism

The workspace core does not know what "OpenAI" or "Gemini" is. It operates on an abstract Prompt model. This allows you to build your entire application logic once and swap between different LLM providers just by changing the LLMAdapter.

4. System Architecture: The "Engine Under the Hood"

4.1. Write Model: Atomic Entity Stores

The library organizes data into atomic "Stores." Each store is responsible for a single domain entity (e.g., RoleStore, SessionStore). These stores are designed to be backed by any persistent database via a standardized Database boundary.

4.2. Read Model: Reactive Index Projections

Querying a database for every prompt is too slow. To solve this, the WorkspaceManager maintains a Workspace Index. This index is a reactive, in-memory projection of the underlying stores.

• When a command is executed, the reducer returns a "Patch."
• The manager applies the patch to the Index.
• Subscribers are notified via the Event Bus.

4.3. The Sequential Serializer (Race Condition Prevention)

Concurrency in graph-based history leads to "phantom branches" and corrupt heads. The WorkspaceManager uses a Sequential Serializer (a promise-chaining queue) to ensure that only one mutation can happen at a time.

4.4. Event Bus and Reactivity

The workspace is "Alive." Every mutation emits events that can be listened to by the UI, external logging services, or other parts of the application.

5. Domain Entities: Deep Dive into the State

5.1. Sessions: The Contextual Roots

A Session is the user's primary unit of interaction. It manages the "active timeline."

• Head Pointer: A TurnRef (UUID + Version) identifying the current leaf of the DAG.
• Topic Affinity: Semantic tags that act as "magnets" for RAG context and user preferences.

5.2. Turns: The Atoms of Hyper-History

A Turn is the smallest unit of conversation. It is a multi-modal container that can hold anything from a text string to a sequence of tool executions and internal "thinking" blocks.

5.3. Blobs: Deduplicated Asset Management

Blobs represent the library's solution for large binary data. They utilize a Reference Counting garbage collector.

• Registration: Calculate hash -> Check for existence -> Create or increment ref-count.
• Automatic Cleanup: When a turn is deleted, the system decrements the blob's ref-count. Only when it hits zero is the physical file deleted from storage.

5.4. Roles: Persistent Personas

A Role encapsulates the AI's identity. It includes the "System Prompt" (Persona), model-specific constraints (e.g., temperature, window size), and a set of topics that the role is "interested" in.

6. The Prompt Pipeline: Anatomy of a Request

The PromptBuilder follows a 5-step lifecycle:

6.1. Stage 1: DAG Snapshotting and Ancestry Walking

Starting from the session's Head, the builder performs a recursive walk up the parent pointers. This flattens the graph into a linear Transcript.

6.2. Stage 2: Semantic Retrieval and Topic Alignment

The builder identifies the active Topics of the session. It then queries the ContextIndex. It uses the ContextRetriever to rank entries by relevance.

6.3. Stage 3: Conflict Resolution in Instructions

It assembles the final "System Instruction." It merges the Role instructions with any active Preferences.

6.4. Stage 4: Multi-Modal Blob Resolution Protocols

Every BlobRef in the transcript is resolved. The builder contacts the BlobStore to determine if the data should be sent as inlineData (Base64) or if it has a pre-existing fileId.

6.5. Stage 5: Provider-Specific Adapter Mapping

The finalized, agnostic Prompt is passed to the LLMAdapter. This is where the mapping to the provider's specific wire format occurs.

7. The Content Block Specification

7.1. Basic Blocks

• text: Standard text communication.
• thinking: Internal CoT reasoning. Separate to allow UI to hide/show reasoning paths.

7.2. Asset Blocks

• image: References a BlobRef. Supports mediaType and filename.
• document: Used for PDFs, code snippets, or long-form text files.

7.3. Functional Blocks

• tool_use: A structured request for tool execution. Contains callId, name, and args.
• tool_result: The output of a tool, linked via callId.

7.4. Structural Blocks

• summary: A "compression" block. Replaces older history turns.
• role:transition: Notates that the session persona changed.

8. Command Reference: The Mutation API Manual

8.1. Workspace Domain Commands

• workspace:create: Initializes root metadata.
• workspace:sync: Forces a full re-indexing of all stores.

8.2. Session Domain Commands

| Command | Payload | Side Effects | | :------------------------ | :------------------------ | :------------------------------------- | | session:create | id, role, topics, label | Creates database record + Index entry. | | session:fork | sessionId, turnId | Metadata-only clone of history. | | session:update | sessionId, updates | Patch Index + Update store. | | session:delete | sessionId | Recursive release of all blob refs. | | session:role:switch | sessionId, newRole | Insert transition block + Update role. |

8.3. Turn & DAG Domain Commands

| Command | Payload | Description | | :---------------- | :----------------------------- | :-------------------------------- | | turn:add | sessionId, turn | Appends turn. Retains blobs. | | turn:edit | sessionId, turnId, newBlocks | New Version in DAG. Moves Head. | | turn:branch | sessionId, turn | Diverge from non-head node. | | turn:delete | sessionId, turnId, version | Removes Version + Releases blobs. |

8.4. Role & Persona Domain Commands

• role:add: Register new persona.
• role:update: Modify instructions/constraints.
• role:delete: Remove role.

8.5. Blob & Asset Domain Commands

• blob:register: Calculate hash + Create storage record + Increment ref-count.
• blob:retain: Increment ref-count.
• blob:release: Decrement ref-count + Trigger purge if 0.
• blob:record_remote_id: Map local hash to remote API ID.

9. Extension Framework

9.1. WorkspaceExtensions

Packaging custom logic into a single plugin.

const MyPlugin: WorkspaceExtension = {
  schemas: [
    /* schemas */
  ],
  reducers: { "custom:cmd": myReducer },
  middleware: [myMiddleware],
};

9.2. Middleware: Intercepting the State Stream

Guardians of the workspace. Can abort, augment, or observe commands.

9.3. TurnProcessors: Giving the AI Agency

Scans AI response blocks for side-effects (e.g., auto-saving memories or tasks).

9.4. Custom Indexers

Expanding the Read Model with custom read-projections.

10. Persistence Layer: Schema and Collections

| Collection | Key | Description | | :------------- | :-------------- | :---------------------------- | | workspace | id | Global metadata and settings. | | role | name | Persona instructions. | | preference | id | User instructions. | | context | key | RAG snippets. | | session | id | Session metadata + Head. | | turn | (id, version) | Immutable history records. | | blob_records | sha256 | Metadata + RefCount. |

11. Implementation Guide

11.1. Implementing a Database Boundary

class MyDb implements WorkspaceDatabase {
  async collection(name: string) {
    /* ... */
  }
  async open(schemas: SchemaDefinition[]) {
    /* ... */
  }
}

11.2. Implementing a BlobStorage Boundary

class MyStorage implements BlobStorage {
  async put(data: Uint8Array): Promise<string> {
    /* return sha256 */
  }
  async get(sha256: string): Promise<Uint8Array | null> {
    /* ... */
  }
  async delete(sha256: string): Promise<void> {
    /* ... */
  }
}

11.3. Bootstrapping the Manager

const { manager, sessions, ctx } = await createWorkspace({
  db: new MyDb(),
  blobStorage: new MyStorage(),
  getWorkspace: () => state,
  setWorkspace: (patch) => {
    state = merge(state, patch);
  },
  processor: new MyProcessor(),
});

12. Design Decisions

Why a DAG?

A tree allows branching, but a DAG allows for Merging and Non-Destructive Versioning. It accurately represents the way conversation evolved into multiple "what-if" scenarios.

Why SHA-256?

UUIDs lead to "Asset Proliferation." SHA-256 enables Global Deduplication, critical for reducing storage in RAG-heavy apps.

Why CQRS?

Read patterns (building prompts) are vastly different from Write patterns (saving turns). CQRS optimizes both independently.

13. Comparison with Industry Standards

• vs. OpenAI Threads: OpenAI threads are black boxes. Our Workspace provides full local control and DAG navigation.
• vs. LangChain Memory: LangChain memory is often ephemeral. Our Workspace is a persistent, repo-level database.

14. Advanced Recipes & Design Patterns

The "Shadow Turn" Pattern

Intercept turn:add and inject a hidden system turn before the user message for real-time situational awareness.

Automatic Topic Discovery

Use a cheap model to scan every user turn and suggest new topics to refine RAG retrieval dynamically.

15. Technical Specifications

Blob Hash Protocol

SHA-256 hex digest -> stored under blobs/{sha256}. RefCount = 0 triggers deletion.

Index Sync Protocol

Full re-build of Index via parallel execution of all registered Indexers.

16. Troubleshooting

Stale Index

Ensure reducers return the correct patch. Call workspace:sync to force a rebuild.

Blob Reference Leak

Use session:delete to ensure all blobs in the session DAG are released.

17. Testing Strategy

Integration Testing the DAG

test('can branch history', async () => {
  const session = await sessions.create({ ... });
  await session.addTurn({ blocks: [{ type: 'text', text: 'Prompt 1' }] });
  const turn1Id = session.head()!.id;
  await session.editTurn(turn1Id, [{ type: 'text', text: 'Edited Prompt 1' }]);
  // Verify both versions exist and head points to version 2
});

Unit Testing Reducers

Reducers are pure-ish and can be tested in isolation by providing a mock WorkspaceContext.

18. Glossary of Terms

• Head: The current active turn.
• Transcript: Flattened history for an LLM request.
• Persona: System instructions for a Role.
• Topic: Semantic tag for RAG connectivity.
• Reducer: Command handler for state mutation.

19. API Reference Appendix

WorkspaceManager

• dispatch(command): Atomic mutation.
• workspace(): Sync Index access.
• use(middleware): Register interceptor.

Session

• snapshot(): Prepare prompt data.
• addTurn(blocks): Append to head.
• branchInfo(turnId): Retrieve all versions.

20. License

MIT © Saidimu