@cogstream/types

Shared TypeScript types and Zod schemas for the CogStream platform.

This package is the single source of truth for all domain contracts — primitives, patterns, intents, episodes, user state, and agent events — used by every other package in the @cogstream/* family.

Installation

npm install @cogstream/types

No runtime dependencies beyond zod.

Contents

Primitives

Low-level building blocks derived from raw UI signals.

import { PRIMITIVE_TYPES, type PrimitiveInstance } from '@cogstream/types';

Families: movement, temporal, directional, interaction, error-correction, attention, progression, control, exit, voice.

Patterns

Sequences of primitives with behavioral meaning.

import { PATTERN_TYPES, type PatternInstance } from '@cogstream/types';

Families: friction, navigation, reading, motion, decision, progress, engagement, stability, voice.

Intents

High-confidence interpretations of what the user is trying to accomplish.

import { INTENT_TYPES, type CandidateIntent } from '@cogstream/types';

Families: orientation, decision, execution, recovery, disengagement, voice-strengthened.

Episodes

The core boundary object between sensing (client) and interpretation (server).

import type { EpisodeV2, UIContext, EpisodeMetrics } from '@cogstream/types';

EpisodeV2 is the only payload the sensing SDK ever transmits to the server — raw signals stay in the browser.

User State Model

The interpretation layer's output: a structured summary of the user's current intent, friction, and trajectory.

import type {
  UserStateModel,
  IntentHypothesis,
  FrictionState,
  ProgressState,
  TrajectoryState,
} from '@cogstream/types';

Agent & AG-UI Events

Decision inputs, intervention objects, and the AG-UI event stream that carries agent responses to the frontend.

import type {
  DecisionInput,
  DecisionOutput,
  Intervention,
  InterventionOutcome,
  AGUIEvent,
  ApplicationContextInterface,
} from '@cogstream/types';

Schemas (Zod)

Runtime-validated schemas for API boundaries.

import { validateEpisodeV2 } from '@cogstream/types/schemas';

const result = validateEpisodeV2(incoming);
if (!result.success) {
  console.error(result.error.issues);
}

Graph Seed

Manifest format for seeding the interpretation graph via POST /seed/openapi.

import type { AppGraphSeed } from '@cogstream/types';
import { AppGraphSeedSchema } from '@cogstream/types';

Semantic Hints: Application-Provided Context

CogStream now supports optional semantic hints — developer-provided context about user interactions — to enable richer, privacy-safe interpretation.

Use Case

As users navigate your application, CogStream tracks behavior (clicks, hesitations, form fills) to infer intent and struggle. However, without application context, some interpretations are ambiguous:

• A pause on a form field could indicate:
  • Genuine confusion (bad UX; show help)
  • Thoughtful consideration (user is deliberating; let them think)
  • Distraction (user left and came back; ignore)

Semantic hints let your app provide safe context to disambiguate:

import { addSemanticHint } from '@cogstream/sensing';

// Mark a high-stakes field: identity verification
addSemanticHint('identity-number-input', {
  type: 'label',
  category: 'journey-step',
  value: 'identity-verification',
  complexity: 'high'
});

// Mark a simple field: user's first name
addSemanticHint('firstname-input', {
  type: 'label',
  category: 'field-type',
  value: 'text-input',
  complexity: 'low'
});

When CogStream observes hesitation:

• On high-stakes field → Provides detailed explanation
• On simple field → Provides brief tip or stays silent

Privacy Governance

Semantic hints are designed with privacy first:

✅ Safe Labels (default)

Use type: 'label' for journey context without privacy risk:

{
  type: 'label',
  category: 'journey-step',
  value: 'plan-selection',          // Safe: describes what choice they're making
  complexity: 'medium'
}

Safe categories include:

• journey-step — what decision are they making? ("plan-selection", "identity-verification")
• field-type — what kind of field? ("email-input", "date-picker")
• complexity — "low", "medium", "high"
• screen-section — what part of screen? ("billing", "checkout", "profile")

🔐 Sensitive Content (opt-in)

Use type: 'content' only for user-provided data. Requires structural enforcement:

// ✅ CORRECT: Explicitly marked as sensitive
{
  type: 'content',
  category: 'user-selection',
  value: 'credit-card-type',        // Fact that selection occurred
  sensitive: true                    // REQUIRED: explicit opt-in
}

// ❌ WRONG: Missing sensitive flag — will fail validation
{
  type: 'content',
  category: 'user-selection',
  value: 'credit-card-type'
  // Missing: sensitive: true
}

Content hints encode the fact that user provided something (to detect pattern), not the actual data:

• ✅ "User selected a payment method" (safe)
• ❌ "User selected Visa card ending in 4242" (unsafe)

API Reference

addSemanticHint(elementId, hint, durationMs?)

Attach a semantic hint to a DOM element. The hint persists for the specified duration (default: 5 seconds).

Parameters:

• elementId (string) — CSS id of the element
• hint (SemanticLabel | SemanticContent) — the semantic context
• durationMs (number, optional) — how long hint remains active (default: 5000ms)

Example:

import { addSemanticHint } from '@cogstream/sensing';

// Attach hint for 5 seconds
addSemanticHint('submit-button', {
  type: 'label',
  category: 'action',
  value: 'checkout-submit',
  complexity: 'high'
});

// Or override default duration
addSemanticHint('email-input', {
  type: 'label',
  category: 'field-type',
  value: 'email',
  complexity: 'low'
}, 10000); // 10 seconds

clearSemanticHint(elementId)

Remove a hint early (before duration expires).

import { clearSemanticHint } from '@cogstream/sensing';

clearSemanticHint('submit-button');

Configuration

Set default hint duration in SensingRuntimeConfig:

import { SensingRuntime } from '@cogstream/sensing';

const runtime = new SensingRuntime({
  semanticHintTimeout: 8000  // All hints persist 8s by default
});

Type Safety

Semantic hints are validated at compile time and runtime:

import {
  SemanticHint,
  isSemanticLabel,
  isSemanticContent,
  validateSemanticHint
} from '@cogstream/types/semantic-hints';

// Type-safe creation
const label: SemanticLabel = {
  type: 'label',
  category: 'journey-step',
  value: 'plan-selection',
  complexity: 'high'
};

const content: SemanticContent = {
  type: 'content',
  category: 'user-selection',
  value: 'payment-method',
  sensitive: true  // Required
};

// Type guards
if (isSemanticLabel(hint)) {
  console.log(hint.value); // TypeScript narrows to SemanticLabel
}

// Runtime validation
try {
  validateSemanticHint(incomingHint);
} catch (e) {
  console.error('Invalid semantic hint:', e.message);
}

Best Practices

1. Label journey steps early in the flow

   addSemanticHint('billing-section', {
     type: 'label',
     category: 'journey-step',
     value: 'billing-address',
     complexity: 'medium'
   });

2. Mark high-stakes fields with high complexity

   addSemanticHint('ssn-input', {
     type: 'label',
     category: 'field-type',
     value: 'identity-number',
     complexity: 'high'
   });

3. Only use content hints when absolutely necessary — prefer labels that describe the choice, not the value.

4. Clear hints when elements unmount

   useEffect(() => {
     addSemanticHint('modal-submit', { /* ... */ });
     return () => { clearSemanticHint('modal-submit'); };
   }, []);

5. Match hint duration to element lifespan

   addSemanticHint('overlay-action', { /* ... */ }, 3000);   // Temporary
   addSemanticHint('billing-form', { /* ... */ }, 30000);    // Persistent

Privacy Compliance

Semantic hints support GDPR, CCPA, and similar privacy regulations:

• No PII by default — labels are safe; content requires explicit opt-in
• User control — hints are application-driven, so users can audit what context was shared
• No data retention — hints are ephemeral (5–30 seconds) and not stored after session
• Auditability — hint usage is logged for compliance review

Integration Example: Multi-Step Form

import React, { useEffect } from 'react';
import { addSemanticHint, clearSemanticHint } from '@cogstream/sensing';

export function CheckoutForm() {
  useEffect(() => {
    addSemanticHint('billing-section', {
      type: 'label',
      category: 'journey-step',
      value: 'billing-information',
      complexity: 'medium'
    });
    return () => clearSemanticHint('billing-section');
  }, []);

  useEffect(() => {
    addSemanticHint('identity-number-input', {
      type: 'label',
      category: 'field-type',
      value: 'identity-verification',
      complexity: 'high'
    });
    return () => clearSemanticHint('identity-number-input');
  }, []);

  return (
    <div id="billing-section">
      <input id="identity-number-input" type="text" placeholder="Enter ID number" />
    </div>
  );
}

FAQ

Q: Will semantic hints slow down my app? A: No. Hints are lightweight (metadata only) and don't affect rendering or interaction performance.

Q: Can users see what hints are attached? A: Hints are internal to CogStream. Users can review their own memory on the /memory page and see what patterns were learned.

Q: What if I attach the wrong hint? A: CogStream uses hints as one input among many. Incorrect hints won't harm learning; they'll just provide imperfect context.

Q: Do I need to use semantic hints? A: No. Element-level tracking works without hints. Hints are opt-in to improve interpretation accuracy on high-stakes journeys.

Q: Can I use hints for analytics? A: Hints are designed for interpretation, not analytics. For analytics, use your existing analytics library.

Architecture position

@cogstream/types (no runtime deps)
    ↑
    ├── @cogstream/sensing   — browser signal capture
    ├── @cogstream/agent     — decision + intervention logic
    └── @cogstream/copilotkit — React + CopilotKit integration

Links

• cogstream.ai
• Full specification