@escapeaihq/client-core

Framework-agnostic Seer client building blocks. Vanilla TypeScript, no React or Vue dependencies. Imported by both seer-agent's React web/ app (this monorepo) and the Vue Escape web-app (sibling repo).

Module format

ESM-only. The published package.json points exports[".".import] at ./dist/index.js and exports[".".types] at ./dist/index.d.ts — there's no CJS require path. Both current consumers are ESM-default (Next.js 15 + React, Nuxt 3 + Vue + Vite), so this is fine. If a future CJS consumer (older Jest config, Node CLI script) appears, add a CJS bundle to the build (tsup or similar dual-output) before it bites.

What's in here

| Module | What it does | |---|---| | timeFormat | formatTimestamp(seconds, opts?) — locale-aware M:SS / M:SS.cs formatter. formatCitationTimestamp(seconds) (src/timeFormat.ts:68) is a preset with centiseconds: true, used by chat surfaces for sub-second word/turn citations (renders as "1:23.45"). | | graphSchema | 0.12.0: Graph-schema vocabulary mirrored from the backend single source of truth (shared/graph_schema.py + shared/provenance.py). Types GraphNodeLabel, GraphRelationshipType; two separate provenance unions StoryGraphProvenance ('pegasus' \| 'user' \| 'document' \| 'regenerated') and MemoryFactProvenance ('chat' \| 'document' \| 'system' \| 'regenerated') — never merged. Runtime constants GRAPH_NODE_LABELS, GRAPH_RELATIONSHIP_TYPES, GRAPH_PRIMARY_KEY, GRAPH_EDITABLE_NODE_PROPERTIES, GRAPH_EDITABLE_RELATIONSHIP_PROPERTIES, STORY_GRAPH_PROVENANCE_VALUES, MEMORY_FACT_PROVENANCE_VALUES; authored-canon sets AUTHORED_STORY_PROVENANCE / AUTHORED_MEMORY_PROVENANCE + guards isStoryGraphProvenance / isMemoryFactProvenance. 0.13.0: adds a session-scoped memoized fetchGraphSchema(client) that fetches the live GET /graph/schema payload once per session (resets its cache on rejection so a failed fetch retries; __resetGraphSchemaCache() test hook). The graph-editor PropertyPanel now sources its runtime editable/PK lists from this fetched payload (backend = single runtime source of truth), dropping its hand-maintained literal whitelist. The static GRAPH_* unions/constants are retained as the typed compile-time vocabulary and a back-compat fallback — they are not removed (eliminating the TS copy entirely needs build-time codegen; tech-debt #205). Resolves tech-debt #45 for the graph editor (M7 Phase 1a). 0.15.0 (M7 Phase 3a): widens GraphNodeLabel + GRAPH_NODE_LABELS with Scene/PlotBeat/CharacterArc and GraphRelationshipType + GRAPH_RELATIONSHIP_TYPES with the six user-creatable edges (RELATES_TO/CAUSES/SET_IN/FEATURES/ON_ARC/LEADS_TO); adds GRAPH_PRIMARY_KEY (Scene→sceneId, PlotBeat→beatId, CharacterArc→arcId) + GRAPH_EDITABLE_NODE_PROPERTIES entries. New exports consumed by the 3b creation UI: userCreatableLabels (['Scene','PlotBeat','CharacterArc']), userCreatableEdges (the six edges), and allowedPairs (the TS mirror of the backend _VALID_EDGE_PAIRS: RELATES_TO→[[Entity,Entity]], SET_IN/FEATURES→[[Scene,Entity]], ON_ARC→[[PlotBeat,CharacterArc]], LEADS_TO→[[PlotBeat,PlotBeat]], CAUSES→[[Event,Event]]) — all additive. Graph schema only — no tool/MCP schemas. 0.39.0 (R8.1): adds the studio read-surface vocabulary STUDIO_NODE_LABELS / STUDIO_EDGE_TYPES / STUDIO_NODE_PROPERTIES / STUDIO_CHARACTER_TYPE (lifted from web graphVocabulary.ts, which now re-exports them) so the lifted storyGraph projection resolves its vocabulary in one place. | | storyGraph | 0.39.0 (R8.1): the pure get_video_graph projection lifted from web/app/components/studio/storyGraph.ts (the web module is now a thin re-export shim). deriveScenes(graph): SceneView[] (collapsed scenes with featured characters, location, shots, parent act — sorted by act.index then scene.index) and deriveCast(graph): CharacterView[] (character entities with aliases, scene memberships, arcs, related characters). Edge-direction-agnostic traversal (CONTAINS/INCLUDES/FEATURES/SET_IN/ON_ARC/RELATES_TO matched by EITHER endpoint). The single source of the collapsed-story projection both Studio read-views AND the Director story-timeline mapper (reactor/storyTimeline.ts) consume — two copies would drift. Plus view-model types SceneView/CharacterView/ArcView/ShotView/CharacterRef/LocationRef. | | toolSchemas | 0.11.0: AGENT_TOOL_SCHEMAS, AGENT_TOOL_NAMES, getAgentToolSchema(name) + types AgentToolSchema / AgentToolName / ToolRequestField / ToolFieldType (src/toolSchemas.ts). Framework-agnostic TS mirror of the backend's 12 SERVER-SIDE RAG tools whose canonical source is shared/tool_schemas.py. Exposes each tool's name, description, endpoint, HTTP method, and request field shapes so clients can introspect the agent's tool surface without copy-pasting strings. Keep in sync when the backend tool list changes (no codegen yet). Distinct from the Helios browser-side BEDROCK_TOOL_SCHEMAS in reactor/heliosTools.ts — different vocabulary, do not converge. | | audioProtocol | Binary audio frame codec (1-byte tag + 320 samples PCM16); type defs for phoneme/word/barge-in messages | | audio | MicrophoneRecorder, AudioFramePlayer, PlaybackTracker, AudioSessionClient, AudioPipeline. Web Audio API based. Clean barge-in via GainNode + tracked activeSources. Sentence-boundary viseme scheduling. | | visemeScheduler | Maps Inworld TTS 1.5 phoneme timing to mouth shape changes via requestAnimationFrame, locked to AudioContext.currentTime. Stops the loop when idle. | | visemeGeometry | Pure path math: derive inner-lip donut, mouth clip ellipse, teeth/tongue geometry from existing mouth_paths data | | websocket | ChatWebSocket — auto-reconnecting wrapper around native WebSocket with text/binary multiplexing. sendPlaybackState(state: PlaybackState) publishes { action: 'playback_state', videoId, currentTime, shotId?, paused, clientTs } for cross-device video↔chat sync (src/websocket.ts:493 sendPlaybackState + src/websocket.ts:69 PlaybackState). Fire-and-forget: drops silently when the socket isn't OPEN. See specs/2026-05-17-cross-device-playback-sync.md. Phase 2 vision signaling: sendVisionOffer({ sdp, fps }) (src/websocket.ts:578), sendVisionIce({ candidate, sdpMid, sdpMLineIndex }) (src/websocket.ts:594), and sendVisionClose() (src/websocket.ts:614) forward SDP/ICE over the existing chat-ws WebSocket; inbound vision_answer, vision_ice, and vision_status events are surfaced via typed event handlers. 0.49.0 (dashboard A-DM0): the ChatWsEvent union gains a dm_directive member { type, timestamp?, directive: DmDirectiveKind, params } (mirrors ChatEventType.DM_DIRECTIVE in shared/chat_dispatcher.py) — a DM-issued scene-control directive (enter_world/exit_world/seek_video/set_experience_source). Additive; ConversationRuntime.applyFrame drops it via its default arm (no consumer wired yet — A-DM0 is dark). 0.51.0 (dashboard B5 — server-anchored session): sendSessionAttach({ action:'session_attach', videoId, connectionId }) joins a server-anchored chat session (anchor (userId,videoId), matching SessionAttachMessage + the PlaybackStateStore row) — best-effort like the vision sends. sendSessionDetach() leaves it WITHOUT closing the socket (mirrors sendVisionDetach; backend session_detach action) so a long-lived adopted socket can rebind away from a video session without leaving a stale peer. Inbound session_attached (ack with the replayed cross-device anchoredState + the peers list) and presence (attached/detached fan-out) ride the SAME typed-event dispatcher as the vision events (on('session_attached'|'presence', …)), with payload types SessionAttachedEvent / PresenceEvent / SessionAnchoredState. The ChatWsEvent union also gains session_attached + presence variants (exhaustive-switch arms). All additive; a client that never calls sendSessionAttach is byte-identical. Mirrors api/chat/ws.py::_handle_session_attach / _detach_session. | | vision/ | Phase 2: VisionPublisher (src/vision/VisionPublisher.ts:1) — framework-agnostic class taking any MediaStreamTrack + ChatWebSocket + optional VisionRtcCredentials and managing the RTCPeerConnection lifecycle (SDP offer/answer, ICE trickle, restart-on-disconnect). Dual-mode signaling: credentials.mode === 'kvs' dynamically imports the optional amazon-kinesis-video-streams-webrtc peer dep and connects via Amazon KVS WebRTC against a backend-minted SigV4-presigned wss:// URL; otherwise (or when no credentials passed) uses the existing chat-WS vision_offer/vision_ice/vision_close protocol. ChatWebSocket.sendVisionAttach({clientId}) (KVS mode only) binds the chat session to the KVS viewer ClientId so the backend dispatcher can find the FrameBuffer. visionScheduler.ts adjusts published fps from backend vision_status back-pressure (pure helper). types.ts exports VisionPublisherConfig, VisionPublisherOptions, VisionRtcCredentials, VisionStatus, VisionStatusListener, VisionErrorListener. Source-agnostic by design — consumers wire <video>.captureStream(), getUserMedia(), or future Reactor/Helios/PlayCanvas surfaces (Issue #150). 0.10.0 additions: toVisionRtcCredentials(response) (src/vision/credentials.ts:23) — pure mapping helper that converts a ChatVisionRtcCredentialsResponse API wire shape into VisionRtcCredentials, coercing null ICE fields to undefined. startVisionPublisher(opts) (src/vision/startVisionPublisher.ts:42) — framework-agnostic factory: construct→subscribe→start→cleanup in one call, returning VisionPublisherHandle { publisher, stop() }. Used by both /watch and /chat-ws React hooks so the construct/teardown sequence has exactly one implementation. 0.31.0 (R3a — vision-source picker): sourceKinds.ts — VISION_SOURCE_KINDS (webcam/playing-video/server-render, the frozen constant + VisionSourceKind union) and the VisionSourceDescriptor tagged union ({kind:'webcam',constraints?} vs {kind:'playing-video'|'server-render', element, waitForFrame?}) so the controller never queries the DOM. VisionSourceController.ts (src/vision/VisionSourceController.ts:89) — the framework-agnostic owner of "turn a chosen local source into a published feed": start(descriptor) acquires (getUserMedia vs element.captureStream(), with the waitForFrame black-frame gate lifted from the watch hook), mints credentials AFTER acquisition, then DELEGATES to startVisionPublisher; switchSource() tears the previous publisher + tracks down BEFORE acquiring the next; stop() is idempotent, leaves zero live tracks, and epoch-guards in-flight starts (a racing stop/switch releases the late stream, no error); currentKind + isStarting expose the published source and the in-flight phase. Environmental failures → onError (restartable); misuse (start() while active) throws. ws must be the SAME authenticated ChatWebSocket as the chat turns (frames are keyed to the WS session server-side). Land-dark until R3b–R3d wire the surfaces; sources are LOCAL only (cross-device vision deferred). | | api | ApiClient REST client + all backend response/request type definitions. Constructor takes baseURL and chatBaseUrl from the caller (no env-var coupling). Image-tool wrappers since 0.3.0: extractImage, regenerateImage, pollVisemeBundleUntilDone. Bundle resolution since 0.5.0: loadBundleManifest (handles the inline-vs-S3 body_svg split internally) + canonical VisemeBundleManifest type. Reactor/Helios wrappers: getReactorToken, getSeedImageBlob, upsampleReactorPrompt. Chat-model registry (since the LLM-provider phase): listChatModels() returns ChatModelsResponse with ChatModelInfo entries (id, provider, capabilities: ChatModelCapability[], context window, etc.) — public endpoint so it works before login. 0.10.0 additions: getSystemPrompt() returns SystemPromptComponents (src/api.ts:205,2132) with { film_qa, guardrails, audio_markup, full_default } — calls GET /api/v1/chat/system-prompt. 0.13.0 additions (M7 Phase 1a): getGraphSchema() returns GraphSchemaResponse ({ traceId, labels: Record<string,{pk,editable[]}>, edges: Record<string,{editable[]}> }) — calls GET /api/v1/tools/graph/schema. Graph-editor signature changes: deleteGraphNode / deleteGraphEdge now send query params (no request body — tech-debt #42), and UpdateNodeRequest / DeleteNodeRequest input types dropped idProperty (backend derives the PK — tech-debt #44). These are input-type narrowings (callers pass less); additive otherwise. 0.14.0 additions (M7 Phase 1b): NarrativeAct (src/api.ts:751) gains optional actId / index / title / summary / t0 / t1 — additive only; the existing act / timeRange / description fields are retained (per the client-core API-is-a-contract rule, no public export removed/renamed). Acts and themes / mainCharacterNames are now read from the graph — Act-labelled GraphNodes and the Video node properties returned by getVideoGraph() — rather than parsed from getEnrichments(); GraphNode / VideoGraphResponse carry Act nodes with no schema change. 0.15.0 additions (M7 Phase 3a): createGraphNode(data: CreateNodeRequest): Promise<CreateNodeResponse> (POST /api/v1/tools/graph/nodes) and createGraphEdge(data: CreateEdgeRequest): Promise<CreateEdgeResponse> (POST /api/v1/tools/graph/edges) — both additive (no public export removed/renamed, per the API-is-a-contract rule). New request/response types CreateNodeRequest { videoId, label, properties } / CreateNodeResponse { traceId, nodeId, properties } and CreateEdgeRequest { videoId, sourceId, targetId, type, properties? } / CreateEdgeResponse { traceId, edgeId, properties }. The 201 JSON body is parsed by the existing makeRequest path (only 204 is special-cased); a 422 surfaces the FastAPI detail message unchanged. Source-document CRUD (M7 Phase 4, package version 0.14.0): shapes mirror the authoritative backend Pydantic models (api/tools/schemas.py). uploadDocument(file, meta, onProgress?) (multipart XHR mirroring uploadVideo; file sent under the multipart field upload; POST /api/v1/ingestion/documents → { traceId, document }, HTTP 202 new / 200 idempotent dedupe), listDocuments({ videoId?, seriesId?, limit?, nextToken? }), getDocument(docId) (→ { traceId, document } — suggestions are nested in document.suggestions, no top-level field), acceptSuggestion(docId, sid) (201 { traceId, suggestionId, graphId }, writes graph node/edge with provenance document), rejectSuggestion(docId, sid) (204), deleteDocument(docId) (204) — all on the tools/ingestion base URL, not chat. New types/constants: DocumentKind, DocumentStatus + DOC_STATUS_*, SuggestionStatus + SUGGESTION_STATUS_*, Suggestion, DocumentRecord (mirrors DocumentResponse: suggestions always present; no ownerUserId/s3Key/contentHash), UploadDocumentMeta, UploadDocumentResponse { traceId, document }, ListDocumentsResponse, GetDocumentResponse { traceId, document }, AcceptSuggestionResponse { traceId, suggestionId, graphId }. 0.16.0 additions (M7 Phase 6 — narrative regeneration): startRegeneration(videoId, request: RegenerateRequest): Promise<RegenerationRunResponse> (POST /api/v1/tools/graph/{videoId}/regenerate → 202; statusUrl/Location to the run's poll URL; a 409 lock / 429 cap / 422 bad-scope surfaces the FastAPI detail), listRegenerations(videoId, { limit?, cursor? }?) (GET .../regenerations → RegenerationListResponse newest-first + nextCursor), getRegeneration(videoId, runId, { signal? }?) (GET .../regenerations/{runId}, 404 absent), and pollRegenerationUntilDone(videoId, runId, { intervalMs?, timeoutMs?, signal? }?) (5s cadence, abortable — resolves on complete/cancelled, throws on failed, mirrors pollVisemeBundleUntilDone). New types/constants RegenerationRunStatus + REGENERATION_STATUS const map + REGENERATION_TERMINAL_STATUSES, RegenerationScopeKind + REGEN_SCOPE_BARE_KINDS/REGEN_SCOPE_TARGETED_KINDS/REGEN_SCOPE_SEPARATOR + buildRegenerationScope(kind, targetId?) (builds acts/characters/scene:<id>/arc:<id>; throws on a bare+target or targeted-without-id contract violation), RegenerateRequest { scope, promptAddon? }, RegenerationRunResponse { traceId, runId, videoId, scope, status, statusUrl, createdAt, promptAddon?, startedAt?, completedAt?, delta, error? }, RegenerationListResponse { traceId, videoId, runs, nextCursor }. 0.21.0 additions (M9 Story Engine Phase 8a — Adventure Runtime): startAdventure(request: StartAdventureRequest, { signal? }?): Promise<AdventureResponse> (POST /api/v1/adventures → 201, Location to the session GET URL; a 422 Persona is not a Dungeon Master / 404 / 403 surfaces the FastAPI detail), getAdventure(sessionId, { signal? }?): Promise<GetAdventureResponse> (GET /api/v1/adventures/{sessionId}, 404 absent/not-owned), deleteAdventure(sessionId, { signal? }?): Promise<void> (DELETE .../{sessionId} → 204). New camelCase-wire types TurnRecord { turn, beatId, summary }, AdventureState { sessionId, ownerUserId, storyId, videoId, dmPersonaId, partyPersonaIds, currentBeatId?, visitedBeatIds, flags, inventory, turnHistory, createdAt, updatedAt }, StartAdventureRequest { storyId, videoId, dmPersonaId, initialBeatId? }, AdventureResponse { traceId, sessionId, state }, GetAdventureResponse { traceId, state }. PersonaConfig / CreatePersonaRequest / UpdatePersonaRequest gain optional snake_case is_dungeon_master (read defaults false). ChatWebSocket.send() options + buildHelloPayload() gain optional adventureSessionId (spread only when set → land-dark). 0.22.0 additions: listAdventures(storyId?, { signal? }?): Promise<ListAdventuresResponse> (GET /api/v1/adventures, owner-scoped newest-first, optional storyId filter) + ListAdventuresResponse { traceId, adventures }. 0.23.0 additions (M10 Story Engine Phase 9a — multi-persona scene orchestrator): addPartyMember(sessionId, personaId, { signal? }?): Promise<GetAdventureResponse> and removePartyMember(sessionId, personaId, { signal? }?): Promise<GetAdventureResponse> — both PATCH /api/v1/adventures/{sessionId}/party with { action, personaId } (add: 422 for a DM / non-character / unowned persona; remove: idempotent 200 no-op; 404 for a missing adventure). New types/constants PARTY_ACTION ({ ADD:'add', REMOVE:'remove' }) + PartyAction + PartyUpdateRequest { action, personaId }; StartAdventureRequest gains optional partyPersonaIds?: string[]. ChatWebSocket.send() options + buildHelloPayload() gain optional partyPersonaIds?: string[] (spread only when non-empty → land-dark; advisory — the server's AdventureState.party_persona_ids is authoritative). ChatWsEvent gains a turn_changed member { type, timestamp?, personaId, previousPersonaId? } (mirrors ChatEventType.TURN_CHANGED) and an optional personaId? on the token/done/citation/image/audio_chunk members (orchestrator-stamped; undefined elsewhere → land-dark; per-turn TTS voice swap on audio deferred). R1c additions (Platform Experience Restructure — image adapter registry, no version bump, release is R1e): listImageModels(): Promise<ImageModelsResponse> (GET /api/v1/images/models, authenticated — unlike chat/models it is NOT in DEFAULT_PUBLIC_PATHS; served on the chat base URL via chatRequest) + ImageModelInfo { id, displayName, provider, capabilities: string[], isDefault, isFallback } / ImageModelsResponse { models }. Additive only (no public export removed/renamed). 0.32.0 additions (Platform Experience Restructure R5a+R5c — cast-with-persona projection): getCast(videoId): Promise<CastWithPersonaResponse> (GET /api/v1/tools/videos/{videoId}/cast, tools base URL) — one call returns the character cast pre-joined to bound-persona display fields (the server resolves the PORTRAYS binding; replaces the CastGrid getVideoGraph+listPersonas client-side entity_id join since R5c). New types CastArc/CastScene/CastCharacter/CastPersonaSummary/CastEntry/CastWithPersonaResponse (CastCharacter.scenes: CastScene[] carries {sceneId, title} so a Cast drawer renders scene titles without the graph payload). New castWithPersona(response): CastEntry[] (src/castProjection.ts) — the framework-agnostic pure normalizer (deterministic ordering, list coalescing, title/name fallbacks, input never mutated) every Cast consumer shares. 0.32.0 additions (R5b — story-detail graph read): getVideoGraph(videoId, detail?) gains an optional detail arg — GRAPH_DETAIL.STORY requests the server-collapsed story view (Acts/Scenes/cast/narrative edges; Shots collapsed to per-Scene shotCount properties; DialogLine/Event/MENTIONS omitted), GRAPH_DETAIL.RAW the full graph; omitting it sends NO query param (byte-identical pre-R5b request). New exports GRAPH_DETAIL + GraphDetail (mirror of shared/graph_schema.py). 0.33.0 additions (Platform Experience Restructure R6b — image filtering params): listImages(params) gains optional model?: string + status?: ImageStatus filters, mirroring the new GET /api/v1/images query params. model ids come from R1's image-model registry (discoverable via listImageModels()); the backend rejects an unregistered id with 400 — no client-core image-model descriptor is added (R1's imageAdapter.ts stays the sole catalogue owner). status reuses the existing ImageStatus union (ImageRecord['status']) — no parallel status enum. Omitting both leaves the request byte-identical to 0.32.0 (additive only). 0.44.0 additions (C3a persona-generation, spec Step 11 — auto-generate trigger + job tracking): getPersonaGenSettings(): Promise<PersonaGenerationSettings｜null> (GET /api/v1/personas/settings/generation, null when unsaved), updatePersonaGenSettings(body): Promise<PersonaGenerationSettings> (PUT partial merge — UpdatePersonaGenSettingsRequest all-optional), generatePersonasForVideo(videoId): Promise<GeneratePersonasJobResponse> (POST /api/v1/videos/{videoId}/personas/generate, 202 envelope, idempotent on the active (user,video) job), getPersonaJob(jobId): Promise<AutoGenerationJob> (GET /api/v1/persona-jobs/{jobId}, owner-scoped poll — all in src/api.ts), and listPersonasByVideo(sourceVideoId) + a new optional sourceVideoId on listPersonas({ sourceVideoId }) (→ the source_video_id query param). New exported types PersonaGenerationSettings, UpdatePersonaGenSettingsRequest, AutoGenerationJob, PersonaJobProgress, PersonaJobError, GeneratePersonasJobResponse, PersonaJobStatus, PersonaJobErrorStage; PersonaConfig (+CreatePersonaRequest/UpdatePersonaRequest for the first four) gain optional snake_case entity_name/is_auto_generated/source_video_id/available_clip_indices/clips_used. camelCase wire keys (jobId/traceId) on GeneratePersonasJobResponse ONLY — the backend serializes that model with aliases; AutoGenerationJob/PersonaGenerationSettings are snake_case (no backend aliases). 0.45.0 additions (C3a persona-generation, spec Step 13 — review/audition page): auditionPersona(personaId, body?, { signal? }?): Promise<Blob> (POST /api/v1/personas/{id}/audition — renders a short TTS sample in the persona's current voice; returns the audio/wav blob the review page plays via URL.createObjectURL+<audio>; an omitted body uses the backend default sample text; bypasses the JSON wrapper since the response is binary, same pattern as getSeedImageBlob) and reclonePersona(personaId): Promise<ReclonedPersonaResponse> (POST /api/v1/personas/{id}/reclone — re-clones the voice from a different subset of the cached persona-clips WITHOUT re-downloading video; swaps voice_id, appends to clips_used). New exported types AuditionRequest, ReclonedPersonaResponse (camelCase wire — personaId/voiceId/clipsUsed), RecloneConflictCode, and a RecloneConflictError class — reclonePersona bypasses the generic JSON error path to raise this typed error on a 409 (whose detail is a structured object the flattener would render as [object Object]) so the review UI can branch on voice_clone_max_retries_exhausted vs no_candidate_clips_remaining and disable the Reclone control. Additive only; no public export removed/renamed (../Web-App pins this package). The persona-gen pages (settings/review/list — spec Steps 12–14) consume these. 0.46.0 additions (C3a-f1 — persona-gen trigger relocated to Studio Cast): generatePersonasForVideo(videoId, opts?: { entityIds?: string[] }) gains an OPTIONAL second arg — a non-empty entityIds sends a snake_case { entity_ids: [...] } body (mirrors the new GeneratePersonasRequest backend model) restricting generation to a character subset (Cast per-member "Generate persona"); omitting it (or entityIds: []) sends NO body — byte-identical to the prior 1-arg call (whole-video). The prior signature still type-checks (additive). 0.49.0 additions (dashboard A-DM0 — DM scene control): DmDirectiveKind ('enter_world'｜'exit_world'｜'seek_video'｜'set_experience_source' — single-sourced with the backend DM_DIRECTIVE_KINDS) + DmDirectiveEvent { type:'dm_directive'; timestamp?; directive: DmDirectiveKind; params: Record<string,unknown> }. Additive type exports only; no public export removed/renamed. See CHANGELOG.md. | | admin | 0.19.0 (GitHub #215 Phase 2): AdminClient — the prompt-defaults editor surface. Constructed with the chat base URL + session token (the admin router is mounted in api/chat/app.py). Methods: listPromptDefaults() → PromptDefaultsListResponse; getPromptDefault(key) / updatePromptDefault(key, {value}) / revertPromptDefault(key) → PromptDefaultDetail. Types: PromptDefaultSource ('db'｜'code'), PromptDefaultSummary, PromptDefaultDetail, UpdatePromptDefaultRequest. 0.24.0 (M12 Slice 5 — eval-observability reader): added listEvalRuns(params, {signal}) → EvalRunsListResponse (one cursor page) and listAllEvalRuns(params, {signal}) → AllEvalRunsResult ({runs, truncated}; follows nextCursor until null, capped at MAX_EVAL_RUN_PAGES = 50 — stops not throws, and sets truncated: true when the cap is hit with pages still pending so the page warns rather than presenting a partial series as complete). params carries the XOR selector benchmarkId or promptVersion (server returns 422 on neither/both) + optional limit/cursor. Types: EvalRunSummary, EvalRunsListResponse, ListEvalRunsParams, AllEvalRunsResult (mirror api/admin/schemas.py exactly). The /observability web page is the consumer (charts metric scores per promptVersion, one line per modelId, grouped client-side). Additive only — no ApiClient export changed. Shares the internal http.ts requestJson helper that ApiClient.makeRequest now also delegates to (DRY, private — not re-exported). The server enforces the editable-key carve-out (extraction keys are not editable → 404 → thrown Error). R6d (read-only regen base prompts): added listRegenBasePrompts() → RegenBasePromptsResponse (GET /api/v1/admin/regen-prompts; types RegenBasePrompt/RegenBasePromptsResponse mirror api/admin/schemas.py; editable always false, no write method exists). Also additive on api.ts: RegenerateRequest.appliedDocumentIds?: string[] | null (cap 20; server silently dedups + drops unknown ids) and RegenerationRunResponse.appliedDocumentIds?: string[] mirror the R6c backend fields. 0.47.0 (C4 Phase 3 — admin monitoring tools, data layer): added a separate read-only MonitoringClient (also chat-base + session token) wrapping /api/v1/admin/monitoring/*: listMetrics() → MonitoringMetricCatalogResponse (the allowlist; pick a metricId — no raw namespace/name pass-through), getMetricSeries(metricId, {rangeSeconds, periodSeconds}, {signal}) → MonitoringMetricSeriesResponse (CloudWatch GetMetricData; unknown id → 404 → thrown Error), listTraces({jobId, videoId, rangeSeconds}, {signal}) → MonitoringTraceSummariesResponse (X-Ray; bad annotation id → 422), getTraceDetail(xrayTraceId, {signal}) → MonitoringTraceDetailResponse (found=false when X-Ray has no such trace). Types mirror api/admin/schemas.py exactly. Additive only — no ApiClient export changed. | | reactor/ | Pure builders for the /interactive page in seer-agent (and reusable by the Vue Web-App). buildRegenPrompt(input) produces deterministic Gemini Flash Image prompts with locked 16:9 + identity guarantees plus a creative-direction escape hatch. buildUpsamplePrompt(input) returns {system, user} message blocks for any LLM transport, encoding the Helios prompt-guide rules and weaving in entity/persona/previous-prompt context. aggregateEntityContext and aggregatePersonaContext adapt our existing ImageRecord / EnrichmentRecord / PersonaConfig types into the canonical context shapes. Phase 2: mapEnrichmentToTimeline(enrichment, opts) consumes a EnrichmentRecord and returns a TimelineBeat[] (chunk-pinned prompts) for the /interactive/director timeline UI. Defensively probes multiple Pegasus result shapes. Phase 3: heliosTools.ts — HeliosCommand (discriminated union), parseHeliosCommands(text) (extracts [SET_SCENE: "…"]-style inline markers from streamed chat output), stripHeliosCommands(text) (removes markers from the displayed transcript), and the future-use BEDROCK_TOOL_SCHEMAS Converse-tool-spec export. dmSceneControlPrompt.ts (A-DM2, 0.50.0) — the SINGLE canonical [SCENE:...] scene-control vocabulary (getDmSceneControlPromptText() + DM_SCENE_CONTROL_ADDON_SENTINEL), mirroring the backend shared/adventure_prompts.py::DM_SCENE_CONTROL_INSTRUCTIONS (parsed by DM_SCENE_TAG_PATTERN). toolPromptAddon.ts — HELIOS_WORLD_STEERING_ADDON_ID, getHeliosSteeringAddonText() (now delegates to getDmSceneControlPromptText() — the legacy [SET_SCENE] world-steering prose is RETIRED, tech-debt #113; the export + addon id are kept for the pinned consumer), idempotent buildHeliosSteeringSystemPrompt(base) for personas with prompt_addon_ids: ['helios-world-steering']. NOTE: the heliosTools.ts runtime command layer above (HeliosCommand/parseHeliosCommands/BEDROCK_TOOL_SCHEMAS) is a SEPARATE, unchanged concern (world-adapter command dispatch, not model-facing steering prose). entitySearch.ts — findEntityImageBySeedName(apiClient, videoId, name) resolves an entity name to its ImageRecord (searches extracted then character, case-insensitive). Per-(videoId, category) in-process cache so repeat SWAP_SEED commands skip the API round-trip; forceRefresh: true and clearEntitySearchCache() bypass it. heliosCommandDispatcher.ts — dispatchHeliosCommand(cmd, ctx) executes a HeliosCommand against a structural Reactor bridge (sendCommand + uploadFile); pure async function, no React. HeliosToolContext.signal?: AbortSignal cancels in-flight upsample + seed-image work on unmount. seedImageOrchestrator.ts — loadSeedImageFile(imageId, apiClient) + sendSeedFileToReactor(file, reactor, opts) + convenience loadAndApplySeedImage. Used by all three /interactive React pages AND by the SWAP_SEED branch of the dispatcher, so the load → upload → set_image sequence has exactly one implementation. No @reactor-team/js-sdk dependency — that stays in the consuming app. applyScenePrompt.ts — applyScenePrompt({prompt, apiClient, reactor, currentChunk, isRunning, ...}): shared submit policy for a live Helios session. !isRunning → set_prompt + start; running → schedule_prompt at currentChunk + 2. Upsample failure falls back to the raw prompt; abort propagates. Used by both InteractiveController and the watch-page HeliosWorldControls so the two pages cannot diverge. LongLive-v2 director initiative (Step 2, additive, land-dark — no version bump, R1e releases): longLiveCommandDispatcher.ts — dispatchLongLiveCommand(cmd, ctx), the dispatchHeliosCommand analogue for the Reactor LongLive-v2 world model over the SAME Reactor transport (only the modelName differs). LongLiveCommand is its OWN discriminated union (set_shot/scene_cut/schedule_shot/schedule_scene_cut/pause/resume/reset/set_seed) — NOT a HeliosCommand. Mappings to reactor.sendCommand: set_shot{prompt} → set_shot then start only when !ctx.hasStarted (the opening-shot-then-start guard, mirroring Helios set_scene chunk-0); scene_cut{prompt} → scene_cut (a HARD, memory-purging transition Helios lacks); schedule_shot/schedule_scene_cut use the snake_case at_session_chunk wire field; pause/resume/reset → {}; set_seed{seed} → set_seed. LongLiveToolContext is NARROWER than HeliosToolContext: reactor: Pick<HeliosReactorBridge, 'sendCommand'> (reuses the structural bridge — no rename, no uploadFile: LongLive is text-to-video with NO image conditioning, hence NO swap_seed), hasStarted?, signal?, onUserWarning?. Same failure policy: clean abort silent (reuses isAbortError), other failures logged + skipped. SDK-free (no @reactor-team/js-sdk import). R8.1 (0.39.0): storyTimeline.ts — deriveStoryTimeline(scenes: SceneView[], options?) maps the collapsed R5 detail=story projection (one SceneView per scene) to TimelineBeat[]: one beat per scene, prompt grounded in the scene summary + featured-character names + location label, scene-id provenance in the label. The story-graph-driven sibling of mapEnrichmentToTimeline (which stays the labelled FALLBACK for graph-less videos). chunkDistribution.ts (internal — NOT barrel-exported) holds the shared computeChunkIndices/enforceSpacing so BOTH mappers share ONE chunk-distribution algorithm (extracted from scenesFromEnrichment.ts; DRY — one place can break). StoryTimelineOptions mirrors MapEnrichmentOptions. Spec: specs/interactive-helios-prototype.md. | | adapters/ | R1a (Platform Experience Restructure): the framework-agnostic model-adapter registry primitive every later R1 adapter (image / world / director domains) registers into. Zero React / DOM / provider-SDK imports — same discipline as reactor/ and vision/. capabilities.ts — Capability is the single capability union, formed by extending the existing ChatModelCapability (api.ts:499, the source of truth that mirrors shared/model_config.py:CAPABILITY_*) with the net-new TS-only domains ImageCapability ('image_gen'｜'image_edit'｜'identity_preserve'), WorldCapability ('world_stream'｜'set_scene'｜'swap_seed'｜'pause_resume'), DirectorCapability ('timeline_schedule'｜'enrichment_driven'). There is no parallel CAPABILITY const map — extend the union, don't re-spell the four chat strings. registry.ts — AdapterRegistry<TDescriptor extends ModelAdapterDescriptor> ({ domain, routes, fallbackRouteName }): register() is additive and throws on a duplicate id (no silent overwrite); get/list/listWithCapability(cap); resolveRoute(name) returns the named route's adapter and degrades to the fallback route's adapter when the route's adapterId is unregistered — NEVER throws (parity with shared/model_router.py:344-360); satisfies(id, required) is true iff the id is registered and advertises every required cap. Route names are the exported ROUTE_DEFAULT/ROUTE_FALLBACK/ROUTE_HIGH_QUALITY string constants (the ROUTE_* analogue of shared/model_router.py:56-59), never inline literals. ModelAdapterDescriptor.metadata is opaque/forward-compat — carried, not interpreted. R1b adds chatAdapter.ts — loadChatAdapterRegistry(client) hydrates an AdapterRegistry<ChatAdapterDescriptor> from GET /api/v1/chat/models and is a FACADE over the backend: model selection stays server-side in shared/model_router.py, so there is no route() / client-side selection table here — it only mirrors the catalogue (one descriptor per model, capabilities copied VERBATIM) so the UI can list / listWithCapability / auto-switch. ROUTE_DEFAULT/ROUTE_FALLBACK both mirror the resolved default id (parity with api/chat/app.py:686-697: defaultModelId when present in models, else the first model by (provider, id); empty models → '' sentinel so the constructor guard still passes and resolveRoute degrades to undefined, never throws). R1c adds imageAdapter.ts — loadImageAdapterRegistry(client) hydrates an AdapterRegistry<ImageAdapterDescriptor> from GET /api/v1/images/models (via ApiClient.listImageModels(), served on the chat base URL since the images API is mounted on the chat app). Same FACADE discipline as the chat adapter — image generation still resolves to Gemini in the pipeline unchanged (R1c is land-dark); this only mirrors the catalogue so a future UI (R6) can list / listWithCapability('identity_preserve'). ROUTE_DEFAULT mirrors the isDefault-flagged model id (else first by id), and ROUTE_FALLBACK the isFallback-flagged model (the registry's real fallback; mirrors default when none flagged) — empty models → '' sentinel so the constructor guard still passes and resolveRoute degrades to undefined. ImageModelInfo/ImageModelsResponse wire types live in api.ts (capabilities: string[] — cast to Capability[] only inside the adapter to avoid an api.ts↔capabilities.ts cycle). R1d adds worldAdapter.ts + directorAdapter.ts — the net-new WORLD (step-into-video) and DIRECTOR registries, seeded with the one model that exists today: Helios. worldAdapter.ts — WorldAdapterDescriptor extends ModelAdapterDescriptor with supportedCommands: readonly string[] (model-agnostic — the registry holds heterogeneous world models; a sibling such as a Reactor LongLive adapter has its OWN command vocabulary set_shot/scene_cut/… that is not a HeliosCommand); heliosWorldAdapter (id:'helios', provider 'reactor', caps ['world_stream','set_scene','swap_seed','pause_resume'], supportedCommands = the EXACT HeliosCommand variant set from reactor/heliosTools.ts:32-37) CENTRALIZES the 'helios' model-name literal so the R8 page rewire stops hard-coding it; createWorldRegistry() pre-registers it with ROUTE_DEFAULT+ROUTE_FALLBACK both → its id; requiredCapabilityForCommand(cmd) is the pure exhaustive-switch mapping (set_scene→'set_scene', swap_seed→'swap_seed', pause_world/resume_world→'pause_resume', reset_world→'world_stream'). dispatchWorldCommand(adapterId, cmd, ctx) routes in FRONT of the existing dispatchHeliosCommand (NOT a rewrite): the Helios branch delegates with IDENTICAL args — the live Helios path is UNCHANGED (LAND-DARK) — and an unknown id throws the typed UnsupportedWorldAdapterError WITHOUT touching the bridge. It reuses the structural HeliosReactorBridge via HeliosToolContext — no @reactor-team/js-sdk import, the adapters/ submodule stays SDK-free. directorAdapter.ts — DirectorAdapterDescriptor extends ModelAdapterDescriptor with worldAdapterId; heliosDirectorAdapter (id:'helios-director') backs onto heliosWorldAdapter and its capabilities are the director members ['timeline_schedule','enrichment_driven'] plus the four world caps it inherits via worldAdapterId (deduped); createDirectorRegistry() pre-registers it. LongLive-v2 director initiative (Step 2 — additive, land-dark, no version bump): worldAdapter.ts adds longLiveWorldAdapter (id:'longlive-v2', displayName 'LongLive 2.0', provider 'reactor', caps ['world_stream','set_scene','scene_cut','pause_resume'], supportedCommands = its 8 LongLive command strings). The ASYMMETRIC capability matrix holds via registry.satisfies: Helios HAS swap_seed and NOT scene_cut; LongLive HAS scene_cut (a hard memory-purging transition) and NOT swap_seed (text-to-video, no image conditioning). createWorldRegistry() now registers both adapters; world ROUTE_DEFAULT+ROUTE_FALLBACK STAY Helios (plain /interactive unchanged — LongLive is resolvable by id + listWithCapability('scene_cut') but is NOT the world default). dispatchWorldCommand is widened via FUNCTION OVERLOADS that keep exact (adapterId, cmd, ctx) typing per adapter — 'helios' → dispatchHeliosCommand (BYTE-IDENTICAL to R1d), 'longlive-v2' → dispatchLongLiveCommand, unknown → throw. No per-command runtime capability gate (the R1d review removed the unreachable satisfies scaffolding): the overloads enforce the cmd/adapter match at compile time, and the asymmetric-matrix INFO is consumed via registry.satisfies() by the Step-3 director UI (to hide the seed-image control for LongLive), not a dispatch gate (resolves tech-debt #414). directorAdapter.ts adds longLiveDirectorAdapter (id:'longlive-director', backs onto longLiveWorldAdapter, inherits scene_cut + the director members, NOT swap_seed). THE DIRECTOR-DEFAULT FLIP: createDirectorRegistry() registers both directors and points ROUTE_DEFAULT → 'longlive-director', ROUTE_FALLBACK → 'helios-director' (the world default stays Helios; only the DIRECTOR default flips). LAND-DARK — nothing reads the director route until Step 3 wires the web director page; the live /interactive and /interactive/director pages are UNCHANGED (their modelName="helios" literals stay; R8 migrates the pages). Purely additive to client-core — no existing public export removed/renamed (the sibling Vue Web-App pins this package); Steps 1–2 carried no version bump (the release is R1e). R1e (0.25.0 — R1's release): index.ts adds the aggregator createPlatformRegistries(client): Promise<PlatformRegistries> — the single entry every downstream wave imports to get all four registries pre-wired. world + director are synchronous seeds (Helios world default; LongLive-v2 director default); chat + image load from their backend catalogues via the existing facades in parallel. It makes NO selection decision — it only composes loadChatAdapterRegistry/loadImageAdapterRegistry/createWorldRegistry/createDirectorRegistry (DRY). PlatformRegistries = { chat, image, world, director }. The seer-agent web/ /admin/adapters showroom inspector is the first consumer; additive-only (no export removed/renamed). | | conversation/ | R2.1 (0.26.0 — Conversation panel core): the framework-agnostic, headless conversation state primitive that lets React (R2.2) and a game engine (PlayCanvas, R7) render the same controller. Zero DOM — same discipline as reactor//adapters/. participant.ts — ConversationParticipant { participantId; personaId?; role; presence; modelId?; capabilities? } + the frozen constant objects ParticipantRole (persona/dm/user) and PresenceKind (PRESENCE_NONE/PRESENCE_FACE_2D/PRESENCE_AVATAR_3D) — constants, never inline literals. presenceRegistry.ts — PresenceRendererRegistry (mirrors reactor/promptAddonRegistry.ts): register(kind, factory)/resolve(kind)/has(kind); a factory returns an opaque PresenceHandle (mount/update/destroy) so React and a game engine register their own renderer; THROWS on duplicate register (fail-loud) and returns the PRESENCE_NONE no-op for any unregistered kind (fail-soft — this is what lets the reserved PRESENCE_AVATAR_3D stub render nothing until R7; tech-debt #421). binding.ts — ConversationContextBinding { kind: 'video'｜'studio_session'｜'persona'｜'none'; videoId?; personaId?; entityId?; adventureSessionId?; partyPersonaIds?; systemPrompt?; systemPromptOverride? } + the ConversationContextKind constant + a single toHelloArgs(binding) mapper that maps 1:1 onto buildHelloPayload so there is exactly one hello-builder (DRY). 0.48.0 (dashboard B1): the PERSONA kind + the personaId field make a standalone persona-face chat a DISTINCT binding so the floater's switch-persona path rebinds (conversationBindingKey keys on personaId) instead of colliding on a shared NONE key; personaId is also a buildHelloPayload arg (same falsy-drop land-dark contract as the other id fields), and the controller still overlays the active participant's personaId on top. effectiveModel.ts — resolveEffectiveModel({ boundModelId, capabilities, needed, override, registry }) is the single, pure capability auto-switch (no I/O; R3's vision picker CONSUMES it — there is no parallel resolveVisionCapability): returns { modelId; reason: 'bound'｜'override'｜'auto_switch'; switchedFrom?; needed }. Capability facts come ONLY from the injected R1 chat registry (listWithCapability/satisfies, typed as a narrow Pick<> so the helper depends on just those two methods) — R2 never hardcodes which models are vision-capable, and fails SOFT to bound when no model advertises the needed capability (a config state the panel surfaces, not a crash). controller.ts — ConversationController, a thin state+reducer over the existing ChatWebSocket (it does NOT reimplement transport/reconnect/protocol): holds participants/activeParticipant/contextBinding/modelOverride? + a reference to the R1 chat registry; connect()/send(text)/setModelOverride/clearModelOverride/addParticipant/removeParticipant/setActiveParticipant/dispose(); emits a typed onState(snapshot) callback (no framework events); the sole buildHello() builds the hello once via buildHelloPayload(toHelloArgs(binding)); re-derives advisory partyPersonaIds (matching chat-ws/adventureMode.ts) on participant changes. Socket adoption: the constructor takes an optional existingSocket?: ChatWebSocket — when supplied it ADOPTS that socket (opens none) and dispose() does NOT close it (the owner does) — the Watch shared-socket invariant (R2.4) so the dispatcher reads one FrameBuffer. ADDITIVE-ONLY: Citation/PlaybackState/ChatModelCapability are IMPORTED from websocket.ts/api.ts, never re-declared (avoids the ambiguous-export trap). R2.3 (0.30.0): the controller's hello is mode:'text', always (it previously inherited buildHelloPayload's tts_stt default, which would have routed the first self-driving connect() into the backend voice pipeline — the controller owns TEXT conversations; the voice hello stays page-owned per the R2.2 finding), onMessage(handler): () => void is the additive reply-event seam (a passthrough to ChatWebSocket.onMessage/offMessage) so a renderer can consume token/citation/done/error/turn_changed frames without reaching into a privately-owned socket, and send(text, opts?) gains additive opts.messages (ConversationSendOptions/ConversationHistoryMessage) — the transcript-owning renderer passes prior turns per send because the backend builds conversation_history solely from each send's messages field (omitted/empty ⇒ wire unchanged). R7.1 (0.35.0 — portable conversation runtime): the multi-persona runtime joins the subsystem. router.ts — advisory routing helpers lifted verbatim from web/app/chat-ws/adventureMode.ts (MultiPersonaFrameFields, turnChangedSpeakerId, resolvePersonaPill+PillPersona, advisoryPartyPersonaIds); ADVISORY ONLY — the server agents/orchestrator/persona_router.py resolve_addressed_persona is authoritative, the client never routes a turn, it only renders the server's turn_changed. roster.ts — ADVENTURE_MUTATION_TOOLS (the SINGLE copy of the four DM mutation tool names, mirroring shared/tool_schemas.py), isAdventureMutationTool, shouldRefreshAdventureRail (the one land-dark gate on adventure-state refresh), filterPartyCandidates+PartyCandidatePersona. runtime.ts — ConversationRuntime (src/conversation/runtime.ts:33): a pure reducer/observer over the existing ChatWsEvent union — applyFrame(event) / getState(): ConversationRuntimeState / subscribe(listener) (fault-isolated, returns unsubscribe) / acknowledgeAdventureState() / releaseTurn(personaId) (R7.2, additive: the host removed the persona holding the turn — activePersonaId returns to null; the only host-driven speaker transition, turn_changed remains the only way the speaker MOVES to a persona). The speaker ONLY moves on a server turn_changed while an adventure is active; done finalizes the per-message persona pill; a DM-mutation tool_result sets staleAdventureState for the host to re-fetch+ack. Immutable state-per-transition, listeners fire only on actual change. types.ts — RosterParticipant/ConversationRuntimeState/ConversationRuntimeOptions (REUSES the R2.1 PresenceKind constants — never re-declared, per the ambiguous-export trap). PlayCanvas portability: the runtime has zero React/DOM imports and runs in plain node — a 3D host feeds it the same frames and renders ConversationRuntimeState directly; the renderer seam (PresenceAdapter) lands in R7.3. Caveat for any engine host: client routing stays advisory — never promote the router helpers to a turn decision. R7.3 (0.36.0 — PresenceAdapter seam, the portability contract): presence.ts — PresenceAdapter, the renderer-agnostic interface the runtime drives: mount(participant)/unmount(personaId)/setActiveParticipant(personaId｜null)/setSpeaking(personaId, speaking)/applyViseme(personaId, viseme: VisemeSymbol) — every method keyed by personaId (N participants → N presence slots, renderer-independent); applyViseme takes the scheduled viseme value from the existing visemeScheduler pipeline, NEVER a DOM node (the React adapter applies it to SVG, a PlayCanvas adapter to morph targets); presenceKindHasStage(kind) is the single text-only gate (a PRESENCE_NONE seat receives nothing beyond mount/unmount). Adapter selection stays on the R2.1 PresenceRendererRegistry — no parallel kind→factory map. presenceBus.ts — bindPresence(runtime, adapter): PresenceBinding, pure glue (no renderer dependency): bind mounts the roster + stages the initial speaker; a speaker swap emits setActiveParticipant(next) + setSpeaking(prev,false) + setSpeaking(next,true); an unstaged/unknown speaker clears the stage to null; PresenceBinding.applyViseme is the SINGLE viseme delivery path (R7.5's voice module consumes it — never instantiate a second path); dispose() is idempotent. Headless conformance harness: src/__tests__/conversation/headlessRenderer.ts (RecordingPresenceAdapter) + presenceBus.test.ts pin the exact call sequence a real renderer receives for a 3-participant cue chain, in plain node — the executable proof of PlayCanvas-portability; an engine implements PresenceAdapter against that sequence with zero client-core changes. R7.4 (0.37.0 — Studio cast roster projection): rosterProjection.ts — castToRoster({ cast, dmPersona, partyPersonaIds }) (src/conversation/rosterProjection.ts:55): the single DRY projection turning R5's one-call cast-with-persona response (getCast → CastEntry[]) + the active AdventureState into RosterParticipant[] (DM seat first, then party members in party order; de-duplicated, DM never doubled; every server-party member gets a seat even when absent from the cast so a turn_changed speaker can always be staged). This is the one place the renderer-facing roster is built — it replaces the pre-R7.4 client-side entity_id ∈ storyCharacterEntityIds join across two stores. Pure (no I/O/DOM/React). R7.5 (0.38.0 — overlay voice parity): voice.ts — VoiceSession (src/conversation/voice.ts): the framework-agnostic voice pipeline driving a full STT/TTS/mic/viseme round-trip over the SAME ChatWebSocket the controller owns/adopts. A THIN composition (not a re-implementation): it composes the existing AudioPipeline (player + VisemeScheduler + AudioSessionClient barge-in + sentence anchoring) + MicrophoneRecorder, and builds the voice (tts_stt) hello through the SINGLE buildHelloPayload — no second viseme scheduler, no second hello-construction site. buildHello() is fixture-diffed against the page's startVoiceChat() payload (the parity oracle, voice.test.ts); start() mirrors the page's iOS user-gesture priming order EXACTLY (player + recorder.prime() in the synchronous prefix — callers MUST invoke it from a click handler's sync prefix); interrupt() = barge-in; dispose() NEVER closes the socket (the Watch shared-socket invariant). Visemes flow through one onViseme callback the renderer routes onto R7.3's PresenceAdapter.applyViseme (the single delivery path). ConversationController gains additive voiceSocket() (hands the shared socket to a VoiceSession) + voiceHelloArgs() (the single hello-args input: context + active persona/model). The text-only overlay default is unchanged (mode:'text'; voice is per-conversation opt-in); /chat-ws stays the page-owned parity oracle (its collapse onto VoiceSession is tech-debt #470). SP1 (0.52.0 — persona system-prompt on the binding): binding.ts gains optional systemPrompt?/systemPromptOverride? (both already valid buildHelloPayload/HelloArgs fields), and toHelloArgs maps them with the same falsy-drop as the id fields — an empty/absent prompt drops both keys, the override flag only rides when a prompt is present (mirrors buildHelloPayload's systemPrompt ? {…} : {} gate). The fix: the text send path reads the prompt from the frame (api/chat/ws.py SendActionMessage.systemPrompt) and does NOT load the persona server-side (only the tts_stt hello does, via _merge_persona_into_hello), so a standalone persona chat that carried only personaId replied in the default assistant voice. controller.send(text, opts?) now threads systemPrompt/systemPromptOverride onto the frame with client-override-wins (an explicit opts.systemPrompt beats the binding's prompt — matching _merge_persona_into_hello's contract), and buildHello() carries the prompt automatically (it already spreads toHelloArgs()). The web floater passes persona.custom_system_prompt/system_prompt_override into the open() binding so a TYPED turn replies in-character. Land-dark: a persona with no custom prompt is byte-identical on the wire. |

Layout

src/
  audioProtocol.ts
  audio.ts
  visemeScheduler.ts
  visemeGeometry.ts
  timeFormat.ts                    ← 0.10.0: formatTimestamp + formatCitationTimestamp
  graphSchema.ts                   ← 0.12.0: graph whitelists + 2 provenance unions; 0.13.0: session-cached fetchGraphSchema(); 0.15.0: Scene/PlotBeat/CharacterArc + userCreatableLabels/userCreatableEdges/allowedPairs (M7 P3a); 0.39.0: STUDIO_NODE_LABELS/STUDIO_EDGE_TYPES/STUDIO_NODE_PROPERTIES/STUDIO_CHARACTER_TYPE read-surface vocab (R8.1)
  storyGraph.ts                    ← 0.39.0: deriveScenes/deriveCast + SceneView/CharacterView/ArcView (lifted from web studio; the single story projection Studio + Director share) (R8.1)
  toolSchemas.ts                   ← 0.11.0: AGENT_TOOL_SCHEMAS (mirror of shared/tool_schemas.py)
  websocket.ts
  api.ts
  vision/                          ← Phase 2 vision publisher
    VisionPublisher.ts             ← RTCPeerConnection lifecycle + SDP/ICE
    visionScheduler.ts             ← fps back-pressure (pure helper)
    types.ts                       ← VisionPublisherConfig, VisionStatus, …
    credentials.ts                 ← 0.10.0: toVisionRtcCredentials()
    startVisionPublisher.ts        ← 0.10.0: startVisionPublisher() + types
    sourceKinds.ts                 ← 0.31.0 (R3a): VISION_SOURCE_KINDS + descriptors
    VisionSourceController.ts      ← 0.31.0 (R3a): source acquisition + lifecycle
    index.ts                       ← submodule re-exports
    __tests__/
      VisionPublisher.test.ts
      visionScheduler.test.ts
  reactor/
    types.ts
    regenTemplate.ts
    upsamplePrompt.ts
    entityContext.ts
    scenesFromEnrichment.ts        ← Phase 2 timeline mapper (raw Pegasus enrichment → beats; the labelled fallback)
    chunkDistribution.ts           ← R8.1: shared computeChunkIndices/enforceSpacing (BOTH mappers; internal, not barrel-exported)
    storyTimeline.ts               ← R8.1: deriveStoryTimeline (collapsed R5 story projection → beats; sibling of scenesFromEnrichment)
    heliosTools.ts                 ← Phase 3 marker grammar
    toolPromptAddon.ts             ← Phase 3 system-prompt addon
    toolPromptAddonId.ts           ← Phase 3 addon ID constant
    entitySearch.ts                ← Phase 3 SWAP_SEED lookup (cached)
    heliosCommandDispatcher.ts     ← Phase 3 HeliosCommand executor (abort-aware)
    longLiveCommandDispatcher.ts   ← LongLive-v2: dispatchLongLiveCommand (set_shot/scene_cut/schedule_*/set_seed; SAME Reactor bridge, text-only, abort-aware)
    seedImageOrchestrator.ts       ← Phase 3+ load → upload → set_image
    sceneEntry.ts                  ← Step-Into-Scene orchestrator (extract + upsample + bootstrap)
    applyScenePrompt.ts            ← Shared upsample + chunk-0-vs-running submit policy
    index.ts                       ← submodule re-exports
  adapters/                        ← R1a: framework-agnostic model-adapter registry (no React/DOM/SDK)
    capabilities.ts                ← Capability union = ChatModelCapability (api.ts) + image/world/director caps
    registry.ts                    ← AdapterRegistry + ModelAdapterDescriptor/NamedRoute + ROUTE_* constants
    chatAdapter.ts                 ← R1b: loadChatAdapterRegistry — FACADE over /chat/models (selection stays server-side)
    imageAdapter.ts                ← R1c: loadImageAdapterRegistry — FACADE over /images/models (catalogue; R1 owns registry, R6 consumes)
    worldAdapter.ts                ← R1d + LongLive: helios/longLiveWorldAdapter + createWorldRegistry (both; world default stays helios) + dispatchWorldCommand (per-adapter overloads → dispatchHelios/LongLiveCommand; SDK-free)
    directorAdapter.ts             ← R1d + LongLive: helios/longLiveDirectorAdapter + createDirectorRegistry (both; DIRECTOR default FLIPPED to longlive-director, fallback helios-director)
    index.ts                       ← submodule re-exports
  index.ts                      ← top-level re-exports
  __tests__/
    api-pollers.test.ts
    audio.test.ts
    loadBundleManifest.test.ts
    visemeGeometry.test.ts
    visemeScheduler.test.ts
    adapters-capabilities.test.ts  ← R1a: Capability ↔ imported ChatModelCapability assignability
    adapters-registry.test.ts      ← R1a: register/get/list/listWithCapability/resolveRoute/satisfies
    adapters-chat.test.ts          ← R1b: loadChatAdapterRegistry facade (verbatim caps, default tiebreak, empty-safe)
    adapters-image.test.ts         ← R1c: loadImageAdapterRegistry facade (verbatim caps, isDefault route seed, identity_preserve filter, empty-safe)
    adapters-world.test.ts         ← R1d + LongLive: createWorldRegistry seeds, supportedCommands↔HeliosCommand parity, asymmetric matrix via satisfies, dispatchWorldCommand helios delegation + longlive routing + unsupported-adapter throw
    adapters-director.test.ts      ← R1d + LongLive: createDirectorRegistry seeds, inherited world caps via worldAdapterId, DIRECTOR default flip (longlive-director) + fallback helios-director
    longLiveCommandDispatcher.test.ts ← LongLive: each LongLiveCommand → correct sendCommand call(s) (set_shot+start guard, snake_case at_session_chunk, abort no-op)
    reactor/
      api-reactor.test.ts
      entityContext.test.ts
      regenTemplate.test.ts
      upsamplePrompt.test.ts
      scenesFromEnrichment.test.ts
      heliosTools.test.ts
      toolPromptAddon.test.ts
      entitySearch.test.ts
      heliosCommandDispatcher.test.ts
      seedImageOrchestrator.test.ts

Consumption — locally in this repo

The repo root is an npm workspace covering packages/* (client-core's own dev tooling). web/ is not a workspace member — it depends on this package via "@escapeaihq/client-core": "file:../packages/client-core", which makes npm create a symlink in web/node_modules/@escapeaihq/client-core. Edits here show up in web/ immediately. Next.js compiles the TS source through transpilePackages: ['@escapeaihq/client-core'], so no build step is required during dev.

Web was deliberately moved out of the root workspace: npm's workspace lockfiles strict-filter platform-specific optional deps, and a lockfile generated on macOS would break next build on Vercel/Linux when it couldn't resolve lightningcss-linux-x64-gnu. The file: symlink gives the same dev experience without polluting web's lockfile.

Consumption — sibling repo (web-app)

For the sibling Vue web-app, @escapeaihq/client-core is a regular dep pulled from the public npmjs.com registry (MIT). No auth required. package.json declares "@escapeaihq/client-core": "0.5.0" (pinned exact while pre-1.0). Plain npm install / npm ci fetches the compiled dist/ artifact — Nuxt dev SSR and production builds both consume that.

Optional: local live edits via npm link, for when you want to iterate here and see changes in web-app without re-publishing:

# one-time setup, after both repos are cloned side-by-side
cd ~/projects/seer-agent/packages/client-core && npm link
cd ~/projects/web-app && npm link @escapeaihq/client-core

Unlink with npm unlink @escapeaihq/client-core to fall back to the registry version. The symlink path used to be the default historic workflow (when the package was on auth-required GitHub Packages); since the move to public npm, the registry path works directly and the symlink is only needed for active cross-repo development.

Where to run npm install when adding/upgrading deps

The repo deliberately runs three install lifecycles, one per consumer:

| Goal | Command | Lockfile that gets updated | |---|---|---| | Add/upgrade a dep used by @escapeaihq/client-core (jest, ts-jest, typescript, etc.) | npm install <pkg> --workspace=@escapeaihq/client-core (from repo root) | repo-root package-lock.json | | Add/upgrade a dep used by web/ (Next.js, React, Tailwind, etc.) | cd web && npm install <pkg> | web/package-lock.json | | Pull in remote changes after a branch update | npm ci from repo root, then cd web && npm ci | both — neither lockfile is rewritten |

web/ is not a workspace member (the cross-platform optional-deps issue described above), so npm install at the repo root will not refresh web/node_modules. Always run installs in web/ separately.

Build & publish

The repo-checked-in package.json points main/types/exports at ./src/index.ts so in-monorepo consumers (web/) can import TypeScript source directly through Next.js's transpilePackages. The published artifact needs to point at ./dist/index.js instead — the publish workflow rewrites these in place via npm pkg set before running npm publish. (npm does not honor publishConfig.main/.types/.exports overrides the way pnpm/yarn-berry do, hence the rewrite.) The on-disk file is never committed with the rewritten paths.

A second post-build step (scripts/add-ext.mjs) appends the correct suffix to extension-less relative imports in dist/*.js. TypeScript's emitter doesn't add them, and Node strict ESM (used by Nuxt/Rollup in the sibling repo) refuses to resolve from './foo' without an explicit extension. The script is directory-aware: a specifier whose target is a directory on disk gets /index.js (./reactor → ./reactor/index.js), a file target gets .js. This matters for the four directory modules re-exported from src/index.ts (reactor, vision, adapters, conversation) — those source re-exports use explicit /index (export * from './reactor/index') so tsc emits the directory form. (A bare .js on a directory specifier was the v0.42.0–0.49.0 packaging bug that broke the sibling Nuxt build; fixed in v0.49.1, tech-debt #486.) After rewriting, the script runs a build-time guard that fails the build if any real relative import/export specifier in dist doesn't resolve on disk under strict ESM — so a broken barrel can never be published again. When adding a new directory module to the barrel, always re-export it as ./<dir>/index, not ./<dir>.

# from the package directory
npm run typecheck   # tsc -p tsconfig.json --noEmit
npm test            # jest (jsdom)
npm run build       # tsc -p tsconfig.build.json && node scripts/add-ext.mjs → dist/

Direct npm publish from a dev machine is not part of the flow — releases go through CI; see below.

Cutting a release

CI publishes to public npm (npmjs.com) on every push of a client-core-v* tag (.github/workflows/publish-client-core.yml). The package ships as MIT, installable without auth. The flow:

# 1. bu