@uitoolbar/visual-editor

SolidJS-based visual editor overlay library. Provides element selection, highlighting, agent management, and React component detection for building visual editing tools.

Features

• Element Selection - Hover and click detection with element filtering
• Selection Overlay - Animated highlight box with lerp smoothing
• Crosshair - Canvas-based crosshair for precise targeting
• Labels - Floating element info labels
• Grabbed Boxes - Animation feedback for selected elements
• Theme System - CSS variables with light/dark mode support
• Agent Manager - Session management with streaming status updates
• React Component Detection - Uses bippy for fiber tree introspection
• Context Provider - Centralized state management

Installation

pnpm add @uitoolbar/visual-editor

Quick Start

import { VisualEditorProvider, VisualEditorOverlay } from '@uitoolbar/visual-editor'

function App() {
  return (
    <VisualEditorProvider>
      <YourApp />
      <VisualEditorOverlay />
    </VisualEditorProvider>
  )
}

Components

VisualEditorProvider

Context provider for overlay state.

<VisualEditorProvider
  defaultActive={false}
  onElementSelect={(element) => console.log('Selected:', element)}
  onElementHover={(element) => console.log('Hovered:', element)}
>
  {children}
</VisualEditorProvider>

VisualEditorOverlay

Main overlay container with all visual components.

<VisualEditorOverlay
  showCrosshair={true}
  showLabel={true}
  lerpFactor={0.15}
/>

SelectionBox

Animated element highlight box.

<SelectionBox
  bounds={{ x: 100, y: 100, width: 200, height: 50 }}
  isActive={true}
  color="var(--ve-selection-color)"
/>

Crosshair

Canvas-based crosshair lines.

<Crosshair
  x={mouseX}
  y={mouseY}
  visible={isActive}
/>

SelectionLabel

Floating label with element info.

<SelectionLabel
  tagName="button"
  bounds={elementBounds}
  componentName="Button"
/>

GrabbedBox

Animation for "grabbed" elements.

<GrabbedBox
  bounds={elementBounds}
  onComplete={() => removeBox()}
/>

Hooks

useVisualEditor

Access editor state and controls.

const {
  isActive,
  setActive,
  hoveredElement,
  selectedElement,
  mousePosition,
} = useVisualEditor()

useAnimatedBounds

Lerp-animated bounds for smooth transitions.

const animatedBounds = useAnimatedBounds(targetBounds, {
  factor: 0.15,
  threshold: 0.5,
})

Theme

CSS variables for customization:

:root {
  --ve-selection-color: #3b82f6;
  --ve-selection-bg: rgba(59, 130, 246, 0.1);
  --ve-crosshair-color: rgba(59, 130, 246, 0.5);
  --ve-label-bg: #1e293b;
  --ve-label-text: #f8fafc;
  --ve-z-index: 999999;
}

Preset Themes

import { themes, applyTheme } from '@uitoolbar/visual-editor'

applyTheme(themes.dark)
applyTheme(themes.light)
applyTheme(themes.highContrast)

Types

interface OverlayBounds {
  x: number
  y: number
  width: number
  height: number
  borderRadius?: string
  transform?: string
}

interface ElementInfo {
  element: Element
  tagName: string
  id?: string
  className?: string
  componentName?: string
}

interface VisualEditorState {
  isActive: boolean
  hoveredElement: Element | null
  selectedElement: Element | null
  mousePosition: { x: number; y: number }
  bounds: OverlayBounds | null
}

Agent Manager

Session management for agent interactions:

import { createAgentManager, AgentProvider } from '@uitoolbar/visual-editor'

const agentManager = createAgentManager()

// Configure with provider and callbacks
agentManager.setOptions({
  provider: myProvider,
  onStart: (session) => console.log('Started:', session.id),
  onStatus: (status, session) => console.log('Status:', status),
  onComplete: (session) => console.log('Complete'),
  onError: (error, session) => console.error('Error:', error),
  onAbort: (session) => console.log('Aborted'),
})

// Start a session
await agentManager.session.start({
  element: selectedElement,
  prompt: 'Make this button blue',
  position: { x: 100, y: 200 },
})

// Abort current session
await agentManager.session.abort(sessionId)

// Retry failed session
await agentManager.session.retry(sessionId)

// Dismiss session (apply changes)
agentManager.session.dismiss(sessionId)

// Undo session (revert changes)
await agentManager.session.undo(sessionId)

// Undo without sessionId (operates on current session)
await agentManager.session.undo()

AgentProvider Interface

interface AgentProvider<T = any> {
  send: (context: AgentContext<T>, signal: AbortSignal) => AsyncIterable<string>
  checkConnection?: () => Promise<boolean>
  getCompletionMessage?: () => string | undefined
  supportsFollowUp?: boolean
  undo?: (sessionId?: string) => Promise<void>  // Revert changes made by the provider
  canUndo?: () => boolean                        // Check if undo is available
}

React Component Detection

Uses bippy for React fiber introspection:

import { 
  setupInstrumentation, 
  getNearestComponentName, 
  getStack 
} from '@uitoolbar/visual-editor'

// Initialize instrumentation (call once at startup)
setupInstrumentation()

// Get component name for an element
const componentName = await getNearestComponentName(element)
// Returns: "Button" | "Header" | null

// Get full stack info
const stack = await getStack(element)
// Returns: { componentName, filePath, lineNumber, frames }

Filtered Components

The following are automatically filtered out:

• React internals (names starting with _)
• Next.js framework components (InnerLayoutRouter, ErrorBoundary, etc.)
• Provider/Context components
• Single-character names

Important: Instrumentation Timing

For component detection to work, instrumentation must be initialized before React renders. In Next.js 15+, use the instrumentation-client.ts file:

// src/instrumentation-client.ts
import { setupExtensionBridge } from '@uitoolbar/visual-editor'

if (typeof window !== 'undefined') {
  setupExtensionBridge()
}

This ensures the React DevTools hook is installed before React loads.

Extension Bridge

For Chrome extensions that need to communicate with the page-world script (where bippy runs):

import { setupExtensionBridge } from '@uitoolbar/visual-editor'

// In your page-world script (not content script)
setupExtensionBridge()

The extension content script can then request React context:

// Content script
const getReactContext = (x: number, y: number): Promise<ReactContext> => {
  return new Promise((resolve) => {
    const requestId = Date.now().toString()
    
    const handleResponse = (event: MessageEvent) => {
      if (event.data?.type !== 'UISTUDIO_REACT_CONTEXT_RESULT') return
      if (event.data?.requestId !== requestId) return
      
      window.removeEventListener('message', handleResponse)
      resolve({
        componentName: event.data.componentName,
        filePath: event.data.filePath,
        lineNumber: event.data.lineNumber,
      })
    }
    
    window.addEventListener('message', handleResponse)
    window.postMessage({
      type: 'UISTUDIO_GET_REACT_CONTEXT',
      requestId,
      x,
      y,
    }, '*')
  })
}

Development

# Run dev mode
pnpm dev

# Run tests
pnpm test

# Build
pnpm build