SMILE — Seamless Media and Interactive Learning Experience

A React-based framework for creating interactive, animated educational presentations with synchronized audio, declarative scene definitions, and an extensible plugin system.

Overview

SMILE enables creators to produce rich learning experiences that combine smooth animations, interactive quizzes, and audio narration — all defined declaratively through JSON scene definitions. It supports a plugin architecture for custom shapes and animations, a tile-based responsive layout system, and LLM-friendly scene authoring.

Key Features

• Declarative Scene Definitions — Define scenes as JSON with shapes, animations, audio, and layout
• Plugin Architecture — Extensible shape and animation plugins with a central registry
• Tile-Based Responsive Layout — All positions/sizes use virtual grid units that scale to any screen size
• Audio Synchronization — Timeline animations synchronized with audio playback, with buffering handling
• Interactive Components — Built-in quiz plugin with multi-choice cards and 3D flip animations
• Playback Controls — Full media player controls: play/pause, seek, volume, fullscreen
• Event System — Pub/sub event system for inter-component and plugin communication
• LLM-Compatible — Scene definitions designed for GenAI-driven content generation

Quick Start

Installation

npm install @duongtdn/smile

Basic Usage

import React from 'react';
import { SmilePlayer, PluginRegistry, builtin } from '@duongtdn/smile';

// Register built-in plugins
PluginRegistry.registerPlugin(builtin);

const sceneDefinition = {
  id: 'my-scene',
  layout: {
    aspectRatio: '16:9',
    gridSize: [32, 18],
    backgroundColor: '#1a1a2e'
  },
  shapes: [
    {
      id: 'title',
      type: 'text',
      position: { x: 4, y: 3 },
      size: { width: 24, height: 4 },
      pluginRef: 'builtin',
      props: {
        content: 'Hello SMILE!',
        format: { fontSize: 2, color: '#ffffff', fontWeight: 'bold' }
      }
    }
  ],
  animations: [
    {
      target: 'title',
      pluginRef: 'builtin',
      animation: { type: 'typing', params: { duration: 50, staggerDelay: 80 } },
      at: 0
    }
  ],
  audio: { src: '/narration.mp3' }
};

function App() {
  return (
    <div style={{ width: 720 }}>
      <SmilePlayer scene={sceneDefinition} />
    </div>
  );
}

Architecture

┌─────────────────────────────────────────────────┐
│ SmilePlayer                                     │
│  ┌─────────────────────────────────────────────┐│
│  │ PlaybackEventProvider (React Context)       ││
│  │  ┌─────────────────────────────────────────┐││
│  │  │ Scene                                   │││
│  │  │  ├── Shape Rendering (via PluginRegistry│││
│  │  │  ├── Layout & Resize (ResizeObserver)   │││
│  │  │  ├── PlaybackControl                    │││
│  │  │  │    ├── useAnimationTimeline          │││
│  │  │  │    ├── useAudioPlayback              │││
│  │  │  │    ├── useSynchronizer               │││
│  │  │  │    └── useFullscreen                 │││
│  │  │  └── BufferingSpinner                   │││
│  │  └─────────────────────────────────────────┘││
│  └─────────────────────────────────────────────┘│
└─────────────────────────────────────────────────┘
         │                       │
    PluginRegistry          PlaybackEventSystem
    ├── builtin             (pub/sub events)
    │   ├── Shapes: Circle, Rectangle, Line, Text, CompositeRectangle
    │   └── Animations: draw, move, fadeIn, fadeOut
    └── quiz
        └── Shapes: MultiChoice (with enter animation)

Project Structure

packages/
├── index.ts                      # Public API exports
├── core/
│   ├── plugin-registry/          # Singleton plugin management
│   └── runtime/
│       ├── components/           # React UI (SmilePlayer, Scene, PlaybackControl)
│       ├── events/               # Playback event system (pub/sub)
│       ├── hooks/                # React hooks (timeline, audio, sync, fullscreen)
│       ├── lib/                  # TimelineAnimationFactory (anime.js integration)
│       └── types/                # TypeScript type definitions
├── plugins/
│   ├── types.ts                  # Plugin interface definitions
│   ├── builtin/                  # Built-in shapes & animations
│   └── quiz/                     # Quiz plugin (MultiChoice)
playground/                       # Development playground app
docs/                             # Documentation

Core Concepts

Scene Definition

A scene is defined as a SceneDefinition object:

interface SceneDefinition {
  id: string;
  layout: LayoutConfig;       // aspectRatio, gridSize, backgroundColor
  shapes?: Shape[];           // Visual elements
  animations?: Animation[];   // Timeline-based animations
  audio?: AudioConfig;        // Audio narration
}

Tile-Based Coordinate System

All positions and sizes use tile units rather than pixels. The grid is defined by gridSize (e.g., [32, 18]):

• Tile width = container pixel width / grid columns
• Tile height = container pixel height / grid rows

This makes scenes fully responsive — on resize, tile dimensions are recalculated, and all elements reposition automatically.

Plugin System

Plugins provide shapes (visual elements) and animations (transformations over time). Each plugin implements the Plugin interface:

interface Plugin {
  name: string;
  shape?: Record<string, ShapePlugin>;
  animation?: Record<string, AnimationPlugin>;
}

Plugins are registered with PluginRegistry.registerPlugin(plugin) and resolved automatically when scenes reference them.

Animation Timeline

Animations are orchestrated via anime.js timelines. Each Animation definition targets one or more shapes and specifies a type, parameters, and timeline position:

interface Animation {
  target: string | string[];         // Shape ID(s)
  pluginRef?: string;                // Which plugin provides the animation
  animation: {
    type: string;                    // e.g., 'draw', 'move', 'typing'
    params?: Record<string, any>;    // Duration, easing, direction, etc.
  };
  at?: number | string;              // Timeline position (ms or relative)
}

Built-in Shapes

| Shape | Description | Key Props | |-------|-------------|-----------| | Circle | SVG circle/ellipse | fill, lineColor, lineWidth, lineStyle | | Rectangle | SVG rectangle with optional rounded corners | fill, lineColor, lineWidth, lineStyle, rx, ry | | Line | SVG polyline with optional arrow | points, lineColor, lineWidth, lineStyle, showArrow | | Text | Rich text with per-character animation support | content, format, ranges | | CompositeRectangle | Rectangle with embedded text | All Rectangle + Text props, align, padding |

Built-in Animations

| Animation | Compatible Shapes | Description | |-----------|-------------------|-------------| | draw | Circle, Rectangle, Line, CompositeRectangle | Progressive stroke reveal | | move | All | Position change (absolute or relative +=/-=) | | fadeIn | All | Opacity 0 → 1 | | fadeOut | All | Opacity 1 → 0 | | typing | Text | Character-by-character reveal | | untyping | Text | Character-by-character hide | | floatIn | Text | Directional slide + fade in | | floatOut | Text | Directional slide + fade out | | enter | MultiChoice (quiz) | Full quiz entrance sequence |

Playback Event System

Components and plugins communicate via a pub/sub event system:

| Event | Purpose | |-------|---------| | scene-ready | Scene initialized and ready to play | | create-timeline | Request to build/rebuild animation timeline | | timeline-recreated | Timeline has been rebuilt | | halt-animation / unhalt-animation | Pause/resume during resize | | playing / paused / seeking | UI state notifications | | resume / pause | Programmatic play/pause control | | lock-hide-controls / unlock-hide-controls | Control visibility during interaction |

Development

Prerequisites

• Node.js (v18+)
• npm

Setup

git clone https://github.com/duongtdn/smile.git
cd smile/core
npm install

Scripts

| Command | Description | |---------|-------------| | npm start | Start playground dev server (port 3330) | | npm run build | Build the library to dist/ | | npm test | Run all tests | | npm run test:watch | Run tests in watch mode | | npm run test:components | Run component tests only (jsdom) | | npm run test:node | Run node tests only | | npm run test:coverage | Run tests with coverage report |

Tech Stack

• React 19 — UI framework
• TypeScript — Type-safe development
• anime.js v4 — Animation engine
• Jest — Testing framework
• Testing Library — React component testing
• Webpack — Bundling (playground dev server)

Documentation

• Usage Guide — How to use SMILE in your application
• Developer Guide — How to extend SMILE with custom plugins
• Architecture & Design — Technical architecture and detailed design
• Plugin Guide — Plugin system API reference
• Event System — Playback event system documentation
• Move Animation — Move animation plugin documentation
• Virtual Grid — Tile-based layout system specification
• Test Summary — What is tested and coverage overview
• Quiz Plugin — Quiz plugin documentation

License

UNLICENSED

Author

Duong Nguyen (duongtdn)