@vsllabs/webgl-react

Installation:

To install the package, run:

Using npm:

$ npm install @vsllabs/webgl-react

Using yarn:

$ yarn add @vsllabs/webgl-react

Table of Contents

• Usage Example (Normal mode)
• Usage Example (Streaming mode)
• Documentation
• Required Parameters
• Returned Values
• Example Workflow

Usage Example (Normal mode):

Below is an example of how to use the useVslWebGL hook within a React component:

import { useState } from 'react'
import { useVslWebGL } from '@vsllabs/webgl-react';

const app = () => {
    // example text input state
    const [inputText, setInputText] = useState('')

    // invoke the useVslWebGL hook with your personal API_KEY and build URLs
    const { VSLWebGl, unityProvider, translateTextToASL, isUnityLoaded, isTranslating, replay } = useVslWebGL({
        API_KEY: 'Your API Key here',
        streaming_mode: false,    // Optional (false by default)
        // *We will provide these URLs for you
        loaderUrl: 'Unity build Url',
        dataUrl: 'Unity build Url',
        frameworkUrl: 'Unity build Url',
        codeUrl: 'Unity build Url',
     })

  return (
    <div
      style={{
        width: "400px",
        height: "700px",
        position: "relative",
        margin: "auto",
      }}
    >
      {!isUnityLoaded && (
        <div
          style={{
            position: "absolute",
            width: "100%",
            height: "100%",
            top: 0,
            left: 0,
            background: "#999",
            display: "grid",
            placeItems: "center",
          }}
        >
          {/* Your Loading spinner here */}
          Loading...
        </div>
      )}

      {/* The WebGL component for rendering, can be styled as needed */}
      <VSLWebGl
        id="unityWebGl"
        style={{ width: "100%", height: "100%" }}
        unityProvider={unityProvider}
      />

      {/* Example input for the text state */}
      <input
        type="text"
        placeholder="Enter Text..."
        value={inputText}
        onChange={(ev) => setInputText(ev.target.value)}
        style={{
          border: "1px solid black",
          borderRadius: "8px",
          minHeight: "30px",
          width: "100%",
          marginBottom: "16px",
          padding: "8px",
        }}
      />

      {/* Translate button, triggers translation when Unity is loaded and input is provided */}
      <button
        style={{
          border: "1px solid black",
          borderRadius: "4px",
          padding: "8px 16px",
          marginRight: "16px",
          backgroundColor: "green",
          color: "white",
        }}
        type="button"
        onClick={() => translateTextToASL(inputText)}
      >
        {isTranslating ? "Loading..." : "Translate"}
      </button>

      {/* Replay button, replays the last translation */}
      <button
        style={{
          border: "1px solid black",
          borderRadius: "4px",
          padding: "8px 16px",
        }}
        type="button"
        onClick={replay}
      >
        Replay
      </button>
    </div>
  );
}

Usage Example (Streaming mode):

Below is an example of the streaming mode with basic UI using the browser speech recognition for illustration:

import { useState, useEffect } from "react";
import { useVslWebGL } from "@vsllabs/webgl-react";
import "./App.css"; // Importing the styling CSS file (file content is below)

const App = () => {
  const [inputText, setInputText] = useState("");
  const [isStreaming, setIsStreaming] = useState(false);

  const {
    VSLWebGl,
    unityProvider,
    translateTextToASL,
    isUnityLoaded,
    isTranslating,
    replay,
  } = useVslWebGL({
        API_KEY: 'Your API Key here',
        streaming_mode: isStreaming,
        // *We will provide these URLs for you
        loaderUrl: 'Unity build Url',
        dataUrl: 'Unity build Url',
        frameworkUrl: 'Unity build Url',
        codeUrl: 'Unity build Url',
     });

  useEffect(() => {
    if (isStreaming) {
      const SpeechRecognition =
        window.SpeechRecognition || window.webkitSpeechRecognition;

      if (!SpeechRecognition) {
        alert("Speech Recognition is not supported in this browser.");
        return;
      }

      const recognition = new SpeechRecognition();
      recognition.continuous = true;
      recognition.interimResults = false;
      recognition.lang = "en-US";

      recognition.onresult = (event) => {
        const lastResult = event.results[event.results.length - 1];
        if (lastResult.isFinal) {
          const spokenText = lastResult[0].transcript.trim();
          setInputText(spokenText);
          translateTextToASL(spokenText);
        }
      };

      recognition.onerror = (event) => {
        console.error(
          "Speech recognition error:",
          event.error || event.message || event
        );
      };

      recognition.start();

      return () => recognition.stop();
    }
  }, [isStreaming]);

  return (
    <div id="appContainer">
      {!isUnityLoaded && (
        <div id="loadingOverlay">
          <div className="spinner" />
        </div>
      )}

      <div id="webglContainer">
        <VSLWebGl
          id="unityWebGl"
          style={{ width: "100%", height: "100%" }}
          unityProvider={unityProvider}
        />
      </div>

      <input
        id="textInput"
        type="text"
        value={inputText}
        onChange={(e) => setInputText(e.target.value)}
        placeholder="Enter text to translate"
      />

      <div id="buttonGroup">
        <button
          id="translateBtn"
          type="button"
          onClick={() => translateTextToASL(inputText)}
          disabled={!isUnityLoaded || isTranslating}
          className={isTranslating ? "btn translate disabled" : "btn translate"}
        >
          {isTranslating ? "Loading..." : "Translate"}
        </button>

        <button
          id="replayBtn"
          type="button"
          onClick={replay}
          className="btn replay"
          className={isStreaming ? "btn replay disabled" : "btn replay"}
          disabled={isStreaming}
        >
          Replay
        </button>

        <button
          id="streamBtn"
          type="button"
          onClick={() => setIsStreaming((prev) => !prev)}
          className={`btn stream ${isStreaming ? "stop" : "start"}`}
        >
          {isStreaming ? "Stop Streaming" : "Start Streaming"}
        </button>
      </div>
    </div>
  );
};

export default App;

Streaming mode app.css file:

#appContainer {
  width: 800px;
  height: 850px;
  margin: 0 auto;
  position: relative;
  display: flex;
  flex-direction: column;
  align-items: center;
  gap: 1rem;
  padding: 1rem;
  background-color: white;
  border-radius: 1rem;
  box-shadow: 0 10px 15px rgba(0, 0, 0, 0.1);
}

#loadingOverlay {
  position: absolute;
  top: 0;
  left: 0;
  width: 100%;
  height: 100%;
  background-color: #d1d5db;
  display: flex;
  align-items: center;
  justify-content: center;
  border-radius: 1rem;
  z-index: 10;
}

.spinner {
  animation: spin 1s linear infinite;
  border-top: 4px solid #3b82f6;
  border-radius: 9999px;
  width: 3rem;
  height: 3rem;
  border-right: 4px solid transparent;
}

@keyframes spin {
  to {
    transform: rotate(360deg);
  }
}

#webglContainer {
  width: 600px;
  height: 600px;
  position: relative;
  border-radius: 1rem;
  overflow: hidden;
}

#textInput {
  width: 100%;
  padding: 0.5rem 1rem;
  border: 1px solid #d1d5db;
  border-radius: 0.375rem;
  outline: none;
  transition: box-shadow 0.2s;
}

#textInput:focus {
  box-shadow: 0 0 0 2px #3b82f6;
}

#buttonGroup {
  display: flex;
  gap: 0.5rem;
  width: 100%;
}

.btn {
  flex: 1;
  padding: 0.5rem 1rem;
  border-radius: 0.375rem;
  color: white;
  transition: background-color 0.2s;
  cursor: pointer;
  border: none;
}

.btn.translate {
  background-color: #2563eb;
}

.btn.translate:hover {
  background-color: #1d4ed8;
}

.btn.translate.disabled {
  background-color: #93c5fd;
  cursor: not-allowed;
}

.btn.replay {
  background-color: #6b7280;
}

.btn.replay.disabled {
  background-color: #5c616b;
  cursor: not-allowed;
}

.btn.replay:hover {
  background-color: #4b5563;
}

.btn.stream.start {
  background-color: #16a34a;
}

.btn.stream.start:hover {
  background-color: #15803d;
}

.btn.stream.stop {
  background-color: #dc2626;
}

.btn.stream.stop:hover {
  background-color: #b91c1c;
}

Documentation

The useVslWebGL hook provides the necessary setup and functionality for integrating the VSL WebGL component within a React application. It returns an object with various properties and functions for rendering, controlling, and interacting with the WebGL component.

Required Parameters

• API_KEY: Your unique API key for accessing the VSL WebGL services.
• loaderUrl, dataUrl, frameworkUrl, codeUrl: URLs provided by VSL for accessing the Unity WebGL build. Each URL is necessary for loading the Unity environment properly.

Optional Parameters

• streaming_mode: Toggle streaming mode. (boolean)

Returned Values

The following values and functions are returned by useVslWebGL:

| Value | Explanation | | :------------ | :--------------------------------------------- | | VSLWebGl | A JSX component for rendering the Unity WebGL. Can be styled and controlled within a parent component or container.Example: <VSLWebGl style={{ ... }} unityProvider={unityProvider} /> | | unityProvider | Required prop for the VSLWebGl component, provides the Unity instance.Pass this to the unityProvider prop of VSLWebGl to initialize the Unity environment. | | translateTextToASL | Function to trigger text translation within the Unity WebGL.Arguments: Accepts a single argument (the text to translate).Example: translateTextToASL("Hello, world!") | | isUnityLoaded | Indicates whether the Unity WebGL component has fully loaded. Useful for checking readiness to show loaders and before triggering translation.Example: Disabling the translate button until Unity is ready. | | isTranslating | Represents the loading state during the translation process. Helpful for displaying loading indicators.Example: {isTranslating ? 'Translating...' : 'Translate'} | | replay | Function to replay the last translated text within the Unity WebGL.Arguments: No arguments required. | | changeBgColor | Changes the background color of the WebGL component.Arguments: Accepts a single argument, a string representing a hex color value (e.g., #FFFFFF for white).Example: changeBgColor('#FF5733') to set the background color to a shade of orange. | | setAnimationSpeed | Controls the speed of animations within the WebGL environment.Arguments: Accepts a single string argument, which can be one of four options: "0" (for pause), "0.5", "1" (default), or "1.5".Example: setAnimationSpeed("1.5") to set the animation speed to 1.5x. | | toggleCameraRotation | Toggles the rotation of the camera in the WebGL environment.Arguments: Accepts a single boolean argument to enable (true) or disable (false > default) camera rotation.Example: toggleCameraRotation(true) to enable camera rotation. | | error | If any errors occur during loading or translation, this string provides an error message explaining the issue.Example: Display error in your UI if it’s not an empty string.Note: Errors are also logged in the console | | streaming_mode | Set to true to use streaming mode. Example: When streaming mode is on, text provided to translateTextToASL function will be added to a queue where the animations will play in sequential manner. Note: When set to false and a new text is provided to translateTextToASL while an animation is playing, will override it and start animating the new sentence. Note: When streaming mode is on the reply function cannot be used, invoking it will do nothing |

Example Workflow

1. Initialize the Hook: Call useVslWebGL with the required parameters to initialize the WebGL component.
2. Render the Component: Use in your component, styled to fit your layout.
3. Translate Text: Use the translateTextToASL function to translate input text when Unity is loaded (isUnityLoaded).
4. Replay Last Translation: Use the replay function to repeat the last translation as needed.
5. Handle Errors: Check the error value to catch and display any issues that occur during loading or translation.

License

MIT