@iteleport/speechly-browser-client

v2.0.0-beta.2

Published

2 years ago

Browser client for Speechly API

Downloads

Real-time automatic speech recognition and natural language understanding tools in one flexible API

Website | Docs | Discussions | Blog | Podcast

Speechly browser client

With the browser-client you can add voice features to any website. It handles authentication, audio capture, network streaming and connection management with the Speechly Voice API.

Check out the browser-client-example repository for a demo app built using this client.

NOTE: If you are using React, you can use our React client instead. It provides the same functionalities, but provides a programming model that is idiomatic to React.

Usage with Node

Install the package:

# Using Yarn
yarn add @speechly/browser-client

# Using NPM
npm install --save @speechly/browser-client

Start using the client:

import { BrowserClient, BrowserMicrophone, Segment } from '@speechly/browser-client'

// Create a new client.
// NOTE: Configure and get your appId from https://api.speechly.com/dashboard
const client = new BrowserClient({ appId: 'your-app-id' })

// Create a microphone
const microphone = new BrowserMicrophone()
// Initialize the microphone - this will ask the user for microphone permissions
// and establish the connection to Speechly API.
// Make sure you call `initialize` from a user action handler
// (e.g. from a button press handler).
await microphone.initialize()

// bind the microphone to the client
await client.attach(microphone.mediaStream)

// React to the updates from the API.
client.onSegmentChange((segment: Segment) => {
  console.log('Received new segment from the API:',
    segment.intent,
    segment.entities,
    segment.words,
    segment.isFinal
  )
})

// Start recording.
// This can be bound to e.g. a button press.
await client.startContext()

// Stop recording after a timeout.
// This can be bound to e.g. a button press.
setTimeout(async function () {
  await client.stopContext()
}, 3000)

Usage with browsers

This sample HTML loads Speechly's browser-client ES modules via a CDN that mirrors npm packages. The page displays a text field that you dictate text into. See browser's console log for raw segment feed from Speechly.

Please use a HTML server to view the example. Running it as a file will not work due to browser's security restrictions. For example run serve . on command line and open localhost:3000 in your browser.

<html>
  <body>

    <input id="textBox" type="text" placeholder="Hold to talk..." autofocus />

    <script type="module">
      // Load Speechly ES module from a CDN. Note script type="module"
      import { BrowserClient, BrowserMicrophone } from "../core/speechly.es.js"

      const widget = document.getElementById("textBox")

      // Create a Speechly client instance.
      // NOTE: Configure and get your appId from https://api.speechly.com/dashboard
      const speechly = new BrowserClient({
        appId: "your-app-id",
        debug: true,
        logSegments: true,
      })
      const microphone = new BrowserMicrophone()

      speechly.onSegmentChange(segment => {
        // Clean up and concatenate words
        let transcript = segment.words
          .map(w => w.value.toLowerCase())
          .join(" ");
        // Add trailing period upon segment end.
        if (segment.isFinal) transcript += ".";
        widget.value = transcript;
      });

      const startListening = async () => {
        if (microphone.mediaStream === undefined) {
          await microphone.initialize()
          speechly.attach(microphone.mediaStream)
        }
        return speechly.startContext();
      }

      const stopListening = async () => {
        if (speechly.isListening()) {
          return speechly.stopContext();
        }
      }

      // Bind start listening to a widget hold, release anywhere to stop
      widget.addEventListener("mousedown", startListening)
      document.addEventListener("mouseup", stopListening)
    </script>
  </body>

</html>

Documentation

You can find the detailed browser-client API documentation in the GitHub repository.

You can also refer to Speechly Docs for more information.

Contributing

See contribution guide in CONTRIBUTING.md.

Published

Vulnerabilities

Links

Maintainers

Keywords