Voice Control Library

Web application interface in the form of a library for controlling web content via voice commands. Only after a proper voice command from the end user, which must be sequential and distinct for its adequate recognition, can the library handle page content. VCL uses W3C speech api for voice recognition to text form.

Functionality

The main advantage library is simple implementation new and creative functions that improve work on the internet. Library is fully available on Google Chrome browser.

Check your web browser can to use this library on : https://whatwebcando.today/speech-recognition.html

Library you can turn on :

vcl.execute();

or the keyboard shortcut Ctrl + Shift + V

API

You can choose :

• from 120 human language
• debug mode in web browser console
• continuous recording voice
• count of maximal alternatives recognize result

Client

You can implement :

• query selectors corresponding to elements on the page
• phrases by which will be unique on what functionality goes and which element are set
• listener can be implemented or not
  • if is set, execute first
• actions various charater that executes only if return value is true

Configuration example

const vcl = new VCL({
    "#btn": {
        phrases: ['klikni na gombík', 'klikni na button',
            'klikni na tlačítko', 'stlač button', 'stlač tlačidlo'],
        listener: event => {
            event.element.innerText = event.result;
        },
        actions: {
            click: true,
            mouseover: event => {
                return (event.element.innerText === "Button");
            }
        }
    },
    global: {
        speechApi: {
            W3C: {
                lang: 'sk-SK',
                continuous: false,
                maxAlternatives: 1
            },
            GoogleCloudApi: {},
            debug: true
        },
    }
});

Documentation

Table of Contents

• VCL
• -------------------- Voice Control Library (VCL) --------------------
  • Parameters
  • execute
• SpeechApi
  • Parameters
  • dictate
  • result
  • stop
• W3CApi
  • Parameters
  • dictate
  • result
  • stop
• ServiceObjects
  • Parameters
  • trie
  • controlObjects
  • divideControlObjects
    • Parameters
• Trie
  • root
  • insert
    • Parameters
  • search
    • Parameters
  • delete
    • Parameters
  • update
    • Parameters
  • getDataNode
    • Parameters
  • getPath
    • Parameters
• TrieNode
  • Parameters
  • parent
  • children
  • update
    • Parameters
  • unlink
  • hasChildren
  • deleteChild
    • Parameters
  • addChild
    • Parameters
  • hasChild
    • Parameters
• ControlObject
  • Parameters
• Runner
  • run
    • Parameters
  • executeEvents
    • Parameters
  • executeListener
    • Parameters
• VCLEvent
  • Parameters
  • actions
  • phrases
  • element
  • result
  • listener

VCL

-------------------- Voice Control Library (VCL) --------------------

Web application interface in the form of a library for controlling web content via voice commands. Only after a proper voice command from the end user, which must be sequential and distinct for its adequate recognition, can the library handle page content. VCL uses W3C speech api for voice recognition to text form.

The VCL class is at the core of the entire library that handles execution controlling web content via voice commands.

Parameters

• config Object Can be object, JSON, or external file with object structure. Must contains global part configuration and also at least one querySelector with his properties.

execute

The only user-accessible function.

I. Starts recording audio.

II. On result from speech API
1. If set debug mode, allow write in web browser console. 2. Search query selector assigned to HTMLElement on page by matching speech recognition and indexed phrases.
3. Start executing listener and actions functions.
4. Stop recording audio.

SpeechApi

Facade that separates API functionality for speech recognition from control and others VCL services. Provides the isolation of what a library user can and where they should not be accessed.

Parameters

• config Object SpeechApi configuration.

dictate

Calls the configured function

result

Returns Promise

stop

Calls the configured function

W3CApi

W3C Speech API It enables developers to use scripting to generate text-to-speech output and to use speech recognition as an input for forms, continuous dictation and control.

Parameters

• config (Object | null) W3C Speech API configuration

dictate

Begin to listen to the audio.

result

Asynchronous function that return transcript result in text form.

Returns Promise<String>

stop

Stop listening to more audio.

ServiceObjects

Services input configuration settings related to page elements and their properties. Phrases are indexed to the associated selector using the Trie class

Parameters

• config Object

trie

Get Trie.

Returns Trie

controlObjects

Get map all control objects.

Returns Map

divideControlObjects

In Map on key position set querySelector from configuration and as his value set instance of ControlObject with his attributes. Insert all phrases from the configuration into the trie that indexes to their querySelectors.

Parameters

• config Object

Trie

Trie data structure.

root

Get root node of the Trie

Returns TrieNode

insert

Insert word to the Trie and map data on the word. If data is not provided it is automatically generated as an increasing number.

Parameters

• word string
• data Object?

search

Searching Trie for data indexed by the provided word. If the word is in the Trie a data object is returned. If the word is not found in the Trie null is returned.

Parameters

• word string

Returns (Object | null)

delete

Delete word from the Trie. If the word is not in the Trie false is returned otherwise true.

Parameters

• word string

Returns boolean

update

Parameters

• word string
• data any

Returns boolean

getDataNode

Parameters

• word string

Returns TrieNode

getPath

Parameters

• word string

Returns Array<TrieNode>

TrieNode

Node of a trie data structure. The node can be root or any other node in Trie data structure. The has access to its parent and all of its children. If the node is index of the word inserted into the Trie it has flag isEndOfWorld set to true and word is set to the indexed word. If the node is the end of a word it can contain some additional index data.

Parameters

• parent Object Parent config object (optional, default {key:"",node:null})
• isRoot boolean? Boolean flag of root node. If the node is root it is not check for parent (optional, default false)

parent

Get parent object consisting of the child index and parent node.

Returns {key: string, node: TrieNode}

children

Get map of all node's children.

Returns ({} | {TrieNode}) Child index is one character of a inserted word and value is child's node object.

update

Update indexed data of the node.

Parameters

• data any If data is set to some value the node is automatically set as the end of a word. If data has false value indexed word is removed from the node.

unlink

Remove parent object from this node. If this function is finished all reference to this node from the Trie root is lost.

hasChildren

Check if the node has any child nodes attached to it.

Returns boolean True if has any children, otherwise false.

deleteChild

Delete child indexed by the provided character. If the child does not exists nothing happens. If the child does exists, the child node object is deleted.

Parameters

• char string

addChild

Add a child to the node. If a child already exists on the index it is overridden by the new child.

Parameters

• char string
• node TrieNode

Returns (TrieNode | null) If a child is overridden the old child node is return, otherwise false.

hasChild

Check is the node has a child indexed by the provided character.

Parameters

• char string

Returns boolean True if a child exists, otherwise false.

ControlObject

Keeps data configured by user. Find HTMLElement if exists on page. Simplify phrases by putting all the letters in the lowercase and removes accent from them.

Parameters

• querySelector String
• controlElementValue Object

Runner

At first verify configured event can be execute. Then execute all listener function and event functions configured by user.

run

If listener exist, first execute listener and then all others events.

Parameters

• recognizeResult String
• controlObject ControlObject

executeEvents

Execute all configured actions. First check if action is function or boolean and then is execute only if return value from action function is true.

Parameters

• vclEvent VCLEvent

executeListener

If listener is function, execute it.

Parameters

• vclEvent VCLEvent

VCLEvent

All attributes user can access using and are apply only to their particular selector. VCLEvent class contains attributes like : listener - is function execute as first, element - representing HTMLElement on the page, actions - are events that configured by user, phrases - thanks to which we can use speech to recognize what the library should do on which element, result - string of recognize result from speech API.

Parameters

• recognizeResult String Recognize text result from speech API.
• controlObject ControlObject Object contains configured properties for one HTMLElement.

actions

Get array of event function.

Returns Array<function>

phrases

Get array strings of all phrases set one querySelector

Returns Array<string>

element

Get HTMLElement

Returns Element

result

Get string of recognize result from speech API.

Returns String

listener

Get listener function.

Returns function