futoin-executor

v2.3.7

Published

a year ago

FutoIn Service base with strict API definition for Node and Browser

Downloads

207

0High
0Medium
0Low

andvgal

futoin async rpc orb communication distributed browser webpack

Stability: 3 - Stable

About

Documentation --> FutoIn Guide

FutoIn Executor is a peer which processes a request - handles a FutoIn interface method as described in FTN3: FutoIn Interface Definition. It is not necessary a server - e.g. client may handle callback request from server.

Strict FutoIn interface (iface) definition and transport protocol is defined in FTN3 spec mentioned above. As it is based on JSON, both client and server can be implemented in a few minutes almost in any technology. However, Invoker and Executor concept provide significant benefits for efficiency, reliability and error control.

Executor class is the core which encapsulate pure FutoIn Executor logic. It can be extended to platform-specific implementation (e.g. Browser or Node)

The primary communication channel is WebSockets and HTTP as secondary. Large raw data upload and download is also supported through HTTP(S) only.

Executor - neutral Executor concept logic NodeExecutor - available only if used in Node.js

Note: Invoker and Executor are platform/technology-neutral concepts. The implementation is already available in JS and PHP. Hopefully, others are upcoming

Features

Microservice implementation with zero boilerplates
Easy scaling from single monolithic process to cluster per microservice
Universal in-process, WebSockets and HTTP request handling
REST-like HTTP integration (mapping of path components and parameters to message structure)
Raw data for request and response (with HTTP fallback)
Callback requests for bi-directional channels like (WebSockets, in-process)
Advanced fair resource usage / DoS protection limits based on client source address
- message limit control
- max concurrent requests with queuing limits
- max requests per period with queuing limits
- optional limit aggregation per network address prefix
- advanced efficient limit config selection using futoin-ipset
API specification enforcement (defined in human-friendly FTN3 format)
- works even for in-process calls and covers JS type safety issues
- checks parameters with optional default value substitution
- checks result
- checks advanced message lengths as per FTN3 v1.8
- checks exception types to avoid exposure of internal information
- strict semver-like interface versioning - implicit backward compatibility support
- transport security, authentication and authorization are integrated as well
- optional HMAC signing
High performance and low memory usage

FutoIn reference implementation

Reference implementation of:

FTN6: FutoIn Executor Concept
Version: 1.7

FTN3: FutoIn Interface Definition
Version: 1.9

FTN5: FutoIn HTTP integration
Version: 1.3

FTN4: FutoIn Interface - Ping-Pong
Version: 1.0 (service)

Spec: FTN6: Interface Executor Concept v1.x
Spec: FTN3: FutoIn Interface Definition v1.x
Spec: FTN5: FutoIn HTTP integration v1.x
Spec: FTN4: FutoIn Interface - Ping-Pong v1.x

Author: Andrey Galkin

Installation for Node.js

Command line:

$ npm install futoin-executor --save

$ yarn add futoin-executor

Hint: checkout FutoIn CID for all tools setup.

All public classes can be accessed through module:

var Executor = require('futoin-executor').Executor;

or included modular way, e.g.:

var Executor = require('futoin-executor/Executor');

Browser installation

Pre-built ES5 CJS modules are available under es5/. These modules can be used with webpack without transpiler - default "browser" entry point points to ES5 version.

Webpack dists are also available under dist/ folder, but their usage should be limited to sites without build process.

Warning: check AsyncSteps and AsyncEvent polyfill for older browsers.

The following globals are available:

SimpleCCM - global reference to futoin-invoker.SimpleCCM class
AdvancedCCM - global reference to futoin-invoker.AdvancedCCM class
futoin.Invoker - global reference to futoin-invoker module

Examples

The best examples are live projects:

futoin-database - neutral database query & transaction concept
futoin-eventstream - DB transaction focused event system
futoin-xferengine - very complex bank-grade financial transaction engine

Please note that the examples here expect interface definition files listed below. All sources are available under examples/ folder.

Server implementation example


var AdvancedCCM = require( 'futoin-invoker/AdvancedCCM' );
var NodeExecutor = require( 'futoin-executor/NodeExecutor' );
var async_steps = require( 'futoin-asyncsteps' );

// Initialize CCM
var ccm = new AdvancedCCM( { specDirs : __dirname } );

// Initialize Node.js Executor implementation
var executor = new NodeExecutor(
    ccm,
    {
        specDirs : __dirname,
        httpAddr : 'localhost',
        httpPort : 3000,
        httpPath : '/api/'
    }
);
executor.on( 'notExpected', function( err, error_info ){
    console.log( 'Server: ' + err + ': ' + error_info );
} );

// Normally, you would want to define one in separate module
var service_impl = {
    getProgress : function( as, reqinfo )
    {
        var p = reqinfo.params();
        
        if ( reqinfo.params().resource === 'MyResource' )
        {
            as.success( { progress : 75 } )
        }
        else
        {
            as.error( 'InvalidResource' );
        }
    },

    subscribeProgress : function( as, reqinfo )
    {
        var p = reqinfo.params();
        
        if ( reqinfo.params().resource !== 'MyResource' )
        {
            as.error( 'InvalidResource' );
        }

        var channel = reqinfo.channel();
        channel.register( as, 'org.example.receiver:1.0' );
        
        as.add( function( as )
        {
            var iface = channel.iface( 'org.example.receiver:1.0' );
            reqinfo.result().ok = true;

            // Stupid simple scheduling of events to be sent
            //---
            var add_progress = function( progress, delay )
            {
                setTimeout( function(){
                    async_steps().add( function( as ){
                        iface.progressEvent( as, 'MyResource', progress );
                    } )
                    .execute();
                }, delay );
            };
            
            for ( var i = 0; i <= 10; ++i )
            {
                add_progress( i * 10, i * 100 );
            }
            //---
        } );
    }
};

// Register Service implementation
async_steps()
.add(
    function( as )
    {
        executor.register( as, 'org.example.service:1.0', service_impl );
    },
    function( as, err )
    {
        console.log( err + ': ' + as.state.error_info );
        console.log( as.state.last_exception.stack );
    }
)
.execute();

Polling client implementation example

var async_steps = require( 'futoin-asyncsteps' );
var AdvancedCCM = require( 'futoin-invoker/AdvancedCCM' );

var ccm = new AdvancedCCM( { specDirs : [ __dirname ] } );

async_steps()
// Do it once for program's entire life time
.add(
    function( as )
    {
        // Register interface and endpoint
        ccm.register( as, 'myservice',
                      'org.example.service:1.0',
                      'http://localhost:3000/api/' );
    },
    function( as, err )
    {
        console.log( err + ': ' + as.state.error_info );
        console.log( as.state.last_exception.stack );
    }
)
// Regular service call
.add(
    function( as )
    {
        var myservice = ccm.iface( 'myservice' );
        
        // equal to myservice.call( as, 'getProgress', { progress: 'MyResource' } );
        myservice.getProgress( as, 'MyResource' );
        
        as.add( function( as, res ){
            // Use result
            console.log( 'Progress: ' + res.progress );
        } );
    },
    function( as, err )
    {
        // Handle error
        console.log( err + ': ' + as.state.error_info );
    }
)
.execute();

Event receiving client implementation example

var async_steps = require( 'futoin-asyncsteps' );
var AdvancedCCM = require( 'futoin-invoker/AdvancedCCM' );
var Executor = require( 'futoin-executor/Executor' );

var opts = { specDirs : __dirname };
var ccm = new AdvancedCCM( opts );
var client_executor = new Executor( ccm, opts );

async_steps()
// Do it once for program's entire life time
.add(
    function( as )
    {
        // Register interface and endpoint
        ccm.register(
                as,
                'myservice',
                'org.example.service:1.0',
                'ws://localhost:3000/api/',
                null,
                {
                    executor : client_executor
                }
        );

        // Register client-side Executor Service
        client_executor.register(
                as,
                'org.example.receiver:1.0',
                {
                    progressEvent: function( as, reqinfo )
                    {
                        var progress = reqinfo.params().progress;
                        console.log( 'Progress: ' + progress );

                        if ( progress >= 100 )
                        {
                            ccm.close();
                        }
                    }
                }
        );
        
        // Subscribe for events
        as.add( function( as )
        {
            // Note: it is a sub-step as we need to wait for 
            // registration to complete
            var myservice = ccm.iface( 'myservice' );
            
            // equal to myservice.call( as, 'subscribeProgress',
            // { progress: 'MyResource' } );
            myservice.subscribeProgress( as, 'MyResource' );
        } );
    },
    function( as, err )
    {
        console.log( err + ': ' + as.state.error_info );
        console.log( as.state.last_exception.stack );
    }
)
.execute();

Example output of execution

Starting Example Server
Starting Example Client
Progress: 75
Starting Example Client with Callback
Progress: 0
Progress: 10
Progress: 20
Progress: 30
Progress: 40
Progress: 50
Progress: 60
Progress: 70
Progress: 80
Progress: 90
Progress: 100

FutoIn iface definitions

Please see FTN3: FutoIn Interface Definition v1.x for all advanced features.

org.example.types-1.0-iface.json

This one is only used to share Type definitions. It does not participate in inheritance

{
    "iface" : "org.example.types",
    "version" : "1.0",
    "ftn3rev" : "1.1",
    "types" : {
        "Percent" : {
            "type" : "integer",
            "min" : 0,
            "max" : 100
        }
    },
    "desc" : "Shared interface to define common types"
}

org.example.service-1.0-iface.json

Actual Server-side Service definition

{
    "iface" : "org.example.service",
    "version" : "1.0",
    "ftn3rev" : "1.1",
    "imports" : [
        "org.example.types:1.0"
    ],
    "funcs" : {
        "getProgress" : {
            "params" : {
                "resource" : {
                    "type" : "string"
                }
            },
            "result" : {
                "progress" : {
                    "type" : "Percent"
                }
            },
            "throws" : [
                "InvalidResource"
            ]
        },
        "subscribeProgress" : {
            "params" : {
                "resource" : {
                    "type" : "string"
                }
            },
            "result" : {
                "ok" : {
                    "type" : "boolean"
                }
            },
            "throws" : [
                "InvalidResource"
            ]
        }
    },
    "requires" : [
        "AllowAnonymous"
    ],
    "desc" : "Service-side Service"
}

org.example.receiver-1.0-iface.json

Client-side Service for bi-directional transport channels, like WebSockets.

{
    "iface" : "org.example.receiver",
    "version" : "1.0",
    "ftn3rev" : "1.1",
    "imports" : [
        "org.example.types:1.0"
    ],
    "funcs" : {
        "progressEvent" : {
            "params" : {
                "resource" : {
                    "type" : "string"
                },
                "progress" : {
                    "type" : "Percent"
                }
            },
            "desc" : "Progress receiving event callback"
        }
    },
    "requires" : [
        "AllowAnonymous"
    ],
    "desc" : "Client-side Service"
}

API documentation

The concept is described in FutoIn specification: FTN6: Interface Executor Concept v1.x

Modules

Classes

Members

Constants

futoin-executor

BasicAuthFace

BasicAuth is not official spec - it is a temporary solution until FTN8 Security Concept is finalized

Kind: global class

BasicAuthFace.register(as, ccm, endpoint, [credentials], [options])

BasicAuth interface registration helper

Kind: static method of BasicAuthFace

| Param | Type | Default | Description | | --- | --- | --- | --- | | as | AsyncSteps | | step interface | | ccm | AdvancedCCM | | CCM instance | | endpoint | string | | endpoint URL | | [credentials] | * | | see CCM register() | | [options] | object | {} | registration options | | [options.version] | string | "1.0" | iface version |

BasicAuthService

BasicService is not official spec - it is a temporary solution until FTN8 Security Concept is finalized

Kind: global class

BasicAuthService
- instance
- static
  - .register(as, executor) ⇒ BasicAuthService

basicAuthService.addUser(user, secret, details, [system_user])

Kind: instance method of BasicAuthService

| Param | Type | Description | | --- | --- | --- | | user | string | user name | | secret | string | user secret (either password or raw key for HMAC) | | details | object | user details the way as defined in FTN8 | | [system_user] | boolean | is system user |

basicAuthService._getUser(as, user)

Get by name. Override, if needed.

Kind: instance method of BasicAuthService
Note: as result: {object} user object or null (through as)

| Param | Type | Description | | --- | --- | --- | | as | AsyncSteps | steps interface | | user | string | user name |

basicAuthService._getUserByID(as, local_id)

Get by ID. Override, if needed.

Kind: instance method of BasicAuthService
Note: as result: {object} user object or null (through as)

| Param | Type | Description | | --- | --- | --- | | as | AsyncSteps | steps interfaces | | local_id | number | local ID |

basicAuthService.addUser(user, secret, details, [system_user])

Kind: instance method of BasicAuthService

| Param | Type | Default | Description | | --- | --- | --- | --- | | user | string | | user name | | secret | string | | user secret (either password or raw key for HMAC) | | details | object | | user details the way as defined in FTN8 | | [system_user] | boolean | false | is system user |

BasicAuthService.register(as, executor) ⇒ BasicAuthService

BasicAuthService registration helper

Kind: static method of BasicAuthService
Returns: BasicAuthService - reference to implementation instance (to register users)

| Param | Type | Description | | --- | --- | --- | | as | AsyncSteps | steps interface | | executor | Executor | executor instance |

BrowserExecutorOptions ⇐ ExecutorOptions

Kind: global class
Extends: ExecutorOptions

BrowserExecutorOptions ⇐ ExecutorOptions

new BrowserExecutorOptions()

Pseudo-class for BrowserExecutor options documentation

BrowserExecutorOptions.clientTimeoutMS

Client timeout MS

Kind: static property of BrowserExecutorOptions
Default: 600

BrowserExecutorOptions.allowedOrigins

List of allowed page origins for incoming connections. It is MANDATORY for security reasons.

Example:

'http://localhost:8000'
'http://example.com'

Kind: static property of BrowserExecutorOptions
Default: []

BrowserExecutor

Browser Executor with HTML5 Web Messaging as incoming transport.

It allows communication across open pages (frames/tabs/windows) inside client browser.

Kind: global class

new BrowserExecutor(ccm, opts)

| Param | Type | Description | | --- | --- | --- | | ccm | AdvancedCCM | CCM ref | | opts | BrowserExecutorOptions | executor options |

ChannelContext

Channel Context normally accessible through RequestInfo object.

Kind: global class

ChannelContext
- new ChannelContext(executor)
- .type() ⇒ string
- .isStateful() ⇒ Boolean
- .onInvokerAbort(callable, [user_data])
- .register(as, ifacever, options)
- .iface(ifacever) ⇒ NativeIface

new ChannelContext(executor)

| Param | Type | Description | | --- | --- | --- | | executor | Executor | reference to associated executor |

channelContext.type() ⇒ string

Get type of channel

Standard values: HTTP, WS, BROWSER, TCP, UDP, UNIX

Kind: instance abstract method of ChannelContext
Returns: string - arbitrary string, see FTN6

channelContext.isStateful() ⇒ Boolean

Check if transport is stateful (e.g. WebSockets)

Kind: instance method of ChannelContext
Returns: Boolean - true, if context object is persistent across requests in the same session

channelContext.onInvokerAbort(callable, [user_data])

Set invoker abort handler.

Kind: instance method of ChannelContext
Note: It should be possible to call multiple times setting multiple callbacks.
Note: The callback is set for lifetime of the channel - do not use it on every request!

| Param | Type | Description | | --- | --- | --- | | callable | function | callback | | [user_data] | any | optional parameter to pass to callable |

channelContext.register(as, ifacever, options)

Kind: instance method of ChannelContext
See: AdvancedCCM.register

| Param | Type | Description | | --- | --- | --- | | as | AsyncSteps | steps interface | | ifacever | string | standard iface:version notation | | options | object | standard Invoker options |

channelContext.iface(ifacever) ⇒ NativeIface

Get previously registered interface on bi-directional channel.

NOTE: unlike CCM, there is no point for local alias name as Invoker can have only a single ClientExecutor which can have only a single instance implementing specified iface:version.

Kind: instance method of ChannelContext
Returns: NativeIface - - native interface
See: AdvancedCCM.iface

| Param | Type | Description | | --- | --- | --- | | ifacever | string | standard iface:version notation |

DerivedKey

Derived Key interface for planned FTN8 Master key management.

A dummy so far.

Kind: global class

DerivedKey
- new DerivedKey(ccm, base_id, sequence_id)
- .baseID() ⇒ integer
- .sequenceID() ⇒ integer
- .encrypt(as, data) ⇒ Buffer
- .decrypt(as, data) ⇒ Buffer

new DerivedKey(ccm, base_id, sequence_id)

| Param | Type | Description | | --- | --- | --- | | ccm | AdvancedCCM | reference to CCM | | base_id | integer | master key ID | | sequence_id | integer | sequence number of the derived key |

derivedKey.baseID() ⇒ integer

Get master key ID

Kind: instance method of DerivedKey
Returns: integer - Base ID

derivedKey.sequenceID() ⇒ integer

Get derived key sequence ID

Kind: instance method of DerivedKey
Returns: integer - Sequence ID

derivedKey.encrypt(as, data) ⇒ Buffer

Encrypt data with current derived key. Useful for very senstive information.

Kind: instance method of DerivedKey
Returns: Buffer - encrypted data

| Param | Type | Description | | --- | --- | --- | | as | AsyncSteps | steps interface | | data | string | Buffer | to encrypt |

derivedKey.decrypt(as, data) ⇒ Buffer

Decrypt data using current derived key

Kind: instance method of DerivedKey
Returns: Buffer - decrypted data

| Param | Type | Description | | --- | --- | --- | | as | AsyncSteps | steps interface | | data | Buffer | to decrypt |

ExecutorOptions

Kind: global class

ExecutorOptions

new ExecutorOptions()

Pseudo-class for Executor options documentation

ExecutorOptions.specDirs

Search dirs for spec definition or spec instance directly. It can be single value or array of values. Each value is either path/URL (string) or iface spec instance (object).

Kind: static property of ExecutorOptions
Default: []

ExecutorOptions.prodMode

Production mode - disables some checks without compomising security

Kind: static property of ExecutorOptions
Default: false

ExecutorOptions.reqTimeout

Default request processing timeout

Kind: static property of ExecutorOptions
Default: 5000

ExecutorOptions.heavyReqTimeout

Default request processing timeout for functions marked "heavy". See FTN3

Kind: static property of ExecutorOptions
Default: 60000

ExecutorOptions.securityProvider

FTN8 security interface

Kind: static property of ExecutorOptions

ExecutorOptions.messageSniffer()

Message sniffer callback( iface_info, msg, is_incomming ). Useful for audit logging.

Kind: static method of ExecutorOptions
Default: dummy

Executor

An abstract core implementing pure FTN6 Executor logic.

Kind: global class

Executor

new Executor(ccm, opts)

| Param | Type | Description | | --- | --- | --- | | ccm | AdvancedCCM | instance of AdvancedCCM | | opts | objects | see ExecutorOptions |

executor.ccm() ⇒ AdvancedCCM

Get reference to associated AdvancedCCM instance

Kind: instance method of Executor
Returns: AdvancedCCM - CCM ref

executor.register(as, ifacever, impl, specdirs)

Kind: instance method of Executor

| Param | Type | Description | | --- | --- | --- | | as | AsyncSteps | steps interface | | ifacever | string | standard iface:version notation of interface to be implemented. | | impl | object | function | either iface implementation or func( impl, executor ) | | specdirs | object | array | NOT STANDARD. Useful for direct passing of hardcoded spec definition. |

executor.onEndpointRequest(info, ftnreq, send_executor_rsp)

Entry point for Server-originated requests when acting as ClientExecutor

Kind: instance method of Executor
Emits: notExpected, request, response

| Param | Type | Description | | --- | --- | --- | | info | object | raw Invoker interface info | | ftnreq | object | FutoIn request object | | send_executor_rsp | function | callback( ftnrsp ) |

executor.onInternalRequest(as, info, ftnreq, [upload_data], [download_stream])

Entry point for in-program originated requests. Process with maximum efficiency (not yet ;)

Kind: instance method of Executor
Emits: notExpected, request, response
Note: AS result: ftnrsp, content-type

| Param | Type | Description | | --- | --- | --- | | as | AsyncSteps | steps interface | | info | object | raw Invoker interface info | | ftnreq | object | FutoIn request object | | [upload_data] | object | upload stream, if any | | [download_stream] | object | download stream, if any |

executor.process(as)

Standard entry point used by subclasses. Do full cycle of request processing, including all security checks

NOTE: as.state.reqinfo must point to valid instance of RequestInfo

Kind: instance method of Executor
Emits: notExpected, request, response

| Param | Type | Description | | --- | --- | --- | | as | AsyncSteps | steps interface |

executor.checkAccess(as, acd)

Shortcut to check access through #acl interface.

NOTE: as.state.reqinfo must point to valid instance of RequestInfo

Kind: instance method of Executor

| Param | Type | Description | | --- | --- | --- | | as | AsyncSteps | steps interface | | acd | string | access control descriptor |

executor.initFromCache(as)

NOT IMPLEMENTED, DO NOT USE. Just a compliance with the Executor interface from spec.

Kind: instance method of Executor

| Param | Type | Description | | --- | --- | --- | | as | AsyncSteps | steps interface |

executor.cacheInit(as)

NOT IMPLEMENTED, DO NOT USE. Just a compliance with the Executor interface from spec.

Kind: instance method of Executor

| Param | Type | Description | | --- | --- | --- | | as | AsyncSteps | steps interface |

executor.close([close_cb])

Shutdown Executor and stop whole processing

Kind: instance method of Executor
Emits: close

| Param | Type | Default | Description | | --- | --- | --- | --- | | [close_cb] | callable | | callback to execute after Executor shutdown |

executor.packPayload(coder, msg) ⇒ string

Not standard. Pack message object into JSON representation. If safe limit of 64K is exceeded then error is raised.

Kind: instance method of Executor
Returns: string - string representation of the message
Emits: notExpected

| Param | Type | Description | | --- | --- | --- | | coder | MessageCoder | message coder instance | | msg | object | message to encode into JSON |