proto-rpc

Transport-independent protobuf RPC for Node.js and the Web Browser. Binary alternative to JSON-RPC. Uses protobuf-ts for code generation. Pure TypeScript.

Introduction

proto-rpc is intended for situations where one would normally use JSON-RPC, but would like to switch to a binary protocol with autogenerated, typesafe code.

Unlike existing protobuf RPC frameworks such as gRPC, gRPC-web and twirp, proto-rpc is totally transport-independent; it simply produces Uint8Arrays and transporting them to the recipient is up to you! You can use Websockets, peer-to-peer networks, WebRTC, UDP, or TCP for the transport, just to name a few.

Usage

• install the library

npm install @streamr/proto-rpc

• write RPC service description in a .proto file:

syntax = "proto3";

service HelloRpcService {
    rpc sayHello (HelloRequest) returns (HelloResponse);
}

message HelloRequest {
    string myName = 1;
}
  
message HelloResponse {
    string greeting = 1;
}

HelloRpc.proto

• generate client and server classes using protobuf-ts:

mkdir -p ./proto
npx protoc --ts_out $(pwd)/proto --ts_opt server_generic,generate_dependencies --proto_path $(pwd) HelloRpc.proto

• implement the auto-generated IHelloRpcService server interface in TypeScript

class HelloService implements IHelloRpcService {
    async sayHello(request: HelloRequest, _context: ServerCallContext): Promise<HelloResponse> {
        return { greeting: 'Hello ' + request.myName + '!' }
    }
}

• start a RpcCommunicator for the server side, and register the RPC method you just created. Note that a RpcCommunicator can act both as a client and a server at the same time

const communicator1 = new RpcCommunicator()
const helloService = new HelloService()
communicator1.registerRpcMethod(HelloRequest, HelloResponse, 'sayHello', helloService.sayHello)

• start a RPC communicator for the client side, bind it to an instance of the auto-generated HelloRpcServiceClient class, and convert the auto-generated client into a ProtoRpcClient

const communicator2 = new RpcCommunicator()
const helloClient = toProtoRpcClient(new HelloRpcServiceClient(communicator2.getRpcClientTransport()))

• listen to outgoing packets from the RpcCpommunicators on both the client and server sides, and deliver them to the correct recipient. In real life this would happen over a network connection (Websocket, WebRTC, HTTP..) but here we will simulate the connection using method calls.

communicator1.on('outgoingMessage', (msgBody: Uint8Array, _requestId: string, _callContext?: CallContext) => {
    communicator2.handleIncomingMessage(msgBody)
})
communicator2.on('outgoingMessage', (msgBody: Uint8Array, _requestId: string, _callContext?: CallContext) => {
    communicator1.handleIncomingMessage(msgBody)
})

• make the RPC call and print the result

const response = await helloClient.sayHello({ myName: 'Alice' })
console.log(response.greeting)

• finally, discard the RpcCommunicators to clean up pontially pending async calls and other allocated resources

communicator1.stop()
communicator2.stop()

For a complete code example, see examples/hello

Advanced topics

Passing context information (eg. for routing)

You can pass context information through the RpcCommunicator between the clients, RPC methods and the event handlers. This is especially useful in case you wish to use a single RpcCommunicator as a server for multiple clients, and need to figure out where to route the Uint8Arrays output by the RpcCommunicator.

For a complete code example of passing context information, see examples/routed-hello

Notifications

Unlike gRPC, proto-rpc supports JSON-RPC style notifications (RPC functions that return nothing).

• In the .proto service definitions, the notification functions need to have google.protobuf.Empty as their return type.

service WakeUpRpcService {
    rpc wakeUp (WakeUpRequest) returns (google.protobuf.Empty);
}

• The notification function implementations need to return a google.protobuf.Empty object.

async wakeUp(request: WakeUpRequest, _context: ServerCallContext): Promise<Empty> {
    console.log("WakeUp notification of node " + this.nodeId + " called with reason: " + request.reason)
    const ret: Empty = {}
    return ret
}

• In case of a notification call, RpcCommunicator does not fire the RpcCommunicatorEvents.OUTGOING_MESSAGE event for the return value.

For a complete code example of using notifications, see examples/wakeup

Errors

All standard errors in the library can be found the src/errors.ts file.

• RpcTimeout is thrown by default whenever the client or server times out its operations.
• RpcRequest is thrown by the server to be sent as an error response back to the client if the request fails.
• FailedToParse is thrown whenever parsing a protobuf message fails.
• FailedToSerialize is thrown whenever serializing a protobuf message fails.
• UnknownRcMethod is thrown whenever a server receives an RPC call for a method that it has not registered
• NotImplemented is thrown whenever something internal that is not supposed to be used is called. For example ts-protobuf ServerCallContext methods.

When developing you can use the errors as follows:

throw new RpcError.RpcTimeout()
throw new RpcError.RpcTimeout('RPC Request timed out')
throw new RpcError.RpcTimeout('RPC Request timed out', originalError)

Timeouts

Client side timeouts can be set along side requests via the options parameter. By default the client side timeout is 5000 milliseconds.

Example:

const results = helloClient.sayHello({ myName: 'Alice' }, { timeout: 15000 })
await results.response // This will eventually timeout after 15000ms