@nkcmr/protoc-gen-twirp_js
v0.1.4
Published
This is a `protoc` plugin that will enable the usage of protocol buffers (and JSON) for the purpose of creating a fully code-generated transport layer for Cloudflare workers and the web eco-system.
Downloads
32
Readme
protoc-gen-twirp_js
This is a protoc
plugin that will enable the usage of protocol buffers (and JSON) for the purpose of creating a fully code-generated transport layer for Cloudflare workers and the web eco-system.
I originally created this project because I was doing a lot of work with Cloudflare's durable objects which themselves act like tiny microservices. They are only reachable via a special fetch()
call in the workers runtime that allows you to make HTTP requests. What I always ended up implementing on the Durable Object side was an RPC server; and writing HTTP encoding/decoding logic every time was very tedious and banal. Enter Twirp!
Twirp is an RPC framework that is very similar to gRPC, only that it has been dramatically simplified in a way that makes it actually fairly easy to re-implement in other ways.
install
dependencies:
protoc
— This tool is just a plugin into a bigger toolchain that is theprotoc
toolchain. If you don't already have that installed, go ahead and do that: https://grpc.io/docs/protoc-installation/node
— This plugin is written in JavaScript and runs in Node.js (v16 LTS should work fine)npm
— Is most likely already installed if you installednode
already.
once those things are installed, now we can start to use a "plugin." in the protobuf compiler ecosystem, all the code generators are just plugins, some are officially supported by Google and some are just 3rd party (like this one!).
The protoc
command relies on the plugins being findable via your shell's (bash/zsh) path under a very specific name (hence the ugly name of this repo).
Now we can install this plugin:
npm install -g @nkcmr/protoc-gen-twirp_js
You can verify that it is installed by asking your shell where it is installed:
❯ which protoc-gen-twirp_js
/Users/nkcmr/.npm/bin/protoc-gen-twirp_js
Your output should be something like that (if it isn't, maybe give your terminal/shell a reload).
getting started / example
The basic concept of protobuf/Twirp is that as a programmer, you would start out by describing your service and it's methods, as well as what those methods take as input and return as output, kind of like this:
syntax = "proto3"; // required
package strings.reverser; // not required but good to use
// google defines a number of "utility" protobufs for things
// like timestamp and other things.
import "google/protobuf/timestamp.proto";
// StringReverseRequest represents the "input" of a particular
// method on your service (defined below).
message StringReverseRequest {
string user_string = 1;
}
// StringReverseResponse would be the response. Notice the
// convention of `${MethodName}Request` and
// `${MethodName}Response`.
message StringReverseResponse {
string reversed_string = 1;
google.protobuf.Timestamp reversed_at = 2;
}
// Reverser here is the actual "server" being defined in
// by listing all of the methods or endpoints it has.
service Reverser {
rpc StringReverse(StringReverseRequest) returns (StringReverseResponse);
}
Now, we save that file to reverser.proto
and run this command:
protoc --twirp_js_out=. ./reverser.proto
# ▲ ▲ ▲
# ┌───────────┐ │ │ │
# │ plugin ├───────────┘ │ │
# └───────────┘ │ │
# ┌───────────┐ │ │
# │ output │ │ │
# │ directory │────────────┘ │
# └───────────┘ ┌───────────────┐ │
# │ input .proto │ │
# │ files (1+) │──┘
# └───────────────┘
The result should be that we see a number of files produced:
❯ ls
reverser.proto # our original file
strings.reverser.pb.d.ts
strings.reverser.pb.js
strings.reverser.twirp.d.ts
strings.reverser.twirp.js
The files that have .pb.
in them are the protobuf classes for your code. These are the JS version of the message
blocks in the .proto
file, allowing your code to build and pass around instances of those messages.
The files that have .twirp.
in them are for sending and receiving Twirp RPC calls from your code. To implement a server (in this example), we would use the handleReverserServer
function from strings.reverser.twirp.js
which requires an implementation of the service, like so (in a CF worker example):
import { handleReverserServer } from './strings.reverser.twirp';
import { strings, google } from './strings.reverser.pb';
addEventListener('fetch', event => {
event.respondWith(handleRequest(event.request));
});
function handleRequest(request) {
return handleReverserServer(request, {
async StringReverse(request) {
return strings.reverser.StringReverseResponse.create({
reversed_string: request.user_string.reverse(),
reversed_at: google.protobuf.Timestamp.create({
seconds: Math.floor(Date.now()/1000)
})
})
}
})
}
The "client" or "browser" end of this code might look something like this:
import { sendReverserProtobufRequest } from './strings.reverser.twirp';
import { strings } from './strings.reverser.pb';
async function someBusinessLogic() {
const response = await sendReverserProtobufRequest(
'StringReverse',
strings.reverser.StringReverseRequest.create({ user_string: "foobar" }),
{}
);
console.log(response.reversed_string); // => raboof
}
Now that we have our encoding/decoding/transport code automatically generated, this enables us to focus on the actual implementation of our services instead of finicky boilerplate logic.
plugin options
The plugin itself has a few options that can help with what gets generated, to pass in options to a protobuf plugin, it usually looks like this:
protoc --twirp_js_out=foo=bar,beep=boop:. ./my.proto
With the above, we've passed a dictionary of sorts that has:
- key: "foo", value: "bar"
- key: "beep", value: "boop"
The real options are the following:
no_emit_types
— when set to a non-falsy string, will cause the.d.ts
files to not be emitted.module
— can be set to 2 different options:commonjs
— will cause the output to userequire(...)
calls to do imports.es6
— (default) will cause the output to useimport {...} from './...'
syntax to import other code.
gen
is a pipe (|
) delimited string that can can have 3 distinct values in the set:proto
,server
, andclient
. Passing this option will cause the output to only contain what was asked to be generated, so passinggen=proto|client
will cause the.pb
files and the.twirp
files to be generated, but the.twirp
file will only contain "client-side" code.basename
— can be set to ensure that the basename of the output files to be fixed to a particular string. So passingbasename=my_rpc_files
will cause output file names to bemy_rpc_files.pb.js
andmy_rpc_files.pb.d.ts
and so on.
license
MIT License
Copyright (c) 2022 Nick Comer
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.