Inhumate Contract Tool

Code generation tool for data contracts. Think of it as protoc with a few extra goodies. Generates language bindings from an Inhumate contract — protobuf types, channel and constant definitions, and the index/barrel files that make them importable as one package.

Usage

Simplest way of running (on a machine with Node and NPM installed):

npx inhumate-contract --help

For environments where Node isn't conveniently available (e.g. for Windows development), there are binary releases available from Gitlab or Inhumate Downloades.

The contract tool is primarily used to generate code for data contracts, e.g. to generate Python code for the Gensim contract:

inhumate-contract generate inhumate:gensim python out/

A contract source can be a local path, any git URL (https://…, git@…, ssh aliases), or a shortcut — inhumate:<name> / nm8:<name> for the Inhumate contracts group — with an optional @<version> (or @latest). Remote contracts are fetched with git and cached, so private repos work with whatever authentication your git already uses.

JavaScript without a build step

Besides typescript, there are two JavaScript targets:

• js is the typescript output transpiled to plain JavaScript — the same files, imports and API, without the types. targets.typescript settings apply to it too.
• jsbundle is a single script file for a web page with no build step, meant to sit next to the RTI client bundle. It defines one global named after the contract:

inhumate-contract generate inhumate:gensim jsbundle public/

<script src="inhumate-rti-bundle-1.2.3.js"></script>
<script src="inhumate-gensim-1.0.0.js"></script>
<script>
    const rti = new RTI.Client({ application: "My page" })
    rti.subscribe(Gensim.channel.entity, Gensim.proto.Entity, (entity) => console.log(entity))
</script>

The bundle has everything it needs, protobufjs included. It also carries its own copy of any imported types, the RTI ones too: a script tag cannot import a type from another bundle. Messages are plain objects, so the copies work the same as the RTI bundle's. The file name carries the contract version, like the RTI bundle's; targets.jsbundle.global and targets.jsbundle.file change the global's name and the file name.

Both targets need esbuild from a node_modules (installed along with the tool when you use npm), and jsbundle needs protobufjs too.

Zero-config: the manifest

Instead of a command per target, drop an inhumate-contract.yml at your project root listing everything, then run inhumate-contract (no arguments) as part of your build:

generate:
    - source: inhumate:[email protected]
      language: typescript
      output: js/src/generated
    - source: inhumate:[email protected]
      language: csharp
      output: dotnet/src/Generated
      emit: [constants] # optional; default is everything the language supports
options:
    protobufVersion: 23.3 # optional, tool-wide

It is idempotent: the first run generates, and later runs are a no-op — tracked in inhumate-contract.lock — until an output is removed, the manifest changes, or the tool is upgraded. inhumate-contract sync --force regenerates regardless, --refresh re-fetches remote contracts.

A run also removes what it stopped generating — a module for an imported type the contract no longer uses, or the output of a job you deleted — so the generated directory only ever holds what the manifest currently produces. Only files the lock records as generated are touched; anything you wrote by hand stays.

The manifest is looked for from the current directory upward, so the command works from a subdirectory. If there is none, every manifest below the current directory is run instead — so a tree of sibling projects, each with its own manifest and none at the root, generates in one command:

$ cd contract-demo && inhumate-contract
found 8 manifests below the current directory

==> demo1/cpp-app
generated 3 files: generated

==> demo1/node-app
up to date: src/generated
...

Each project keeps its own lock, so they stay independent, and one that fails does not stop the rest — the command reports which failed and exits non-zero. clean works the same way.

The downward search only happens inside a git repository, and never in your home directory: a checked-out tree is something you meant to have, anywhere else is wherever the shell happened to be. It also skips node_modules, build output and dotted directories, and stops at each manifest it finds, so it stays fast even in a large repository.

Imported contracts

A contract can build on another one's message types with import:. By default the imported types its own .proto files actually refer to are generated alongside it, transitively, so one generate produces code that compiles. Only what is referenced comes along: a contract using one message out of Gensim gets that message's file and whatever that file imports, not all eighteen.

--imports (job field imports:) changes how much comes along:

| Mode | What is generated | | ---- | ----------------- | | referenced | the contract's own files, plus the imported files they refer to (default) | | all | every file of every imported contract — one self-contained bundle | | none | the contract's own files only, leaving imports as pure references |

--include-imports and --no-include-imports are kept as the older spellings of all and none.

Imported contracts you already have

Sometimes the imported contract must not be generated: it already ships as a library the project depends on, the way every project using the RTI client already has the RTI contract's types. A second copy of them is a duplicate class in C#, a duplicate symbol in C++, and two mutually incompatible copies of the same messages in TypeScript and Python.

The RTI contract is treated this way by default — no configuration needed. Any other contract in that position says so once, in its own contract.yml:

targets:
    typescript:
        package: inhumate-rti/lib/generated
    python:
        package: inhumate_rti.generated
    csharp:
        package: Inhumate.RTI
    cpp:
        package: rticontract.hpp

package is where this contract's generated code lives for someone who already has it. It changes nothing about generating this contract; it is read when another contract imports it. So generating Gensim needs no configuration at all —

generate:
    - source: inhumate:[email protected]
      language: typescript
      output: src/gensim

— and the generated Injectable.ts imports Parameter from inhumate-rti/lib/generated, while Injectable_pb2.py imports Parameter_pb2 from inhumate_rti.generated, instead of from a sibling module this run does not write. C# and C++ need no module path: for them the value only names the library, and the C++ one is the header the generated header #includes.

A job can override what a contract declares, for a project that vendors it somewhere else, or opt out with false — which generates that contract into this output after all, for the odd application that wants the types without the RTI client:

- source: inhumate:[email protected]
  language: typescript
  output: src/gensim
  provided:
      inhumate.rti: false

A key can be written as the contract id (inhumate.rti) or as the source the contract imports it by (inhumate:[email protected]); a key that matches neither is an error. --provided <contract>=<module> does the same from the command line, and --provided <contract>=false opts out. inhumate-contract validate <contract> reports what each import will do, per language.

Generating an imported contract yourself

Listing an imported contract as a job of its own does not clash with any of this: within one manifest run, a contract generated by one job is not generated again as another job's import. So this produces each type exactly once, whichever order the jobs are in:

generate:
    - source: ./contract # imports inhumate:[email protected]
      language: csharp
      output: Generated
    - source: inhumate:[email protected] # all of Gensim, not just the referenced part
      language: csharp
      output: Generated
      emit: [proto]

That works because both jobs write into the same directory. Generating one contract into two different output directories for the same language is a warning instead: nothing can merge those, and two copies of the same types will not compile together in C# or C++.

Two jobs may not write the same file. TypeScript and Python name everything but the message modules after what is emitted — constants.ts, index.ts, the barrel, __init__.py, channel.py — so two contracts generated into one directory would leave a single index, belonging to whichever ran last, and the app would silently lose its own contract. Rather than produce that, the run is refused:

$ inhumate-contract
error: two jobs write the same files into src/generated:
  ../contract and inhumate:[email protected], both typescript
  both write: constants.ts, index.ts, proto.ts
  typescript names these after what is emitted, not after the contract, so one job overwrites
  the other. Generate each contract into its own output directory.

C# and C++ name their index after the contract (StatusConstants.cs, demo-status.hpp), so they compose in one directory and are unaffected — the check is on the files that actually collide, not on sharing a directory.

C++ and the protobuf runtime

Generated C++ only works against the libprotobuf it was generated for — the headers carry a hard #error version guard — so a C++ project has to build its runtime from the same version its protoc came from. C++ therefore defaults to protobuf 3.11.2, not the 23.3 every other language gets: protobuf 22 and up pull in abseil, and building that runtime next to a system install ends in unresolved absl::log_internal symbols. 3.11.2 is the last release before that, and the one the RTI C++ client pins, so a cpp job that says nothing gets the version that builds. protobufVersion — on the job, in contract.yml, or on the command line — overrides it as usual.

Nothing extra is needed to compile against a provided contract's headers. An imported contract built for a Windows DLL decorates its message classes with an export macro, and protoc has this contract's Status.pb.h include the imported RuntimeState.pb.h directly — so the generated headers carry an #ifndef fallback defining that macro away. A shared-library build that defines it to dllimport itself still wins.

A cpp job can pin the version and ask for the matching C++ source in one place:

- source: .
  language: cpp
  output: cpp/generated
  protobufVersion: 3.11.2 # this target's protoc; the cpp default, so only worth writing to change it
  protobufSource: cpp/protobuf # unpack the matching C++ source here, to build the runtime from

protobufSource is skipped once that directory holds the right version, so it never disturbs an existing protobuf build. inhumate-contract get-protobuf-source -o <dir> does the same thing from a build script, defaulting to the same C++ version — but it takes any pinned version from the contract, not from a manifest job, so pass --protobuf-version to match a job that pins one. get-protobuf defaults to 23.3 like everything else; get-protobuf --language cpp gets the protoc matching that source.

targets.cpp.dllexportDecl passes --cpp_out=dllexport_decl=…, the equivalent of CMake's protobuf_generate_cpp(... EXPORT_MACRO x), for building the generated code into a Windows DLL. The generated header defines the macro to nothing if nothing else has, so it stays includable alone.

The Contract

contract.yml names the contract, the channels it publishes on and any other constants it wants to hand to code:

id: inhumate.test
name: Test

channels:
    hello:
        name: test/hello # the name on the wire
        type: Hello # the protobuf message published on it, optional

constants:
    defaultUrl: ws://127.0.0.1:8000/
    maxGreetings: 10

Every target gets the same constants under the names its own conventions call for:

| Contract | TypeScript | Python | C# | C++ | | ---------------- | ------------------------ | ------------------------- | ---------------------------- | ------------------------- | | channel hello | channel.hello | channel.hello | TestChannel.Hello | HELLO_CHANNEL | | its type | channelType.hello | channel_type.hello | TestChannelType.Hello | — | | that type's name | channelTypeName.hello | channel_type_name.hello | TestChannelTypeName.Hello | HELLO_CHANNEL_TYPE_NAME | | every channel | channels | constants.channels | TestConstants.Channels | — | | version | constants.version | __version__ | TestConstants.Version | TEST_VERSION | | maxGreetings | constants.maxGreetings | constants.max_greetings | TestConstants.MaxGreetings | MAX_GREETINGS |

channelType is the message type itself — the ts-proto codec, the python message class, the C# MessageParser — not its name, which is channelTypeName. So a channel's messages can be encoded or decoded from the channel alone, without a switch mapping channels to types by hand:

rti.publish(Test.channel.hello, Test.channelType.hello, { message: "hi" })

channels lists the channels in declaration order, which is how to enumerate them and what to index the two above by:

const key = Test.channels.find((key) => Test.channel[key] === name)

In C# the join is the channel name, not the key. A key is not something C# can spell — the generated classes hand out the names as consts and nothing hands out the keys — so TestConstants.Channels lists the names, and the channel classes carry a ByChannel dictionary keyed by them, next to the named members that stay for the call sites that know the channel already:

foreach (var channel in TestConstants.Channels)
    Console.WriteLine($"{channel} carries {TestChannelTypeName.ByChannel[channel]}");

var hello = TestChannelType.ByChannel[TestChannel.Hello].ParseFrom(bytes);
var same = TestChannelType.Hello.ParseFrom(bytes);

An untyped channel is in Channels but in neither dictionary, so TryGetValue is the way to ask whether a channel carries a message type at all.

C++ has none of it: its constants are constexpr names rather than anything indexable, and a typed map would mean handing out protobuf descriptors or prototypes. Channel types there are what they always were — the generated message classes in the contract's proto namespace.

and an index that gathers them, so that using a contract is a one-line import:

import Test from "@inhumate/test-contract"
rti.publish(Test.channel.hello, Test.proto.Hello.fromPartial({ message: "hi" }))

import inhumate_test as Test
hello = Test.proto.Hello()
hello.message = "hi"
rti.publish(Test.channel.hello, hello)

using Inhumate.Test;
using Inhumate.Test.Proto;
rti.Publish(TestChannel.Hello, new Hello { Message = "hi" });

#include "inhumate-test.hpp"
auto hello = new inhumate::test::proto::Hello();
hello->set_message("hi");
rti.Publish(inhumate::test::HELLO_CHANNEL, hello);

Constant groups

A nested mapping under constants: is a group — a set of constants that travels together and gets its own object, module, class or name suffix in each language:

constants:
    capability:
        runtimeControl: runtime

| TypeScript | Python | C# | C++ | | --------------------------- | ---------------------------- | ------------------------------- | ---------------------------- | | capability.runtimeControl | capability.runtime_control | TestCapability.RuntimeControl | RUNTIME_CONTROL_CAPABILITY |

channel and channelTypeName are built-in groups, filled from channels:, and channelType and channels are generated from it too — as a map to the message types and a list of keys, which are not scalars and so not groups. All four names are reserved: declaring a constant or group under one is an error rather than something that quietly overrides the generated one.

Development

Building and Running

npm install
npm start -- generate data/test-contract typescript out          # run from source via tsx
npm start -- generate inhumate:[email protected] typescript out       # …or fetch a contract from git
npm start -- generate data/test-contract csharp out --emit proto   # or leave --emit out for everything
npm start -- get-protobuf -c data/test-contract                  # cache protoc, print its path
npm start -- clear-cache                                         # remove everything downloaded
npm test          # jest
npm run typecheck # tsc --noEmit
npm run build     # esbuild bundle into dist/
npm run format    # prettier

Integration Test

Generates every target and then builds and runs a small program against the generated code, from data/sanity/. macOS and Linux.

scripts/integration_test.sh                 # every target, skipping any whose toolchain is missing
scripts/integration_test.sh python cpp      # only these
INTEGRATION_STRICT=1 scripts/integration_test.sh   # a missing toolchain is a failure

Needs dotnet for csharp, python3 for python, and cmake, make, git and a C++ compiler for cpp — which builds its own protobuf (pinned to v3.11.2, as the RTI C++ client does) rather than using whatever is installed. That build is kept in out/integration/cpp/protobuf between runs.