protodef-protobuf

A transpiler and runtime for using Google Protocol Buffers (.proto files) with ProtoDef in Node.js via node-protodef (supporting both interpreter and compiler).

This allows you to read/write Protocol Buffer-encoded messages in your ProtoDef defined protocols without needing external parsing.

Features

• Proto2 & Proto3 Support: Supports both major versions of Protocol Buffers.
• Full Feature Set: Handles nested messages, enums, maps, packed repeated fields, and extensions.
• High Performance: Generates optimized JavaScript functions using protodef's AOT (Ahead-Of-Time) compiler.
• Runtime Flexibility: Supports ProtoDef's interpreter mode on top of compiler for dynamic schemas.
• Flexible Framing: Includes a protobuf_message container for easily length-prefixing your Protobuf messages, making them embeddable in any protocol.
• Import Support: Handles .proto file imports including Google well-known types with automatic error detection.

Installation

npm install protodef-protobuf protodef

Quick Start

const { ProtoDefCompiler } = require('protodef').Compiler
const pp = require('protodef-protobuf')

// 1. Define your .proto schema
const schema = `
  syntax = "proto3";
  package chat;
  message ChatMessage {
    string user_id = 1;
    string content = 2;
  }
`

// 2. Transpile and create protocol
const generatedSchema = pp.transpile([schema])
const protocol = {
  ...generatedSchema,
  packet_hello: ['protobuf_message', {
    lengthType: 'varint',
    type: 'chat_ChatMessage'
  }]
}

// 3. Compile and use
const compiler = new ProtoDefCompiler()
pp.addTypesToCompiler(compiler)
compiler.addTypesToCompile(protocol)
const proto = compiler.compileProtoDefSync()

// 4. Encode/decode messages
const data = { user_id: 'user123', content: 'Hello, world!' }
const encoded = proto.createPacketBuffer('packet_hello', data)
const decoded = proto.parsePacketBuffer('packet_hello', encoded)

How It Works

This library consists of two main components:

1. Transpiler: Converts your .proto schemas into ProtoDef-compatible JSON
2. Runtime Types: Custom ProtoDef types that handle Protobuf wire format encoding/decoding

The library supports both compiler mode (for performance) and interpreter mode (for flexibility). See the API documentation for detailed comparison.

Detailed Usage

1. Transpile Your Schema

Parse your .proto file(s) and transpile them into ProtoDef format:

const { ProtoDefCompiler } = require('protodef').Compiler
const pp = require('protodef-protobuf')

// Single schema
const schema = `
  syntax = "proto3";
  package chat;
  message ChatMessage {
    string user_id = 1;
    string content = 2;
  }
`

// Multiple schemas (useful for extensions)
const baseSchema = `
  syntax = "proto2";
  package game;
  message Player {
    extensions 100 to 199;
    required int32 id = 1;
  }
`
const extensionSchema = `
  syntax = "proto2";  
  package game;
  extend Player {
    optional string name = 100;
  }
`

// Transpile - automatically merges multiple schemas
const generatedSchema = pp.transpile([schema])
// or: pp.transpile([baseSchema, extensionSchema])

2. Define Your Protocol and Compile

Create your protocol definition and compile it:

const protocol = {
  ...generatedSchema, // Include the generated schema
  
  // Wrap Protobuf messages with length framing
  packet_hello: ['protobuf_message', {
    lengthType: 'varint',        // Length prefix type
    type: 'chat_ChatMessage'     // Your Protobuf message type
  }]
}

// Create and configure the compiler
const compiler = new ProtoDefCompiler()
pp.addTypesToCompiler(compiler)    // Add Protobuf runtime types
compiler.addTypesToCompile(protocol)
const proto = compiler.compileProtoDefSync()

// Now you can encode/decode messages!
const data = { user_id: 'user123', content: 'Hello, world!' }
const encoded = proto.createPacketBuffer('packet_hello', data)
const decoded = proto.parsePacketBuffer('packet_hello', encoded)

Examples

Getting Started

• Basic Compiler - Simple Proto3 message with compiler
• Basic Interpreter - Same example using interpreter mode

Core Features

• Extensions - Working with Proto2 extensions
• Multiple Messages - Complete protocol example
• Message Bytes - Handling binary data

Import Handling

• Google Imports - Using Google well-known types (built-in)
• File System Imports - Importing custom .proto files from disk

Advanced

• gRPC Example - Using with gRPC-style schemas

API Reference

See API.md for detailed documentation of all functions and types.

Supported Features

| Feature | Proto2 | Proto3 | Notes | |---------|--------|--------|-------| | Basic Types | ✅ | ✅ | int32, string, bool, etc. | | Messages | ✅ | ✅ | Nested messages supported | | Enums | ✅ | ✅ | Named values | | Repeated Fields | ✅ | ✅ | Including packed encoding | | Maps | ✅ | ✅ | map<key, value> syntax | | Extensions | ✅ | N/A | Not available in Proto3 spec | | Imports | ✅ | ✅ | Supports import statements | | Oneof | ✅ | ✅ | Wire format only* |

*Oneof constraint validation is not enforced - treat as user validation.

Limitations

• oneof Validation: The library correctly parses the wire format for oneof fields. However, it does not enforce the "only one can be set" constraint on the resulting JavaScript object. This is treated as user-level validation.

License

This project is licensed under the MIT License. See the LICENSE file for details.