FlatBuffers JSON Reflection Addon

A Node.js native addon (N-API) that enables runtime conversion between FlatBuffer binaries and JSON using .bfbs schemas.

Overview

This addon wraps the official FlatBuffers C++ reflection API, providing Node.js developers with the ability to:

• Convert FlatBuffer binary data to JSON at runtime
• Convert JSON back to FlatBuffer binary format
• Use .bfbs (binary schema) files without requiring generated code

Unlike the official JavaScript FlatBuffers library which requires generated code from .fbs files, this addon supports full runtime reflection using the C++ library's capabilities.

Features

• Runtime Schema Loading: Load .bfbs schemas dynamically without code generation
• Bidirectional Conversion: Convert between FlatBuffer binary ⇄ JSON
• Multi-Architecture Support: Prebuilt binaries for:
  • macOS x64 (Intel)
  • macOS arm64 (Apple Silicon)
  • Linux x64
  • Linux arm64
• Zero Configuration: Works out of the box with npm install
• N-API Stability: Binary compatibility across Node.js versions
• Type Safety: Full exception handling with descriptive error messages

Installation

npm install @davidepalombo/flatbuffers-json-addon

The package automatically selects the correct prebuilt binary for your platform. If no prebuilt is available, it will compile from source (requires build tools).

Build Requirements (for source compilation)

• Node.js ≥ 18
• Python 3 (for node-gyp)
• C++17 compiler (gcc, clang, or MSVC)
• Git (for submodules)

Development Requirements (for testing)

To run tests, you need the FlatBuffers compiler (flatc) installed:

Ubuntu/Debian:

sudo apt-get install flatbuffers-compiler

macOS:

brew install flatbuffers

Or build from source: See FlatBuffers Building Documentation

Quick Start

Class-Based API (Recommended)

const { FlatBuffersSchema } = require('@davidepalombo/flatbuffers-json-addon');
const fs = require('fs');

// Load your .bfbs schema file
const schemaBuffer = fs.readFileSync('schema.bfbs');

// Create a schema instance (reusable!)
const schema = new FlatBuffersSchema(schemaBuffer);

// Convert JavaScript object to FlatBuffer binary
const userData = { name: "Alice", age: 25, email: "[email protected]" };
const binary = schema.fromJson(userData);

// Convert back to JavaScript object
const restored = schema.toObject(binary);
console.log(restored); // { name: "Alice", age: 25, email: "[email protected]" }

// Or work with Uint8Array (builder.asUint8Array() compatible)
const uint8Array = schema.fromJsonAsUint8Array(userData);
const json = schema.toJson(uint8Array);

Low-Level API

const { toJson, fromJson } = require('@davidepalombo/flatbuffers-json-addon');
const fs = require('fs');

// Load your .bfbs schema file
const bfbs = fs.readFileSync('schema.bfbs');

// Convert JSON to binary
const jsonString = JSON.stringify({ name: "Bob", age: 30, email: "[email protected]" });
const binary = fromJson(bfbs, jsonString);

// Convert binary back to JSON
const json = toJson(bfbs, binary);
console.log(JSON.parse(json));

API Reference

This package provides two APIs: a class-based API (recommended for most use cases) and a low-level API (for maximum control).

Class-Based API

FlatBuffersSchema

A class that caches the schema for efficient multiple conversions.

Constructor:

new FlatBuffersSchema(bfbsBuffer)

Parameters:

• bfbsBuffer (Buffer | Uint8Array): The .bfbs schema buffer

Methods:

schema.fromJson(json)

Convert JSON to FlatBuffer binary.

Parameters:

• json (string | object): JSON string or JavaScript object

Returns: Buffer

Example:

const binary = schema.fromJson({ name: "Alice", age: 25 });
// or
const binary = schema.fromJson('{"name":"Alice","age":25}');

schema.fromJsonAsUint8Array(json)

Convert JSON to FlatBuffer binary as Uint8Array (compatible with builder.asUint8Array()).

Parameters:

• json (string | object): JSON string or JavaScript object

Returns: Uint8Array

Example:

const uint8Array = schema.fromJsonAsUint8Array({ name: "Alice", age: 25 });
// Use with FlatBuffers: new flatbuffers.ByteBuffer(uint8Array)

schema.fromObject(obj)

Alias for fromJsonAsUint8Array that accepts objects.

Parameters:

• obj (object): JavaScript object

Returns: Uint8Array

schema.toJson(binaryData)

Convert FlatBuffer binary to JSON string.

Parameters:

• binaryData (Buffer | Uint8Array): FlatBuffer binary data

Returns: string - JSON string

Example:

const json = schema.toJson(binary);
const obj = JSON.parse(json);

schema.toObject(binaryData)

Convert FlatBuffer binary to JavaScript object.

Parameters:

• binaryData (Buffer | Uint8Array): FlatBuffer binary data

Returns: object - Parsed JavaScript object

Example:

const obj = schema.toObject(binary);
console.log(obj.name); // Direct access

loadSchema(bfbsBuffer)

Convenience function to create a FlatBuffersSchema instance.

Parameters:

• bfbsBuffer (Buffer | Uint8Array): The .bfbs schema buffer

Returns: FlatBuffersSchema

Example:

const { loadSchema } = require('@davidepalombo/flatbuffers-json-addon');
const schema = loadSchema(fs.readFileSync('schema.bfbs'));

Low-Level API

toJson(bfbs, binaryData)

Converts a FlatBuffer binary to JSON using a .bfbs schema.

Parameters:

• bfbs (Buffer): Node.js Buffer containing the .bfbs binary schema
• binaryData (Buffer): Node.js Buffer containing FlatBuffer data

Returns:

• string: UTF-8 JSON string describing the FlatBuffer object

Throws:

• Error: If .bfbs parsing fails or binary conversion fails

Example:

const json = fbjson.toJson(schemaBuffer, dataBuffer);
const obj = JSON.parse(json);

fromJson(bfbs, jsonString)

Converts JSON back into FlatBuffer binary data.

Parameters:

• bfbs (Buffer): Node.js Buffer containing the .bfbs schema
• jsonString (string): Valid JSON string conforming to the schema

Returns:

• Buffer: Node.js Buffer containing serialized FlatBuffer binary data

Throws:

• Error: If .bfbs parsing fails or JSON is invalid/doesn't match schema

Example:

const jsonData = JSON.stringify({ name: "Alice", age: 25 });
const binary = fromJson(schemaBuffer, jsonData);

Examples

The examples/ directory contains comprehensive examples:

• basic-usage.js - Simple class-based API usage
• builder-integration.js - Working with builder.asUint8Array()
• low-level-api.js - Direct function calls and performance comparison
• error-handling.js - Validation and error handling patterns
• web-server.js - Real-world Express.js API server

See the examples README for detailed documentation.

Working with FlatBuffers Builder

If you're using the official FlatBuffers JavaScript library with generated code and want to convert builder.asUint8Array() output to JSON:

const { FlatBuffersSchema } = require('@davidepalombo/flatbuffers-json-addon');

// Load schema once
const schema = new FlatBuffersSchema(fs.readFileSync('schema.bfbs'));

// Convert builder output to JSON
const builder = new flatbuffers.Builder(1024);
// ... build your object ...
const binaryData = builder.asUint8Array();

const json = schema.toJson(binaryData);
const obj = schema.toObject(binaryData);

// Or create Uint8Array from JSON for FlatBuffers
const uint8Array = schema.fromJsonAsUint8Array({ name: "Alice", age: 25 });
const buf = new flatbuffers.ByteBuffer(uint8Array);

Error Handling

The addon provides clear error messages for common issues:

try {
  const json = fbjson.toJson(invalidSchema, data);
} catch (err) {
  // err.message: "Failed to parse bfbs schema"
}

try {
  const binary = fbjson.fromJson(schema, "invalid json");
} catch (err) {
  // err.message: "Failed to parse JSON"
}

Generating .bfbs Files

Use the FlatBuffers compiler (flatc) to generate .bfbs schemas from .fbs files:

# Generate .bfbs schema
flatc --binary --schema -o output/ schema.fbs

# Generate sample binary data from JSON
flatc -b schema.fbs sample.json

Example Workflow

# 1. Create a schema file (example.fbs)
cat > example.fbs << 'EOF'
namespace Example;

table Person {
  name: string;
  age: int;
  email: string;
}

root_type Person;
EOF

# 2. Generate .bfbs schema
flatc --binary --schema example.fbs

# 3. Create sample JSON
cat > sample.json << 'EOF'
{
  "name": "John Doe",
  "age": 30,
  "email": "[email protected]"
}
EOF

# 4. Use the addon
node << 'EOF'
const fbjson = require('@davidepalombo/flatbuffers-json-addon');
const fs = require('fs');

const bfbs = fs.readFileSync('example.bfbs');
const json = fs.readFileSync('sample.json', 'utf8');

// Convert to binary
const binary = fbjson.fromJson(bfbs, json);
fs.writeFileSync('output.bin', binary);

// Convert back to JSON
const restored = fbjson.toJson(bfbs, binary);
console.log('Restored:', JSON.parse(restored));
EOF

Development

Building from Source

# Clone the repository
git clone https://github.com/davidepalombo/flatbuffers-json-reflection.git
cd flatbuffers-json-reflection

# Initialize submodules
git submodule update --init --recursive

# Install dependencies
npm install

# Build the addon
npm run build

# Run tests
npm test

Project Structure

flatbuffers-json-reflection/
├── src/
│   ├── addon.cc              # N-API wrapper
│   ├── flatbuffers_util.cc   # Core conversion logic
│   └── flatbuffers_util.h    # Header file
├── external/
│   └── flatbuffers/          # FlatBuffers C++ library (submodule)
├── test/
│   ├── example.fbs           # Test schema
│   ├── example.json          # Test data
│   └── test.js               # Test suite
├── .github/
│   └── workflows/
│       ├── prebuild.yml      # Multi-arch build workflow
│       └── test.yml          # CI test workflow
├── binding.gyp               # node-gyp build configuration
├── index.js                  # Entry point
└── package.json

CI/CD

The project uses GitHub Actions for:

• Multi-architecture builds: Generates prebuilds for macOS (x64, arm64) and Linux (x64, arm64)
• Automated testing: Runs tests on multiple Node.js versions and platforms
• NPM publishing: Automatically publishes on version tags

Performance Considerations

• The addon uses the FlatBuffers C++ library which is highly optimized
• Binary ⇄ JSON conversion is performed in native code
• Consider caching the .bfbs schema buffer if performing multiple conversions
• Large buffers are handled efficiently through N-API's buffer APIs

Limitations

• Requires .bfbs binary schema files (not .fbs text files)
• JSON output format follows FlatBuffers' text representation
• No built-in schema validation beyond FlatBuffers' own checks

Future Enhancements

• [ ] Optional base64 output for ubyte[] fields
• [ ] Schema caching for improved performance
• [ ] TypeScript definitions (index.d.ts)
• [ ] Universal Binary support for macOS (combined x64/arm64)
• [ ] Schema validation utilities
• [ ] Streaming API for large files

Contributing

Contributions are welcome! Please:

1. Fork the repository
2. Create a feature branch
3. Make your changes with tests
4. Submit a pull request

License

This project is licensed under the MIT License - see the LICENSE file for details.

Third-Party Licenses

This package includes source code from the following projects:

• FlatBuffers - Apache License 2.0 (Copyright 2014 Google Inc.)
• node-addon-api - MIT License (Copyright 2017 Node.js API collaborators)

For complete license information and compliance details, see:

• THIRD_PARTY_LICENSES.md - Full license texts
• NOTICE - Attribution notices
• LICENSE_COMPLIANCE.md - Compliance guide

All third-party licenses are compatible with MIT and allow commercial use.

References

• FlatBuffers Documentation
• FlatBuffers Reflection
• N-API Documentation
• node-addon-api

Support

For issues, questions, or contributions, please visit: https://github.com/davidepalombo/flatbuffers-json-reflection/issues