simplebuf.js

Library for parsing and serializing binary data

How to use (basic example)

var assert = require('assert')
    , sb = require('simplebuf')
    , t = sb.type;

var layout = [
    sb.field("name", t.string_prefixed32),
    sb.field("address", t.string_fixed(10)),
    sb.field("age", t.uint32)
];

var buffer = new Buffer(1024);

var original = {
    name: "James Joyce",
    address: "Dublin",
    age: 133
};

var written = sb.write(buffer, 0, original, layout, {LE:true});
var res = {};
var consumed = sb.read(res, buffer, 0, layout, {LE:true});

assert.equal( written, consumed );
assert.deepEqual(res, original);

Simplebuf implements the read and write functions for parsing and serializing data respectively. The write function takes a buffer, an initial offset into it, an object to serialize, the layout of that object and options. The read function takes an (empty) object to read into, a buffer, an initial offset, the layout and options.

The options currently only allows to define endianess (see {LE:true} for little-endian). Big-endinan is the default options.

The functions return the amount of bytes written and read respectively. The layout must be defined as an array of fields. Each field has a name and a type.

Simplebuf implements the following types:

Integers

int8 and uint8 - signed and unsigned 8-bit integer int16 and uint16 - signed and unsigned 16-bit integer int32 and uint32 - signed and unsigned 32-bit integer

The following notations are equivalent:

Example:

var layout = [
    sb.field("some field", sb.type.uint16),
    sb.field("some other field", sb.type.uint(16))
];

Strings

Fixed-length string

string_fixed(length) - a fixed length string. If the string is longer it will be truncated, if it is shorter it will be padded with zeros.

Example:

var layout = [
    sb.field("address", sb.type.string_fixed(10))
];

Prefixed string

string_prefixed(bits), string_prefixed8, string_prefixed16, string_prefixed32 - a string prefixed with a uint that contains its length. The bits parameter can be 8, 16 or 32.

Example:

var layout = [
    sb.field("name", sb.type.string_prefixed(32)),
    sb.field("address", sb.type.string_prefixed8)
];
sb.write(buffer, 0, {name: "Bob Dylan", address: "Here comes the address"}, layout);

Dynamic string

string_dynamic(length_field_name) - a string which length is defined in a separate field. Before writing the length field must be manually initialized to the string's length. It's also important that the length field is defined before the string field, otherwise reading will fail with an exception.

Example:

var layout = [
    sb.field("len", sb.type.uint(32)),
    sb.field("padding", sb.type.uint(32)),
    sb.field("id", sb.type.string_dynamic("len"))
];
var original = {len: 4, "padding": 999, "id": "1234"};
sb.write(buffer, 0, original, layout);

Arrays

Prefixed array

array_prefixed(element_type, bits), array_prefixed8(element_type), array_prefixed16(element_type), array_prefixed32(element_type) - an array prefixed with uint length. The bits parameter define the length field and it can be 8, 16 or 32. The element_type is a type or layout of elements contained in the array.

Example:

var elementLayout = [
    sb.field("x", sb.type.uint32),
    sb.field("y", sb.type.uint32)
];

var layout = [
    sb.field("array_of_ints", sb.type.array_prefixed(sb.type.uint32, 32)),
    sb.field("array_of_strings", sb.type.array_prefixed8(sb.type.string_prefixed16)),
    sb.field("array_of_points", sb.type.array_prefixed32(elementLayout))
];

var buffer = new Buffer(1024);
var original = {
    array_of_ints : [1,5,900],
    array_of_strings : ["alpha", "beta", "gamma"],
    array_of_points : [
        {x:1, y:100}, {x:20, y:1000}
    ]
};
sb.write(buffer, 0, original, layout);

Frames

Frames allow nesting objects. One example of using frames is the array_of_points above.

Another example:

var address = [
    sb.field("street", sb.type.string_prefixed16),
    sb.field("city", sb.type.string_prefixed16),
    sb.field("index", sb.type.uint32)
];

var person = [
    sb.field("name", sb.type.string_prefixed16),
    sb.field("address", address)
];

var original = {
        name: "Neo",
        address: {
            street: "Wallstr.",
            city: "NNY",
            index: "1800"
        }
};
sb.write(buffer, 0, original, person);

Partials

Partials allow reusing layouts without creating nested objects. Because all fields of partials will be delivered to the same object, the fields can refer to each other. See below how the text field refers to the size field defined in a different partial. Partials can also be nested, unlike frames this will not create the nested objects when reading data.

Example:

var generic_header = [
    sb.field("senderAddress", sb.type.string_fixed(16)),
    sb.field("senderPort", sb.type.uint16)
];
var message_header = [
    sb.field("senderId", sb.type.uint32),
    sb.field("size", sb.type.uint32),                 // the size of text
    sb.field("reserved", sb.type.uint32)
];
var message_body = [
    sb.field("text", sb.type.string_dynamic("size"))  // the length of text is defined in another partial
];

var udp_message_layout = [
    sb.partial(generic_header),
    sb.partial(message_header),
    sb.partial(message_body)
];

var msg = {
    senderAddress: "192.168.1.88",
    senderPort: "8080",
    senderId: 37,
    size: 16,                                         // the length of text
    reserved: 0,
    text: "Command accepted"
};

sb.write(buffer, 0, msg, udp_message_layout);

Dynamic layouts

Dynamic layouts allow to select the exact layout of a partial or a frame based on arbitrary field value. This can be very useful when defining message catalogs where each message is identified by a message_type field.

dynamic_layout(field_name, layout_dispatch_map). The field_name parameter defines the field that will identify the message. The layout_dispatch_map is a map where keys are different values of the field and the values are the corresponding layouts.

Example: (The message_type field can have an arbitrary name, of course)

var generic_header = [
    sb.field("senderAddress", sb.type.string_fixed(16)),
    sb.field("senderPort", sb.type.uint16)
];
var message_header = [
    sb.field("senderId", sb.type.uint32),
    sb.field("message_type", sb.type.uint32)                 // defines the message to parse
];

var message_join_network = [
    sb.field("group", sb.type.string_prefixed32),
    sb.field("port", sb.type.uint16)
];

var message_keep_alive = [
    sb.field("query", sb.type.string_prefixed32)
];

var messageCatalog = {  
  1002 :  message_join_network,
  1003 :  message_keep_alive
};

var udp_message = [
    sb.partial(generic_header),
    sb.partial(message_header),
    sb.partial(t.dynamic_layout("message_type", messageCatalog))
];

var buffer = new Buffer(1024);
var original = {
    senderAddress: "192.168.1.88",
    senderPort: "8080",
    senderId: 37,
    message_type: 1003,
    query: "PING"
};
sb.write(buffer, 0, original, udp_message);

TODO: array_fixed, array_dynamic, value_convertor, buffer_type, uint64, enums, flags...