Node.js driver for tarantool 1.7+

Node tarantool driver for 1.7+ support Node.js v.4+.

Based on go-tarantool and implements Tarantool’s binary protocol, for more information you can read them or basic documentation at Tarantool manual.

Code architecture and some features in version 3 borrowed from the ioredis.

msgpack-lite package used as MsgPack encoder/decoder.

Table of contents

• Installation
• Configuration
• Usage example
• Msgpack implementation
• API reference
• Debugging
• Contributions
• Changelog

Installation

npm install --save tarantool-driver

Configuration

new Tarantool([port], [host], [options]) ⇐ EventEmitter

Creates a Tarantool instance, extends EventEmitter.

Connection related custom events:

• "reconnecting" - emitted when the client try to reconnect, first argument is retry delay in ms.
• "connect" - emitted when the client connected and auth passed (if username and password provided), first argument is an object with host and port of the Taranool server.
• "change_host" - emitted when nonWritableHostPolicy option is set and write error occurs, first argument is the text of error which provoked the host to be changed.

| Param | Type | Default | Description | | --- | --- | --- | --- | | [port] | number | string | Object | 3301 | Port of the Tarantool server, or a URI string (see the examples in tarantool configuration doc), or the options object(see the third argument). | | [host] | string | Object | "localhost" | Host of the Tarantool server, when the first argument is a URL string, this argument is an object represents the options. | | [path] | string | Object | null | Unix socket path of the Tarantool server. | | [options] | Object | | Other options, including all from net.createConnection. | | [options.port] | number | 6379 | Port of the Tarantool server. | | [options.host] | string | "localhost" | Host of the Tarantool server. | | [options.username] | string | null | If set, client will authenticate with the value of this option when connected. | | [options.password] | string | null | If set, client will authenticate with the value of this option when connected. | | [options.timeout] | number | 0 | The milliseconds before a timeout occurs during the initial connection to the Tarantool server. | | [options.tls] | Object | null | If specified, forces to use tls module instead of the default net. In object properties you can specify any TLS-related options, e.g. from the tls.createSecureContext() | | [options.keepAlive] | boolean | true | Enables keep-alive functionality (recommended). | | [options.noDelay] | boolean | true | Disables the use of Nagle's algorithm (recommended). | | [options.lazyConnect] | boolean | false | By default, When a new Tarantool instance is created, it will connect to Tarantool server automatically. If you want to keep disconnected util a command is called, you can pass the lazyConnect option to the constructor. | | [options.nonWritableHostPolicy] | string | null | What to do when Tarantool server rejects write operation, e.g. because of box.cfg.read_only set to true or during snapshot fetching. Possible values are: - null: just rejects Promise with an error - changeHost: disconnect from the current host and connect to the next from reserveHosts. Pending Promise will be rejected. - changeAndRetry: same as changeHost, but after reconnecting tries to run the command again in order to fullfil the Promise | | [options.maxRetriesPerRequest] | number | 5 | Number of attempts to find the alive host if nonWritableHostPolicy is not null. | | [options.enableOfflineQueue] | boolean | true | By default, if there is no active connection to the Tarantool server, commands are added to a queue and are executed once the connection is "ready", meaning the connection to the Tarantool server has been established and auth passed (connect event is also executed at this moment). If this option is false, when execute the command when the connection isn't ready, an error will be returned. | | [options.reserveHosts] | array | [] | Array of strings - reserve hosts. Client will try to connect to hosts from this array after loosing connection with current host and will do it cyclically. See example below.| | [options.beforeReserve] | number | 2 | Number of attempts to reconnect before connect to next host from the reserveHosts | | [options.retryStrategy] | function | | See below |

Reserve hosts example:

let connection = new Tarantool({
    host: 'mail.ru',
    port: 33013,
    username: 'user'
    password: 'secret',
    reserveHosts: [
        'anotheruser:[email protected]:33033',
        '127.0.0.1:3301'
    ],
    beforeReserve: 1
})
// connect to mail.ru:33013 -> dead
//                  ↓
// trying connect to mail.ru:33033 -> dead
//                  ↓
// trying connect to 127.0.0.1:3301 -> dead
//                  ↓
// trying connect to mail.ru:33013 ...etc

Retry strategy

By default, node-tarantool-driver client will try to reconnect when the connection to Tarantool is lost except when the connection is closed manually by tarantool.disconnect().

It's very flexible to control how long to wait to reconnect after disconnection using the retryStrategy option:

var tarantool = new Tarantool({
  // This is the default value of `retryStrategy`
  retryStrategy: function (times) {
    var delay = Math.min(times * 50, 2000);
    return delay;
  }
});

retryStrategy is a function that will be called when the connection is lost. The argument times means this is the nth reconnection being made and the return value represents how long (in ms) to wait to reconnect. When the return value isn't a number, node-tarantool-driver will stop trying to reconnect, and the connection will be lost forever if the user doesn't call tarantool.connect() manually.

This feature is borrowed from the ioredis

Usage example

We use TarantoolConnection instance and connect before other operations. Methods call return promise(https://developer.mozilla.org/ru/docs/Web/JavaScript/Reference/Global_Objects/Promise). Available methods with some testing: select, update, replace, insert, delete, auth, destroy.

var TarantoolConnection = require('tarantool-driver');
var conn = new TarantoolConnection('notguest:[email protected]:3301');

// select arguments space_id, index_id, limit, offset, iterator, key
conn.select(512, 0, 1, 0, 'eq', [50])
    .then(funtion(results){
        doSomeThingWithResults(results);
    });

Msgpack implementation

You can use any implementation that can be duck typing with next interface:

//msgpack implementation example
/*
    @interface
    decode: (Buffer buf)
    encode: (Object obj)
 */
var exampleCustomMsgpack = {
    encode: function(obj){
        return yourmsgpack.encode(obj);
    },
    decode: function(buf){
        return yourmsgpack.decode(obj);
    }
};

By default use msgpack-lite package.

API reference

tarantool.connect() ⇒ Promise

Resolve if connected. Or reject if not.

tarantool._auth(login: String, password: String) ⇒ Promise

An internal method. The connection should be established before invoking.

Auth with using chap-sha1. About authenthication more here: authentication

tarantool.packUuid(uuid: String)

Method for converting UUID values to Tarantool-compatible format.

If passing UUID without converion via this method, server will accept it as simple String.

tarantool.packDecimal(numberToConvert: Number)

Method for converting Numbers (Float or Integer) to Tarantool Decimal type.

If passing number without converion via this method, server will accept it as Integer or Double (for JS Float type).

tarantool.packInteger(numberToConvert: Number)

Method for safely passing numbers up to int64 to bind params

Otherwise msgpack will encode anything bigger than int32 as a double number.

tarantool.select(spaceId: Number or String, indexId: Number or String, limit: Number, offset: Number, iterator: Iterator, key: tuple) ⇒ Promise

Iterators. Available iterators: 'eq', 'req', 'all', 'lt', 'le', 'ge', 'gt', 'bitsAllSet', 'bitsAnySet', 'bitsAllNotSet'.

It's just select. Promise resolve array of tuples.

Some examples:

conn.select(512, 0, 1, 0, 'eq', [50]);
//same as
conn.select('test', 'primary', 1, 0, 'eq', [50]);

You can use space name or index name instead of id, but this way some requests will be made to get and cache metadata. This stored information will be actual for delete, replace, insert, update too.

For tests, we will create a Space named 'users' on the Tarantool server-side, where the 'id' index is of UUID type:

-- example schema of such space
box.schema.space.create("users", {engine = 'memtx'})
box.space.users:format({
    {name = 'id', type = 'uuid', is_nullable = false},
    {name = 'username', type = 'string', is_nullable = false}
})

And then select some tuples on a client side:

conn.select('users', 'id', 1, 0, 'eq', [conn.packUuid('550e8400-e29b-41d4-a716-446655440000')]);

tarantool.selectCb(spaceId: Number or String, indexId: Number or String, limit: Number, offset: Number, iterator: Iterator, key: tuple, callback: function(success), callback: function(error))

Same as tarantool.select but with callbacks.

tarantool.delete(spaceId: Number or String, indexId: Number or String, key: tuple) ⇒ Promise

Promise resolve an array of deleted tuples.

tarantool.update(spaceId: Number or String, indexId: Number or String, key: tuple, ops) ⇒ Promise

Possible operators.

Promise resolve an array of updated tuples.

tarantool.insert(spaceId: Number or String, tuple) ⇒ Promise

More you can read here: Insert

Promise resolve a new tuple.

tarantool.upsert(spaceId: Number or String, ops: array of operations, tuple: tuple) ⇒ Promise

About operation: Upsert

Possible operators.

Promise resolve nothing.

tarantool.replace(spaceId: Number or String, tuple: tuple) ⇒ Promise

More you can read here: Replace

Promise resolve a new or replaced tuple.

tarantool.call(functionName: String, args...) ⇒ Promise

Call a function with arguments.

You can create function on tarantool side:

function myget(id)
    val = box.space.batched:select{id}
    return val[1]
end

And then use something like this:

conn.call('myget', 4)
    .then(function(value){
        console.log(value);
    });

If you have a 2 arguments function just send a second arguments in this way:

conn.call('my2argumentsfunc', 'first', 'second argument')

And etc like this.

Because lua support a multiple return it's always return array or undefined.

tarantool.eval(expression: String) ⇒ Promise

Evaluate and execute the expression in Lua-string. Eval

Promise resolve result:any.

Example:

conn.eval('return box.session.user()')
    .then(function(res){
        console.log('current user is:' res[0])
    })

tarantool.sql(query: String, bindParams: Array) ⇒ Promise

It's accessible only in 2.1 tarantool.

You can use SQL query that is like sqlite dialect to query a tarantool database.

You can insert or select or create database.

More about it here.

Example:

await connection.insert('tags', ['tag_1', 1])
await connection.insert('tags', ['tag_2', 50])
connection.sql('select * from "tags"')
.then((res) => {
  console.log('Successful get tags', res);
})
.catch((error) => {
  console.log(error);
});

P.S. If you using lowercase in your space name you need to use a double quote for their name.

It doesn't work for space without format.

tarantool.pipeline().<...>.exec()

Queue some commands in memory and then send them simultaneously to the server in a single (or several, if request body is too big) network call(s). This way the performance is significantly improved by more than 300% (depending on the number of pipelined commands - the bigger, the better)

Example:

tarantool.pipeline()
.insert('tags', ['tag_1', 1])
.insert('tags', ['tag_2', 50])
.sql('update "tags" set "amount" = 10 where "tag_id" = \'tag_1\'')
.update('tags', 'tag_id', ['tag_2'], [['=', 'amount', 30]])
.sql('select * from "tags"')
.call('truncateTags')
.exec()

tarantool.ping() ⇒ Promise

Promise resolve true.

~~tarantool.destroy(interupt: Boolean) ⇒ Promise~~

Deprecated

tarantool.disconnect()

Disconnect from Tarantool.

This method closes the connection immediately, and may lose some pending replies that haven't written to client.

Debugging

Set environment variable "DEBUG" to "tarantool-driver:*"

Contributions

It's ok you can do whatever you need. I add log options for some technical information it can be help for you. If i don't answer i just miss email :( it's a lot emails from github so please write me to [email protected] directly if i don't answer in one day.

Changelog

3.1.0

• Added 3 new msgpack extensions: UUID, Datetime, Decimal.
• Connection object now accepts all options of net.createConnection(), including Unix socket path.
• New nonWritableHostPolicy and related options, which improves a high availability capabilities without any 3rd parties.
• Ability to disable the offline queue.
• Fixed bug with int32 numbers when it was encoded as floating. Use method packInteger() to solve this.
• selectCb() now also accepts spaceId and indexId as their String names, not only their IDs.
• Some performance improvements by caching internal values.
• TLS (SSL) support.
• New pipeline()+exec() methods kindly borrowed from the ioredis, which lets you to queue some commands in memory and then send them simultaneously to the server in a single (or several, if request body is too big) network call(s). Thanks to the Tarantool, which made this possible. This way the performance is significantly improved by 500-1600% - you can check it yourself by running npm run benchmark-read or npm run benchmark-write. Note that this feature doesn't replaces the Transaction model, which has some level of isolation.
• Changed const declaration to var in order to support old Node.JS versions.

3.0.7

Fix in header decoding to support latest Tarantool versions. Update to tests to support latest Tarantool versions.

3.0.6

Remove let for support old nodejs version

3.0.5

Add support SQL

3.0.4

Fix eval and call

3.0.3

Increase request id limit to SMI Maximum

3.0.2

Fix parser thx @tommiv

3.0.0

New version with reconnect in alpha.

1.0.0

Fix test for call changes and remove unuse upsert parameter (critical change API for upsert)

0.4.1

Add clear schema cache on change schema id

0.4.0

Change msgpack5 to msgpack-lite(thx to @arusakov). Add msgpack as option for connection. Bump msgpack5 for work at new version.

0.3.0

Add upsert operation. Key is now can be just a number.

ToDo

1. Streams
2. Events and subscriptions
3. Graceful shutdown protocol
4. Prepared SQL statements