File System Key Value

A filesystem based key-value store in Node.JS via HTTP

• Installation
• Features
• Example
• Methods
• Usage
• Notes
• Inspiration
• License

Installation

First, install Node.JS. Then:

[sudo] npm install -g fskv

Features

• HTTP API for storing, modifying, and deleting data
• Flat text files used for storage
• HTTP Caching headers (ETag, Last-Modified, If-None-Match)

Example

Fire up fskv by running:

$ mkdir data
$ fskv
[2013-05-03T01:39:48.227Z] server started on http://localhost:9000

This will start the HTTP server listening on localhost on port 9000, and serve out of ./data.

Basic Example

Now, put some data in the database.

$ curl -X PUT -d 'dave' -i localhost:9000/data/myname
HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
Date: Fri, 03 May 2013 01:48:43 GMT
Connection: keep-alive
Transfer-Encoding: chunked

{"message":"saved","status":"ok"}

The server will respond with a 200 if everything was successful, and with a JSON encoded message.

Now, retrieve the data.

$ curl -i localhost:9000/data/myname
HTTP/1.1 200 OK
Last-Modified: Fri, 03 May 2013 01:48:43 GMT
Content-Length: 4
Content-Type: application/octet-stream
ETag: "4-1367545723000"
Date: Fri, 03 May 2013 01:49:51 GMT
Connection: keep-alive

dave

ETag and Last-Modified supported for both HEAD and GET requests. The body of the response is the value supplied in the PUT request above.

Delete the data.

$ curl -i -X DELETE localhost:9000/data/myname
HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
Date: Fri, 03 May 2013 01:51:06 GMT
Connection: keep-alive
Transfer-Encoding: chunked

{"message":"deleted","status":"ok"}

Like the PUT, a 200 is returned with a JSON encoded message if everything is successful.

Advanced

404 is returned for non-existent keys

$ curl -i localhost:9000/data/myname
HTTP/1.1 404 Not Found
Date: Fri, 03 May 2013 01:52:01 GMT
Connection: keep-alive
Transfer-Encoding: chunked

You can put data more than once... last write wins

$ curl -X PUT -d 'hello' localhost:9000/data/myname
{"message":"saved","status":"ok"}
$ curl -X PUT -d 'goodbye' localhost:9000/data/myname
{"message":"saved","status":"ok"}
$ curl localhost:9000/data/myname
goodbye

Use ?exclusive with PUTs to error if the key exists. Use this to avoid race conditions.

$ curl -X PUT -d 'first' localhost:9000/data/myname?exclusive
{"message":"saved","status":"ok"}
$ curl -X PUT -d 'second' localhost:9000/data/myname?exclusive
{"error":"EEXIST, open 'myname'","code":"EEXIST"}
$ curl localhost:9000/data/myname
first

Stats and Health

You can hit /ping or /stats to see process health.

$ curl localhost:9000/ping
pong
$ curl localhost:9000/stats | json
{
  "started": 1367545690361,
  "nodeversion": "v0.8.8",
  "fskvversion": "0.0.0",
  "pid": 2382,
  "dir": "/Users/dave/dev/fskv/data",
  "mem": {
    "rss": 16261120,
    "heapTotal": 9214976,
    "heapUsed": 4967640
  },
  "arch": "x64",
  "platform": "darwin"
}

Methods

GET /data/:key

Retrieve a key, supports if-none-match with the ETag given.

HEAD /data/:key

Same as GET without the data.

PUT /data/:key

Put data given into the key.

Options

• ?exclusive: Error if the key already exists, allows for an atomic PUT. If the key exists, the PUT will fail with EEXISTS and return a 409.

DELETE /data/:key

Delete the key given.

Usage

Usage: fskv [-b] [-d dir] [-h] [-H host] [-l] [-p port] [-u] [-v]

A filesystem based key-value store in Node.JS via HTTP

Options
  -b, --buffer       buffer logging, defaults to false
  -d, --dir <dir>    the database directory, defaults to ./data
  -h, --help         print this message and exit
  -H, --host <host>  the address to bind to, defaults to localhost
  -n, --no-log       disable logging, logging is enabled by default
  -p, --port <port>  the port to bind to, defaults to 9000
  -u, --updates      check npm for available updates
  -v, --version      print the version number and exit

Notes

• This program does no in-memory caching or expiring of data, it's built to run on the ZFS filesystem with the ARC for caching.
• I don't know if I would use this in production.
• TODO: The Content-Type header is not stored, and is determined by key name extension

Inspiration

My tweet about the filesystem being the best nosql database.

greyhound for using bash+netcat for a filesystem key-value store

License

MIT