raft-logger-redis
v0.2.16
Published
Distributed logging service
Downloads
20
Maintainers
Readme
Welcome to logster! {#welcome}
Logster is a distributed logging system. have you ever wanted to collect logs from multiple sources and view them in one place? Well Logster is the answer.
NOTE:
- Logster is still in alpha state
- Logster is not scalable (Will be soon thanks to redis)
Install
To install Logster you can use npm
or git clone
NPM
$ sudo npm install -g raft-logger-redis
GIT
$ git clone https://github.com/MangoRaft/Logger.git
$ cd Logger
$ sudo npm install -g
Architecture
Logsters architecture is similar to logplex from heroku. The difference logster uses UDP to receive the messages. There are 4 parts to logster, the Logger, UDP-Server, Web-server and the viewer.
The logger send message over UDP to the UDP-Server. The UDP-server send the log messages to the Web-server. From the Web-server drains
can receive logs over HTTP.
Usage
You can use logster through the command line or include it in your program.
In your program
To use logster in your program just require
it. Most of the time you will only use the Logger
or the Viewer
var logster = require('raft-logger');
Logger
To use the logger you need to create a logger You need to pass in the web and udp port and host
Options
- web.host Required. Host for the web server.
- web.port Required. Port for the web server.
- udp.host Required. Host for the udp server.
- udp.port Required. Port for the udp server.
var logger = logster.Logger.createLogger({
web : {
host : '127.0.0.1',
port : 5000
},
udp : {
host : '127.0.0.1',
port : 5000
}
});
Once you have a logging instance you can create the per process logger.
Options
- source Used to group logs.
- channel Used to to identify logs to a specific task.
- session Optional. Used for a pre-defined log session.
- bufferSize Optional. Defaults to 100 line. Used to buffer log line before sending in bulk. Set to
0
for real-time logging - flushInterval Optional. Defaults to 5000ms. Time intival to send logs in bulk
var workerLog = logger.create({
source : 'app',
channel : 'worker.1',
session : 'my-session-id', //Optional
bufferSize : 1, //Optional defaults to 100
flushInterval : 10000 //Optional defaults to 5000ms
});
To start the flushInterval
you must call start()
workerLog.start();
To stop the flushInterval
you must call stop()
workerLog.stop();
To log a line just call log()
workerLog.log("my great text I need logged");
Mark a time. Finish timer, record output.
workerLog.time('my-timmer');
setTimeout(function() {
workerLog.timeEnd('my-timmer');
}, 1000);
Print to stderr Trace :
, followed by the formatted message and stack trace to the current position.
workerLog.trace();
Uses util.inspect
on obj and prints resulting string to log server. This function bypasses any custom inspect()
function on obj
.
workerLog.dir(obj);
The logging instance is designed to be used as a stream so you can pipe to it. This is useful for logging stdin, stdout and reading from a file.
fs.createReadStream('./sample_traffic.log').pipe(workerLog);
Pipe the output of a program to the log server.
ps | logster-redis log
View
The viewer is used to retrive logs from the server.
Create a view call View.createView({options})
Options
- host Required. Host of the web server.
- port Required. Port of the web server.
- session Optional. Used for a pre-defined log session.
- backlog Optional. Defaults to false. Used pull all the logs from the server. If false only new logs will be pulled.
var view = logging.View.createView({
host : 'localhost',
port : 3000,
session : 'my-session-id', //Optional
backlog : true //Optional defaults to false
});
Subscribe for the data
event
view.on('data', function(data) {
console.log(data);
//{ ts: 1423363927109,
// channel: 'worker.1',
// source: 'app',
// msg: '...' }
});
Start listening for log data.
view.start();
CLI
We have a cli program that can be used to setup and use the logging service
Help
$ logster-redis -h
Usage: logster-redis [options] [command]
Commands:
view [options] View logs in teal-time.
log [options] Send logs to the server.
register [options] Register a log session with the server.
server [options] Run the log server.
Options:
-h, --help output usage information
-V, --version output the version number
Server
Starting the server is the first thing you want to do. The servers are setup to scale. You can run it as a cluster or run each part on different servers.
$ logster-redis server -h
Usage: server [options]
run setup commands for all envs
Options:
-h, --help output usage information
-a, --addr [HOST] Bind to HOST address (default: 127.0.0.1)
-p, --port [PORT] Use PORT (default: 5000)
-A, --redis-addr [HOST] Connect to redis HOST address (default: 127.0.0.1)
-P, --redis-port [PORT] Connect to redis PORT (default: 6379)
-o, --redis-auth [PASSWORD] Use redis auth
-w, --web Start Web-Server
-u, --udp Start UDP-Server
-c, --cluster Start server as cluster
#####Example
$ logster-redis server -w -u -c
View
Using -f
to wire the logs to file for persistan storage.
$ logster-redis view -h
Usage: view [options]
run setup commands for all envs
Options:
-h, --help output usage information
-a, --addr [HOST] Bind to HOST address (default: 127.0.0.1)
-p, --port [PORT] Use PORT (default: 5000)
-S, --source [SOURCE] Source to use (database)
-c, --channel [CHANNEL] Channel to use (redis.1)
-e, --session [SESSION] Session to use (default: SESSION)
-b, --backlog Retrieve all logs from the server (default: false)
-f, --file [FILE] File to write to
#####Example
$ logster-redis view -b -e my-session-token -f /path/to/file.log
Log
$ logster-redis log -h
Usage: log [options]
run setup commands for all envs
Options:
-h, --help output usage information
-a, --addr [HOST] Bind to HOST address (default: 127.0.0.1)
-p, --port [PORT] Use PORT (default: 5000)
-S, --source [SOURCE] Source to use (default: stdin)
-c, --channel [CHANNEL] Channel to use (default: process.1)
-e, --session [SESSION] session to use (default: SESSION)
-b, --bufferSize [SIZE] bufferSize to use (default: 100)
-f, --flushInterval [SIZE] Interval to flush the buffer (default: 5000ms)
#####Example
$ ps | logster-redis log -e my-session-token
$ cat /path/to/file.log | logster-redis log -e my-session-token -s my-custom-source -c dev-server.1
$ tail /path/to/file.log | logster-redis log -e my-session-token -s my-custom-source -c dev-server.1
Register
If you want to register a session token but dont want to use it yet.
$ logster-redis register -h
Usage: register [options]
run setup commands for all envs
Options:
-h, --help output usage information
-a, --addr [HOST] Bind to HOST address (default: 127.0.0.1)
-p, --port [PORT] Use PORT (default: 5000)
-e, --session [SESSION] session to use (default: SESSION)
#####Example
$ logster-redis register -e my-custom-session