metrics-reporter

v2.2.0

Published

a month ago

Metrics reporting framework for reporting data point information to aggregators (like Graphite)

0High
0Medium
0Low

ysa23

metrics graphite performance datadog DogStatsD

Metrics

Metrics is a time series reporting framework for aggregators and metrics collectors such as Graphite.

Highlights

Time series reporting
Plugin based: Support different aggregators with pluggable reporters
Built in reporters:
Simple, easy to use API
Focused on performance for high throughput applications and services

Getting started

Installation

Configuration

Import metrics package:

const { Metrics } = require('metrics-reporter');

Initialize the metrics instance with the required reporters:

const { StringReporter, ConsoleReporter } = require('metrics-reporter');

const stringReporter = new StringReporter({ action: metricString => {
        // Do something
    }});
const consoleReporter = new ConsoleReporter();

const reporters = [stringReporter, consoleReporter];// Array of reporters to trigger when a metrics should be reported
const errback = (err) => { console.error(err);};    // Optional - A function to be called when an error occurs
const tags = { tag1: 'value1' };                    // Optional - key-value pairs to be appanded to all the metrics reported

const metrics = new Metrics({ 
  reporters,
  tags,
  errback 
});

Reporting Metrics

Use the space method on the Metrics instance to report custom metrics. space creates a new key to report:

const metric = metrics.space('http');

Spaces can be nested:

const metric = metrics.space('http').space('requests'); // http.requests

Execution time

Use the meter method on a Space to report execution time of a function:

// Callback function
const wrapper = metrics.space('users.get').meter(function(userIds, callback) {
	// read users from database
	callback(...);
});

wrapper([1, 2, 3], (err, result) => { console.log(result); });

The meter method can receive:

A function with a callback (as the last parameter)
Promise
Async function

meter returns a wrapper around the object that was sent. In order to start measuring invoke it according to its type. For example:

// Sync invocation
const wrapperSync = metrics.space('add').meter((a, b) => a + b);
const result = wrapperSync(1, 2);

// Promise invocation
const wrapperPromise = metrics.space('timeout').meter(new Promise(function(resolve) {
  setTimeout(() => console.log('hello'), 10000);
}));
await wrapperPromise();

The meter function will run your code, while measuring the time it took to execute, and report it to the configured reporters.

Note:

In a callback: Metrics are reported only after the callback is called
In a promise and async function: Metrics are reported once the promise fulfills (either success or failure)

If an async function is measured, you can await on it and get its returned value:

const result = await metrics.space('users.get').meter(async () => {
    // Some async code here
})();

Please note the invocation on the return value.

Value

Use the Metrics instance to report a value:

metrics.space('api.response.size').value(512);

Increment

Use the Metrics instance to increment a key:

metrics.space('api.requests').increment();

Tagging support

Tags are specified per space, as an object:

metrics.space('http.requests', { path: 'users_get' }).increment();

When nesting spaces, the tags are aggregated:

metrics
    .space('http', { verb: 'GET' })
    .space('requests', { path: 'users' })
    .increment(); 
    // will increment 'http.requests' with 'verb:GET,path:users' tags

Use tags function to create a space with the same key and additional tags:

metrics
    .space('http', { verb: 'GET' })
    .tags({ path: 'users' })
    .increment(); 
    // will increment 'http' with 'verb:GET,path:users' tags

Note

When the same tag is specified when creating nested spaces, the last value will be reported

Error handling

Metrics support error handling. When creating a Metric object you can send an error callback:

const metrics = new Metrics({ 
  reporters: [new ConsoleReporter()],
  errback: e => {
    // e is a javascript Error object. You can log it on any standard logging framework:
    logger.error(e);
  }
});

The error callback receives a single parameter - an Error instance. The callback will be triggered when any error occurs during the metrics reporting.

Please note: Some reporters require their own error handler. Make sure to initialize errback with them as well.

Reporters

Metrics comes with several built-in reporters

Graphite

Reports metrics to a graphite server (via statsd):

const { Metrics, GraphiteReporter } = require('metrics-reporter');

const graphiteHost = '1.1.1.1';         // Graphite server IP address
const graphitePort = 8125;              // Optional - port number. Defaults to 8125
const spacePrefix = 'My.Project';       // Optional - prefix to all metrics spaces
const batch = true;                     // Optional - Default `true` - Indicates that metrics will be sent in batches
const maxBufferSize = 500;              // Optional - Default `1000` - Size of the buffer for sending batched messages. When buffer is filled it is flushed immediately
const flushInterval = 1000;             // Optional - Default `1000` (1s) - Time in milliseconds. Indicates how often the buffer is flushed in case batch = true
const errback = (err) => {              // Optional - function to be triggered when an error occurs 
    console.error(err) 
};

const graphiteReporter = new GraphiteReporter({
  host: graphiteHost,
  port: graphitePort,
  prefix: spacePrefix,
  batch,
  maxBufferSize,
  flushInterval,
  errback,
});

const metrics = new Metrics({ reporters: [graphiteReporter] });

graphiteReporter.close(); // close should be called when the application terminates

DataDog

Reports metrics to a DataDog (via DogStatsD):

const { Metrics, DataDogReporter } = require('metrics-reporter');

const agentHost = '1.1.1.1';            // DataDog agent IP address
const port = 8125;                      // Optional - Default `8125` - port number. Defaults to 8125
const spacePrefix = 'My.Project';       // Optional - prefix to all metrics spaces
const batch = true;                     // Optional - Default `true` - Indicates that metrics will be sent in batches
const maxBufferSize = 500;              // Optional - Default `1000` - Size of the buffer for sending batched messages. When buffer is filled it is flushed immediately
const flushInterval = 1000;             // Optional - Default `1000` (1s) - Time in milliseconds. Indicates how often the buffer is flushed in case batch = true
const errback = (err) => {              // Optional - function to be triggered when an error occurs 
  console.error(err)
};

const datadogReporter = new DataDogReporter({
  host: agentHost,
  port,
  prefix: spacePrefix,
  batch,
  maxBufferSize,
  flushInterval,
  errback,
});

const metrics = new Metrics({ reporters: [datadogReporter] });

datadogReporter.close(); // close should be called when the application terminates

Note that you'll need a running DataDog agent. In the /docker folder there's a simple docker compose for datadog to get you started

Console

Console reporter comes in handy when you need to debug metrics calls:

const { Metrics, ConsoleReporter } = require('metrics-reporter');

const consoleReporter = new ConsoleReporter();
	
const metrics = new Metrics({ reporters: [consoleReporter] });

When a metrics will be reported, a message will appear in the terminal, that includes the key and the value reported.

String

const { Metrics, StringReporter } = require('metrics-reporter');
const fs = require('fs');

const stringReporter = new StringReporter({
    action: metricString => {
        fs.appendFile('metrics.log', metricsString);
    },
});
	
const metrics = new Metrics({ reporters: [stringReporter] });

Here, StringReporter is used to build a log file from the metrics reports.

InMemory

InMemoryReporter can be used for testing purposed, in order to make sure your code reports metrics as expected.

const { Metrics, InMemoryReporter } = require('metrics-reporter');

const metricsStorage = [];

const memoryReporter = new InMemoryReporter({ buffer: metricsStorage });

const metrics = new Metrics({ reporters: [memoryReporter], errback: error => { /* Do something on error */ } });

When a metric is reported, an object with key, value and tags properties is pushed to the array. Then, the array can be used in order to validate the report.

Prometheus (experimental)

PrometheusReporter is an experimental reporter that generates metrics in the Prometheus text exposition format without any external dependencies:

const { Metrics, PrometheusReporter } = require('metrics-reporter');
const express = require('express');

const prefix = 'myapp_';                // Optional - prefix for all metric names
const softLimit = 5000;                 // Optional - Default `5000` - Reset metrics after scrape when exceeded
const hardLimit = 10000;                // Optional - Default `10000` - Force reset to prevent OOM
const warnAt = 4000;                    // Optional - Default `4000` - Log warning when approaching soft limit
const buckets = [10, 50, 100, 250];     // Optional - Custom histogram buckets (ms)
const logCallback = (logEvent) => {     // Optional - function to handle log events
  console.log(`[${logEvent.level.toUpperCase()}] ${logEvent.message}`, logEvent.params);
};

const prometheusReporter = new PrometheusReporter({
  prefix,
  softLimit,
  hardLimit,
  warnAt,
  buckets,
  logCallback,
});

const metrics = new Metrics({ reporters: [prometheusReporter] });

// Expose metrics endpoint for Prometheus to scrape
const app = express();
app.get('/metrics', (req, res) => {
  res
    .type('text/plain')
    .send(prometheusReporter.getMetrics());
});

app.listen(3000);

Design Principles

The PrometheusReporter implements a double-threshold strategy to prevent memory issues common with high-cardinality metrics:

Soft Limit (default 5000): When the number of unique metrics exceeds this limit, they are reset after the next scrape, ensuring Prometheus receives the data before clearing.
Hard Limit (default 10000): An emergency threshold that immediately resets all metrics to prevent out-of-memory errors, even if Prometheus hasn't scraped yet.
Warning Threshold (default 4000): Logs a warning when approaching the soft limit, helping identify cardinality issues before they become critical.

This approach provides memory safety while maintaining data integrity, unlike traditional Prometheus clients that can suffer from unbounded memory growth.

Metric Type Mapping

increment() → Counter (with _total suffix)
value() → Gauge
report() → Histogram (with buckets, sum, and count)

Configuration Recommendations

Configuration should be according to your use-case, use these as guidelines to an initial configuration and tweak as needed:

Low traffic: Use defaults (soft: 5000, hard: 10000)
High traffic: Increase limits (soft: 20000, hard: 50000)
Microservices: Lower limits (soft: 1000, hard: 2000)
Development: Very low limits for testing (soft: 100, hard: 200)

Log Events

The PrometheusReporter provides structured logging through an optional logCallback function. This allows you to handle log events programmatically instead of relying on console output.

const prometheusReporter = new PrometheusReporter({
  logCallback: (logEvent) => {
    // Handle log events with your preferred logging library
    logger.log(logEvent.level, logEvent.message, {
      code: logEvent.code,
      params: logEvent.params,
      reporter: logEvent.reporter,
      timestamp: logEvent.timestamp
    });
  }
});

Log Event Structure:

level: 'error' | 'warn' | 'info' | 'debug'
code: Event code for programmatic handling
message: Human-readable message
params: Additional parameters relevant to the event
timestamp: Unix timestamp in milliseconds
reporter: Always 'PrometheusReporter'

Event Codes:

APPROACHING_SOFT_LIMIT: Warning when approaching the soft limit threshold
SOFT_LIMIT_EXCEEDED: Info when soft limit exceeded during getMetrics()
HARD_LIMIT_REACHED: Error when hard limit forces immediate reset

We recommend monitoring these events, as threshold violations can cause metric data loss.

Building new reporters

Metrics support creating new reports according to an application needs.

A reporter must contain three methods:

report - for reporting time
value - for reporting a single value (size of response for example)
increment - for an incremented value over time (number of requests for example

The methods get the following parameters:

key (mandatory) - the metric to report
value (mandatory) - the value to report (ms, count or increment for example)
tags (optional) - an object that contains the tags to report on the metric as properties

For example, lets see how to implement a reporter for redis:

const client = require('redis').createClient();

function RedisReporter({
    channel, 
    errback
}) {
  function report(key, val, tags) { 
    client.publish(channel, JSON.stringify({ key, value: val, tags  }));
  }

  function value(key, val, tags) {
    client.set(key, val, (err) => {
        if (!err || !errback) {
            return;
        }
        
        errback(err);
    });
  }
 
  function increment(key, value, tags) {
    const multi = client.multi();
    for(let i = 0; i < value; i++) {
        multi.incr(key);
    }   
        
    multi.exec((err) => {
      if (!err || !errback) {
        return;
      }

      errback(err);
    });
  }
  
  return {
    report,
    value,
    increment,
  }
};

module.exports = {
  RedisReporter,
};

The new reporter will publish a message to a specified channel in redis when a metric is reported.

Development

How to contribute

We encourage contribution via pull requests on any feature you see fit.

When submitting a pull request make sure to do the following:

Run all unit and integration tests to ensure no existing functionality has been affected
Write unit or integration tests to test your changes. All features and fixed bugs must have tests to verify they work

Read GitHub Help for more details about creating pull requests

Running tests

To run tests, in command line run npm test