frugal-iot-logger
v2.0.16
Published
Frugal IoT logger
Downloads
2,187
Readme
Frugal IoT Logger
This is a library for a simple Mqtt Logger suitable for use with the Frugal IoT project.
Please see https://github.com/mitra42/frugal-iot for the full project.
It should be able to work with any MQTT server that allows websocket connections, but it is only (currently) tested against the Frugal-IoT mqtt server, which is mosquitto running at ws://naturalinnovation.org:9012
The default behavior of the frugal-iot-logger is to store to CSV files in a format that can be retrieved by frugal-iot-client or used for other analysis.
It can also be used with Google Spreadsheets (aka gsheets)
Example of usage with expressjs
The main example of the loggers use - that we test against - is in the frugal-iot-server repo
This combines this Logger with a simple HTTP server to serve up the logged data and user interface.
To develop the logger together with the server and client, see Developing on the server, logger and client on the wiki.
To install As a standalone server
Copy https://github.com/mitra42/frugal-iot-logger/examples/standalone
(Or clone the repo, or npm install frugal-iot-logger)
edit config.yaml
and in config.d/organizations put a yaml file for your org, using the example in
examples/standalone/config.d/organizations/dev.yaml
Make it match your hierarchy but note its fixed at 4 levels organization, project, node, topic
node standalone.jsGoogle Spreadsheets
Simple usage with Google spreadsheets is supported. This requires an addition to
an organization's config file (e.g. dev.yaml) with a reference to the app URL
and the list of topics that should be logged to the columns.
A full example is in examples/gsheets, look at gsheet.app for instructions on
setting up a spreadsheet, and at config.d/organizations/dev.yaml->forwarders for
how to configure it.
OpenSearch Integration
Readings can be sent to OpenSearch, to browse in OpenSearch Dashboards or Grafana. One document is
sent per reading, buffered and sent in bulk requests every flushSeconds, and only readings that were
logged (passed the log and duplicates rules) are sent.
Under frugal-iot-server, add it from the dashboard's Forwarders card - nothing to edit, and it
applies at once. See frugal-iot-server's INSTALL-OPENSEARCH.md.
A standalone logger (no server) takes the same settings in an organization's forwarders: list:
forwarders:
- type: opensearch
node: https://localhost:9200 # required
apiKey: ... # or username: / password:
# ca: /path/to/root-ca.pem # for a self-signed certificate
# insecure: true # skip certificate checking - local demos only
# indexPrefix: frugaliot # indexes are <prefix>-<org>-<YYYY.MM>
# flushSeconds: 10
topics: # which readings to send - leave out for all of them
- dev/lotus/+/sht/# # + is one level, a trailing # is everything belowHistory from before this was switched on, or a gap while OpenSearch was down, can be filled from the
CSV files. Under frugal-iot-server: npx --no frugal-iot-backfill-opensearch <org>. Standalone:
node scripts/backfill-opensearch.js --config <config dir> --org <org> --data <data dir>
(add --dry-run to count what would be sent).
Firebase Integration
Automatic upload of MQTT data to Firebase Realtime Database is supported. This allows real-time data access, cloud storage, and easy integration with web/mobile apps.
To enable Firebase:
- Add a
firebasesection to your organization config (e.g.dev.yaml) - Provide your Firebase service account JSON and database URL
- Data is automatically uploaded as it arrives via MQTT
Complete documentation and example: examples/firebase/README.md
Tests
npm testUses Node's built-in test runner, so there is nothing extra to install. Nothing in the suite touches a broker, Firebase, Google Sheets or the network. See test/README.md for what is covered.
Reporting problems
Complicated issues involving interaction between this repo and the frugal-iot client; nodes; or server should be posted in https://github.com/mitr42/frugal-iot/issues
Use in other projects
While this project was designed for the needs of the Frugal-IoT project, which uses a Mosquitto server and has a particular pattern of topics, we would welcome PRs to make it more generically useful to other projects. After all - the more people find it useful - the more bugs will get fixed.
