Mercados Pecuarios - Analytical Reports Library

This repository contains a common library of classes for the Analytical Reports services of the Big Data Platform, which is part of the Mercados Pecuarios Website solution. It provides tools to query and process time-series data from InfluxDB, applying a series of transformations and calculations.

Setup

1. Clone the repository:

   git clone <repository-url>

2. Navigate to the project directory:

   cd mercados-pecuarios-libs-analytical-reports

3. Install the dependencies:

   npm install

Running Tests

To run the test suite, use the following command:

npm run test

This will run all the unit tests and generate a coverage report.

Data Structures

This library uses a set of data structures to define queries and handle data processing. Here is a high-level overview of the most important ones.

GetPointsQuery

The GetPointsQuery interface is used to define a query to retrieve data points from InfluxDB.

export interface GetPointsQuery {
  bucket: string;
  measurement: string;
  start: Date | number;
  end: Date | number;
  field: string | string[];
  scope?: string;
  tagKey?: string;
  fieldTag?: AnalyticalReportField | AnalyticalReportField[];
  tags?: SelectedTags;
  omitColumns?: string[];
  before?: BeforeAction[];
  after?: AfterAction[];
}

DataPoint

The DataPoint interface represents a single data point returned from InfluxDB.

export interface DataPoint {
  _time: string;
  _value: number;
  [key: string]: any;
}

Actions

The library uses a chain of responsibility pattern to process the data before and after the query to InfluxDB. These processing steps are called "actions".

BeforeAction

BeforeAction objects are executed before the query is sent to InfluxDB. They can be used to modify the GetPointsQuery object.

AfterAction

AfterAction objects are executed after the data has been retrieved from InfluxDB. They can be used to transform and enrich the data points.

Example:

Here is an example of how to use an AfterAction to calculate the average of the data points.

const query: GetPointsQuery = {
  // ... other query properties
  after: [
    {
      name: 'Average',
      params: {
        groupColumn: 'month',
        years: [2023],
      },
    },
  ],
};

Environment Variables

This library uses the following environment variables for configuration:

• INFLUXDB_URL
• INFLUXDB_TOKEN
• INFLUXDB_ORG
• INFLUXDB_TIMEOUT
• INFLUXDB_QUERIES_OMIT_COLUMNS
• CONVERSIONS_FILE_PATH
• BASE_FILE_NAME
• MXN_USD_FIELD_KEY
• USD_MXN_FIELD_KEY
• AFTER_LINK_FILE_PATH
• BEFORE_LINK_FILE_PATH
• ACTION_DEFINITIONS_PATH
• ANALYTICAL_REPORTS_COLLECTION_NAME
• WIZARD_ID
• ANALYTICAL_REPORTS_STORAGE_BUCKET
• ANALYTICAL_REPORTS_PATH
• PERIODICITY_ANNUAL_LABEL
• PERIODICITY_AGRICULTURAL_COMMERCIAL_CYCLE
• PERIODICITY_SEMIANNUAL
• PERIODICITY_QUARTERLY
• PERIODICITY_MONTHLY_LABEL
• PERIODICITY_WEEKLY_LABEL