exframe-rating

Rating engine build tools and test runner for insurance rating calculations. Supports building rating engines from CSV source files and running integration tests for both v2 and v3 engine versions.

Overview

This package provides:

• Engine Builder: Compile rating engines from CSV factor files and metadata into executable YAML configurations
• Test Runner: Execute integration tests against company packages with support for v2 and v3 rating engines
• CLI Interface: Unified command-line interface for building engines and running tests

Installation

This package is part of the exframe monorepo and is not published separately. It depends on:

• exframe-rating-engine - Rating engine execution (v2 and v3)
• exframe-query-engine - SQL query engine for v3 rating

Commands

build

Build a rating engine YAML file from CSV source files and factor metadata.

node exframe/packages/exframe-rating/index.js build <rateCode> -i <input-dir> -o <output-dir> [-t]

Arguments:

• rateCode - Rate code identifier (e.g., 20240101)

Options:

• -i, --input <directory> - Path to the input directory containing CSV files and metadata
• -o, --output <directory> - Path to the output directory for generated YAML file
• -t, --test - Test mode: display output in console instead of creating file
• -v, --verify - Verify mode: compare generated rating with document.json

Example:

node exframe/packages/exframe-rating/index.js build 20240101 \
  -i company-packages/ttic-ct-ho3/engine-sources/20240101/ \
  -o company-packages/ttic-ct-ho3/data/engines/

runTest

Run integration tests for a company package's rating engine.

node exframe/packages/exframe-rating/index.js runTest <package-dir> [options]

Arguments:

• packageDir - Path to the company package directory (relative or absolute)

Options:

• --rerun-failed - Only rerun previously failed tests
• -v, --verbose - Show verbose output including passing tests

Example:

# Run all tests
node exframe/packages/exframe-rating/index.js runTest company-packages/ttic-ct-ho3

# Run with verbose output
node exframe/packages/exframe-rating/index.js runTest company-packages/ttic-ct-ho3 --verbose

# Rerun only failed tests
node exframe/packages/exframe-rating/index.js runTest company-packages/ttic-ct-ho3 --rerun-failed

updateTest

Update test expectations with actual rating results. This is useful when engine logic changes and you need to update the expected premiums in your test data.

node exframe/packages/exframe-rating/index.js updateTest <package-dir> [options]

Arguments:

• packageDir - Path to the company package directory (relative or absolute)

Options:

• --rerun-failed - Only rerun previously failed tests
• -v, --verbose - Show verbose output

Example:

node exframe/packages/exframe-rating/index.js updateTest company-packages/ttic-ct-ho3

generateFailedTestDocs

Generate JSON documents with rating results for failed tests. This is used to capture baseline results from one engine version to compare against another.

node exframe/packages/exframe-rating/index.js generateFailedTestDocs <package-dir> [options]

Arguments:

• packageDir - Path to the company package directory (relative or absolute)

Options:

• -v, --verbose - Show verbose output

Example:

node exframe/packages/exframe-rating/index.js generateFailedTestDocs company-packages/ttic-sd-ho3

Workflow for Debugging Engine Differences:

This workflow helps you compare results between two engine versions (e.g., v2 vs v3) to identify what changed.

Initial Setup:

Assuming you have two engine versions:

• 20220515-v2.yml - The OLD/baseline engine (known to pass tests)
• 20220515-v3.yml - The NEW engine (suspected to have differences)

Step 1: Activate the NEW v3 engine and run tests

# Rename engines to activate v3
cd company-packages/ttic-sd-ho3/data/engines
mv 20220515.yml 20220515-v2.yml      # Preserve old engine
mv 20220515-v3.yml 20220515.yml      # Activate new engine
cd ../../../..

# Run tests to capture failures
node exframe/packages/exframe-rating/index.js runTest company-packages/ttic-sd-ho3

This creates two files:

• integration-tests/failed-tests.txt - List of failed test IDs
• integration-tests/failed-tests-metadata.json - Premium results from the NEW v3 engine

Step 2: Switch back to OLD v2 engine

cd company-packages/ttic-sd-ho3/data/engines
mv 20220515.yml 20220515-v3.yml      # Preserve new engine
mv 20220515-v2.yml 20220515.yml      # Activate old baseline engine
cd ../../../..

Step 3: Generate comparison documents with OLD engine's results

node exframe/packages/exframe-rating/index.js generateFailedTestDocs company-packages/ttic-sd-ho3

This creates document-failed-{testId}.json files in engine-sources/20220515/ containing:

• Input data from the test
• Complete rating result from OLD v2 engine (baseline)
• Metadata with BOTH v2 and v3 results for comparison

Step 4: Analyze the differences

Open a generated document (e.g., document-failed-TTIC.SD.HO3_RATE_20220515_TEST_057.json):

{
    "property": { ... },
    "coverageLimits": { ... },
    "rating": {
        "premium": 1408,      // v2 baseline result
        "factors": [ ... ]
    },
    "_metadata": {
        "csvExpectedPremium": 1408,
        "baselinePremium": 1408,        // What v2 calculated
        "newEnginePremium": 1349,       // What v3 calculated
        "variance": -59,                 // Difference
        "baselineEngineNote": "baselinePremium is the result from the OLD/baseline engine",
        "newEngineNote": "newEnginePremium is the result from the NEW engine that failed the test"
    }
}

• _metadata.baselinePremium - What the OLD v2 engine calculated
• _metadata.newEnginePremium - What the NEW v3 engine calculated
• _metadata.variance - Difference between the two
• rating object - Complete factor breakdown from v2 engine

Step 5: (Optional) Manually test with NEW engine

# Switch back to v3 engine
cd company-packages/ttic-sd-ho3/data/engines
mv 20220515.yml 20220515-v2.yml
mv 20220515-v3.yml 20220515.yml
cd ../../../..

# Rename a failed document to document.json for manual testing
cd company-packages/ttic-sd-ho3/engine-sources/20220515
cp document-failed-TTIC.SD.HO3_RATE_20220515_TEST_057.json document.json

Then run the v3 engine manually and compare the detailed output against the rating object from the v2 baseline.

Key Point: The metadata contains results from BOTH engines, making it easy to see exactly what changed without manual re-runs.

Test File Structure

node exframe/packages/exframe-rating/index.js updateTest <package-dir> [options]

Arguments:

• packageDir - Path to the company package directory (relative or absolute)

Options:

• --rerun-failed - Only rerun previously failed tests
• -v, --verbose - Show verbose output

Example:

node exframe/packages/exframe-rating/index.js updateTest company-packages/ttic-ct-ho3

Test File Structure

Company packages should follow this structure for testing:

company-package/ ├── data/ │ └── engines/ │ ├── 20240101.yml # Engine version files (YYYYMMDD.yml) │ ├── 20250101.yml │ └── 20260101.yml └── integration-tests/ └── data/ └── rating-engine.csv # Test data

Test CSV Format

The rating-engine.csv file should contain test cases with these columns:

Required Columns:

• skip - Set to "true" to skip this test
• testId - Unique identifier for the test
• type - Description of the test
• expectedPremium - Expected premium result
• effectiveDate - Effective date (determines which engine version to use)

Input Columns: Additional columns representing nested input fields using dot notation:

• property.territory
• property.yearBuilt
• coverageLimits.dwelling.amount
• deductibles.hurricane.amount
• etc.

Engine Version Support

The test runner automatically selects the appropriate engine version based on the test's effective date.

v2 Engine

• Uses lookup-based factors with conditional logic
• Supports JSON-transpose expressions
• Factor calculations with explicit inputs

v3 Engine

• Uses SQL-based queries with relational data
• Supports complex joins and aggregations
• Query engine for factor lookups

Engine Selection Logic

1. Discovers all .yml files in data/engines/ directory
2. Parses effective dates from filenames (YYYYMMDD.yml)
3. For each test, selects the most recent engine not newer than the test's effective date
4. Loads and executes the appropriate engine version

Test Execution

The test runner:

1. Reads test data from integration-tests/data/rating-engine.csv
2. For each test:
   • Formats the document from CSV row data
   • Selects appropriate engine based on effective date
   • Runs the rating engine
   • Compares actual vs expected premium (tolerance: +/- $1)
   • Reports pass/fail status
3. Saves failed test IDs to integration-tests/failed-tests.txt
4. Optionally updates CSV with actual values in fix mode

Usage

You can also use the test runner programmatically:

import { runTests, generateFailedTestDocuments } from 'exframe-rating/lib/test-runner.js';

// Run tests
const results = await runTests('/path/to/company-package', {
  fix: false,           // Update test expectations
  rerunFailed: false,   // Only rerun failed tests
  verbose: false        // Show verbose output
});

console.log(`Total: ${results.total}`);
console.log(`Passed: ${results.passed}`);
console.log(`Failed: ${results.failed}`);
console.log(`Skipped: ${results.skipped}`);

// Generate documents for failed tests
const genResults = await generateFailedTestDocuments('/path/to/company-package', {
  verbose: false
});

console.log(`Generated: ${genResults.generated}`);

NPM Scripts

Add convenience scripts to your root package.json:

{
  "scripts": {
    "test:rating": "node exframe/packages/exframe-rating/index.js runTest",
    "test:rating:update": "node exframe/packages/exframe-rating/index.js updateTest",
    "build:engine": "node exframe/packages/exframe-rating/index.js build"
  }
}

Then run:

npm run test:rating company-packages/ttic-ct-ho3
npm run test:rating:update company-packages/ttic-ct-ho3
npm run build:engine 20240101 -i company-packages/ttic-ct-ho3/engine-sources/20240101/ -o company-packages/ttic-ct-ho3/data/engines/

Failed Tests

When tests fail, their IDs are automatically added to integration-tests/failed-tests.txt in the package directory. This allows you to:

1. Review which tests failed
2. Rerun only failed tests using --rerun-failed flag
3. Fix issues incrementally without rerunning all tests

The failed tests file is automatically cleared when all tests pass.

Generating Documents for Failed Tests

Use the generateFailedTestDocs command to create JSON documents with complete rating results for failed tests. These documents include:

• Input data (formatted from CSV)
• Complete rating result from the current engine
• Metadata (test ID, rate code, expected vs actual premium)

Use Case: When you need to compare results between two engine versions:

1. Run tests with the new engine to identify failures
2. Switch to old engine (the known-good version)
3. Generate documents with generateFailedTestDocs to capture old engine's results
4. Switch to new engine
5. Manually test by renaming a generated document to document.json and running the rating engine
6. Compare the old engine's result (stored in rating) with the new engine's output

Example Generated Document Structure:

{
    "property": {
        "territory": "57783-46",
        "yearBuilt": 2011,
        ...
    },
    "coverageLimits": { ... },
    "rating": {
        "premium": 1408,
        "factors": [ ... ],
        ...
    },
    "_metadata": {
        "testId": "TTIC.SD.HO3_RATE_20220515_TEST_057",
        "rateCode": "20220515",
        "generatedAt": "2026-05-12T10:30:00.000Z",
        "csvExpectedPremium": 1408,
        "baselinePremium": 1408,
        "baselineEngineNote": "baselinePremium is the result from the OLD/baseline engine",
        "newEnginePremium": 1349,
        "newEngineExpected": 1408,
        "variance": -59,
        "newEngineNote": "newEnginePremium is the result from the NEW engine that failed the test"
    }
}

The rating object contains the complete output from the OLD (baseline) engine. The _metadata shows both engine results and the variance, making it easy to identify the discrepancy.

License

ISC