@axios-software/ghgmet-model-translator

v0.2.3

Published

12 days ago

Transform agricultural field data into model-specific input formats for greenhouse gas modeling

0High
0Medium
0Low

newneonova

axiossoftware

agriculture greenhouse-gas modeling translation daycent dndc carbon emissions

GHGMet Model Translator

Transform agricultural field data into model-specific input formats for greenhouse gas modeling

The GHGMet Model Translator is a production-ready TypeScript library that converts standardized agricultural field datasets into model-specific input formats for agricultural greenhouse gas models. It supports batch processing, comprehensive validation, and extensible architecture for multiple model types.

📦 Installation

# NPM (recommended)
npm install @axios-software/ghgmet-model-translator

# Yarn
yarn add @axios-software/ghgmet-model-translator

# pnpm
pnpm add @axios-software/ghgmet-model-translator

Current Version: 0.1.3 - View Changelog

🚀 Quick Start

👥 For New Developers

Ready to contribute? See QUICKSTART.md for express 5-minute setup!

📦 For Package Users

import { TranslatorEngine, ConfigManager } from '@axios-software/ghgmet-model-translator';
import {
  Daycent30cmTranslator,
  LandscapeDndcTranslator,
} from '@axios-software/ghgmet-model-translator';

// Initialize translator engine
const configManager = new ConfigManager();
const translator = new TranslatorEngine(configManager);

// Register translators for supported models
translator.registerTranslator('daycent_30cm', new Daycent30cmTranslator());
translator.registerTranslator('landscape_dndc', new LandscapeDndcTranslator());

// Translate field data
const result = await translator.translateBatch({
  requestId: 'run-001',
  fields: fieldDatasets,
});

console.log(`✅ Translated ${result.successful.length} fields successfully`);

🎯 Production Status

✅ Production Ready - All 8 development phases complete

✅ Core Translation Engine: Robust field data transformation with comprehensive error handling
✅ Multi-Model Support: DayCent 30cm and Landscape DNDC translators with extensible architecture
✅ Validation Framework: Input validation, data integrity checks, and configurable error handling
✅ Spatial Data Integration: Hybrid provider architecture with external APIs and internal database support
✅ CI/CD Pipeline: Automated testing, security auditing, and dual NPM/GitHub Packages publishing
✅ Production Deployment: Docker support, monitoring, scaling considerations, and performance optimization
✅ Comprehensive Testing: 102 tests passing with 77% coverage across all components
✅ Integration Ready: Real-world usage patterns and Model Controller integration examples

Ready for Production Deployment 🚀

🚀 Features

Multi-Model Support: DayCent 30cm, Landscape DNDC, with plugin architecture for future models
Batch Processing: Handle 1-50 field datasets in a single request
Advanced Validation Framework:
- Management event chronology with seasonal timing validation
- Fertilizer rate validation with nutrient balance checking
- Irrigation amount validation with timing pattern analysis
- Auto-correction capabilities with safety limit enforcement
- Agricultural intelligence with crop-specific validation ranges
Hybrid Spatial Data Architecture:
- Multi-source provider support (external APIs + internal database)
- Automatic fallback chains with configurable selection strategies
- Geographic coverage awareness and intelligent routing
- Performance-optimized caching with spatial locality
- Support for researcher-provided custom spatial data
Fail-Soft Error Handling: Collect all errors in a single pass for better user feedback
Type Safety: Full TypeScript implementation with strict type checking
Cross-Platform: Windows, macOS, and Linux support
CLI Tool: Comprehensive command-line interface for development and testing

🖥️ CLI Tool

The GHGMet Model Translator includes a powerful CLI tool for developers and researchers:

# Install globally
npm install -g @axios-software/ghgmet-model-translator

# Or use npx
npx ghgmet-cli --help

Key CLI Features

File Translation: Convert between DayCent and LDNDC formats
Schema Validation: Comprehensive validation with Danish agricultural standards
File Analysis: Extract detailed information about agricultural datasets
Model Comparison: Compare DayCent vs LDNDC schemas and files
Batch Processing: Handle multiple fields efficiently
Danish Integration: Built-in support for Danish soil and weather data

Quick CLI Examples

# Translate a file
ghgmet-cli translate input.json -o output.json

# Validate against schema and Danish standards
ghgmet-cli validate field-data.json --denmark-validation

# Analyze file contents
ghgmet-cli info agricultural-data.json --coordinates --crops

# Compare model approaches
ghgmet-cli compare --schemas

📖 Full CLI Documentation

📦 Installation

NPM Package (Phase 1)

npm install @axios-software/ghgmet-model-translator

Usage

import { ModelTranslator } from '@axios-software/ghgmet-model-translator';

const translator = new ModelTranslator({
  spatialDataUrl: process.env.SPATIAL_SERVICE_URL,
  weatherApiKey: process.env.WEATHER_API_KEY,
});

// Translate batch of field datasets
const result = await translator.translateBatch({
  requestId: 'batch-001',
  fields: [
    {
      fieldId: 'field-001',
      modelType: 'daycent_30cm',
      geometry: {
        /* GeoJSON polygon */
      },
      management: {
        /* crop rotations, tillage, fertilizers */
      },
      scenarios: [
        /* scenario definitions */
      ],
    },
  ],
});

// Access model-specific outputs
console.log(result.translations[0].payload); // DayCent JSON format

🏗️ Architecture

graph TB
    A[Model Run Controller] --> B[Translator Engine]
    B --> C{Model Type Router}
    C -->|daycent_30cm| D[DayCent 30cm Translator]
    C -->|landscape_dndc| E[Landscape DNDC Translator]

    D --> F[Spatial Data Service]
    E --> F

    F --> G[SSURGO/STATSGO<br/>Soil Data]
    F --> H[PRISM<br/>Climate Data]
    F --> I[NOAA<br/>Weather Data]

    D --> J[DayCent JSON Output]
    E --> K[LDNDC JSON Output]

Key Design Patterns

Plugin Architecture: Extensible translator system
Fail-Soft Processing: Continue on errors to collect comprehensive feedback
Base Class Pattern: Shared validation with model-specific implementations
Strategy Pattern: Different translation strategies per model type

🎯 Supported Models

| Model | Status | Description | | ------------------ | ------------ | ------------------------------------------------ | | DayCent 30cm | ✅ Supported | Agricultural GHG model with 30cm soil depth | | Landscape DNDC | ✅ Supported | Landscape-scale model with hydrology integration | | COMET-Farm | 🚧 Planned | USDA carbon accounting model | | Standard DNDC | 🚧 Planned | Point-scale biogeochemical model |

📊 Input/Output Examples

Input (Standardized Format)

{
  "fieldId": "field-001",
  "modelType": "daycent_30cm",
  "geometry": {
    "type": "Polygon",
    "coordinates": [[[...]]],
    "area": 120.5
  },
  "management": {
    "currentRotation": [
      {
        "year": 2020,
        "crops": [{"name": "Corn", "plantingDate": "2020-05-01"}],
        "fertilizers": [{"type": "Urea", "rate": 150, "units": "lbs/acre"}]
      }
    ]
  }
}

Output (DayCent 30cm Format)

{
  "fieldId": "field-001",
  "modelType": "daycent_30cm",
  "success": true,
  "payload": {
    "weatherFiles": [{ "fileName": "gridx_665_gridy_221.wth" }],
    "siteFiles": [{ "fileName": "31159_665_221_1691220_till_N" }],
    "scheduleFiles": {
      "scenarios": [
        {
          "id": "baseline",
          "events": [
            { "month": 1, "day": 90, "operation": "CROP", "parameters": "C6" },
            { "month": 1, "day": 90, "operation": "FERT", "parameters": "(22.96N, 0.8F)" }
          ]
        }
      ]
    }
  }
}

🛠️ Development

Prerequisites

Node.js 18+
TypeScript 5.0+
Git

Quick Start

# Clone repository
git clone https://github.com/Axios-Software-Inc/ghgmet-model-translator.git
cd ghgmet-model-translator

# Install dependencies
npm install

# Set up environment
cp .env.example .env

# Build project
npm run build

# Run tests
npm test

# Start development server
npm run dev

Project Structure

src/
├── core/                 # Core translation engine
├── models/               # Model-specific translators
│   ├── daycent-30cm/     # DayCent 30cm translator
│   └── landscape-dndc/   # Landscape DNDC translator
├── validation/           # Validation engine
├── spatial/              # Spatial data services
└── types/                # TypeScript definitions

📚 Documentation

Development Guide - Environment setup and workflow
Architecture Overview - System design and patterns
API Reference - Complete API documentation
Deployment Guide - Local and production deployment
Agent Context Guide - AI agent development guide

🔧 Configuration

Environment Variables

# Service Configuration
NODE_ENV=development
LOG_LEVEL=debug

# Spatial Data Sources
GEOSERVER_URL=https://geo.ghgmet.org
WEATHER_API_KEY=your_api_key

# Performance Settings
MAX_PARALLEL_TRANSLATIONS=10
WORKER_THREAD_COUNT=4
TRANSLATION_TIMEOUT_MS=30000

# Feature Flags
ENABLE_DNDC_SUPPORT=true
ENABLE_VALIDATION_AUTOFIX=true

🧪 Testing

# Run all tests
npm test

# Run with coverage
npm run test:coverage

# Run specific test suites
npm run test:unit
npm run test:integration

# Performance testing
npm run test:performance

Test Coverage Goals

Core Logic: 95%+ coverage
Model Translators: 90%+ coverage
Integration Tests: End-to-end workflows
Performance Tests: Load testing with k6

🚀 Deployment

Phase 1: NPM Package

Embedded within Model Run Controller
Synchronous execution
Shared memory and resources

Phase 2: Microservice (Future)

Standalone NestJS service
Independent scaling
gRPC/REST API interface
Kubernetes deployment

🤝 Contributing

Fork the repository
Create a feature branch (git checkout -b feature/amazing-feature)
Commit changes (git commit -m 'feat: Add amazing feature')
Push to branch (git push origin feature/amazing-feature)
Open a Pull Request

Commit Message Format

[type]: Brief description

- Detailed point 1
- Detailed point 2

Refs: #issue-number

Types: feat, fix, docs, style, refactor, test, chore

📈 Performance

| Metric | Target | Maximum | | ----------------------------- | ------- | ------- | | Single Field Translation | < 2s | 10s | | Batch Translation (10 fields) | < 5s | 30s | | Spatial Data Lookup | < 500ms | 2s | | Memory Usage | < 500MB | 2GB |

🔒 Security

Input validation and sanitization
Rate limiting and API key authentication
Audit logging for compliance (HECVAT/VPAT)
No sensitive data storage

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.

🙋‍♂️ Support

Documentation: docs/
Issues: GitHub Issues
Discussions: GitHub Discussions

🗺️ Roadmap

Q4 2025: COMET-Farm translator implementation
Q1 2026: Microservice architecture migration
Q2 2026: Standard DNDC support
Q3 2026: Performance optimization and caching layer

Built with ❤️ by the GHGMet Platform Team

Published

Vulnerabilities

Links

Maintainers

Keywords

Readme

GHGMet Model Translator

📦 Installation

🚀 Quick Start

👥 For New Developers

📦 For Package Users

🎯 Production Status

🚀 Features

🖥️ CLI Tool

Key CLI Features

Quick CLI Examples

📦 Installation

NPM Package (Phase 1)

Usage

🏗️ Architecture

Key Design Patterns

🎯 Supported Models

📊 Input/Output Examples

Input (Standardized Format)

Output (DayCent 30cm Format)

🛠️ Development

Prerequisites

Quick Start

Project Structure

📚 Documentation

🔧 Configuration

Environment Variables

🧪 Testing

Test Coverage Goals

🚀 Deployment

Phase 1: NPM Package

Phase 2: Microservice (Future)

🤝 Contributing

Commit Message Format

📈 Performance

🔒 Security

📄 License

🙋‍♂️ Support

🗺️ Roadmap