Configuration Loader

A versatile and extensible configuration loader for Node.js applications that supports loading configurations from multiple sources including local files, Consul, and Nacos, with YAML parsing and advanced merging capabilities.

中文文档 | English

Features

• ⚡ Dual Module Support: Full ES Module (ESM) & CommonJS (CJS) compatibility
• 🏗️ Extensible Architecture: Built with abstract BaseLoader class for easy extension to new configuration sources
• 📄 YAML Support: Native parsing of YAML configuration files with full feature support
• 🔗 Include Mechanism: Advanced configuration composition through file includes with deep merging
• 🌊 Deep Merge: Intelligent merging of nested objects and arrays from multiple configuration sources
• 📡 Multiple Sources: Support for local files, Consul KV store, and Nacos configuration center
• ⚙️ Environment Configuration: Easy configuration through environment variables
• 🔄 Post-Processing: Custom content transformation with PostLoader functions
• 📝 Observable: Loading traced through the @ticatec/logger-api contract, with configuration values kept out of the log

Installation

pnpm add @ticatec/config-loader @ticatec/logger-api
# or npm
npm install @ticatec/config-loader @ticatec/logger-api

@ticatec/logger-api is a peer dependency - a zero-dependency logging contract. With no provider registered it writes to the console, so nothing else is required.

@ticatec/consul-client and nacos are optional peers; install only the one your configuration source needs:

pnpm add @ticatec/consul-client     # for CONFIG_MODE=consul
pnpm add nacos                      # for CONFIG_MODE=nacos

Quick Start

ES Modules (ESM):

import { loadConfig } from '@ticatec/config-loader';

CommonJS (CJS):

const { loadConfig } = require('@ticatec/config-loader');

import dotenv from 'dotenv';

async function main() {
    dotenv.config();
    
    const substitute = (content: string) => content.replace(/#{service-name}/g, 'my-service');

    const config = await loadConfig(
        'local', // or 'consul', 'nacos'
        'app.yaml',
        'logger.yaml',
        substitute,   // applied to logger.yaml
        substitute    // applied to app.yaml
    );
    
    console.log('App Config:', config.appConf);
    console.log('Logger Config:', config.loggerConf);
}

main();

Configuration Sources

Local Files

The default configuration source loads files from the ./config directory.

Environment Variables: None required

Example Structure:

config/
├── app.yaml
├── database.yaml
└── logger.yaml

Consul KV Store

Load configurations from HashiCorp Consul's Key-Value store via @ticatec/consul-client.

Environment Variable Specifications & Priority:

• CONSUL_HTTP_ADDR - Complete Consul address (e.g. http://127.0.0.1:8500 or https://consul.service:8501). Highest Priority; overrides CONSUL_HOST and CONSUL_PORT.
• CONSUL_HOST - Consul server hostname or IP address (optional, default 127.0.0.1).
• CONSUL_PORT - Consul server port (optional, default 8500, default 8501 in SSL/HTTPS mode).
• CONSUL_HTTP_TOKEN / CONSUL_TOKEN - Consul ACL access token (optional).
• CONSUL_HTTP_SSL / CONSUL_SSL - Set to "true" to enable HTTPS connection (default HTTPS port: 8501). Generic SSL is not supported.
• CONSUL_HTTP_SSL_VERIFY - Set to "false" to disable TLS certificate verification (optional).
• CONSUL_CACERT - Path to CA certificate file in PEM format (optional).
• CONSUL_CLIENT_CERT / CONSUL_CLIENT_KEY - Path to client certificate / private key in PEM format (optional).

Example:

// Set environment variables
process.env.CONSUL_HOST = 'localhost';
process.env.CONSUL_PORT = '8500';
process.env.CONSUL_TOKEN = 'your-consul-token';
process.env.CONSUL_SSL = 'false';

const config = await loadConfig('consul', 'app/config', 'app/logger', null);

You can also pass ConsulOptions directly via getLoader('consul', options) or loadConfig('consul', ..., loaderOptions).

Nacos Configuration Center

Loads configuration from Alibaba Nacos configuration management platform.

Environment Variables & Priority:

• NACOS_SERVER_ADDR - Direct Nacos server address or cluster node list (Recommended, Highest Priority). Supports:
  • Single node address (e.g., 127.0.0.1:8848)
  • Multi-node cluster list (e.g., nacos1:8848,nacos2:8848,nacos3:8848)
  • IPv6 addresses (e.g., [::1]:8848)
  • Full URL formats (e.g., http://localhost:8848 or https://nacos.example.com)
• NACOS_ENDPOINT - Aliyun ACM / MSE domain endpoint (e.g., acm.aliyun.com)
• NACOS_PORT - Nacos server port (optional, default 8848, default 443 in HTTPS mode)
• NACOS_SSL - Set to "true" to enable HTTPS/TLS connection (takes precedence over generic SSL)
• NACOS_NAMESPACE - Nacos namespace ID (optional)
• NACOS_GROUP - Nacos configuration group (optional, default: "default")

Example:

// 1. Direct Nacos single node / cluster example (Recommended)
process.env.NACOS_SERVER_ADDR = '127.0.0.1:8848';
// Or multi-node cluster: process.env.NACOS_SERVER_ADDR = 'nacos1:8848,nacos2:8848';
process.env.NACOS_NAMESPACE = 'production';
process.env.NACOS_GROUP = 'app-configs';

// 2. Aliyun ACM domain endpoint example
// process.env.NACOS_ENDPOINT = 'acm.aliyun.com';
// process.env.NACOS_SSL = 'true';

const config = await loadConfig('nacos', 'app.yaml', 'logger.yaml', null);

Advanced Features

Configuration Includes

The include mechanism allows you to compose configurations from multiple sources:

# main-config.yaml
app:
  name: My Application
  version: 1.0.0

includes:
  - file: database.yaml
    key: database
    params:
      poolSize: 10
  - file: redis.yaml
    key: cache
    params:
      ttl: 3600
  - file: overrides.yaml
    optional: true # Optional include: skipped if file or Consul/Nacos key does not exist

# database.yaml
host: localhost
port: 5432
name: mydb

The resulting configuration will be:

app:
  name: My Application
  version: 1.0.0
database:
  host: localhost
  port: 5432
  name: mydb
  poolSize: 10
cache:
  # redis.yaml content merged here
  ttl: 3600

Post-Processing

Transform configuration content before parsing:

const postProcessor = (content: string): string => {
    return content
        .replace(/#{service-name}/g, 'user-service')
        .replace(/#{environment}/g, process.env.NODE_ENV || 'development');
};

// The 4th argument transforms the LOGGER file, the 5th the APPLICATION file.
// Pass both when the same substitution applies to each.
const config = await loadConfig('local', 'app.yaml', 'logger.yaml', postProcessor, postProcessor);

// Logger file only - the application config is parsed as-is
const loggerOnly = await loadConfig('local', 'app.yaml', 'logger.yaml', postProcessor);

Before 1.1.0 there was no 5th parameter: the transform reached logger.yaml only, while this README's own example implied it applied to the application config. If you relied on the old four-argument form, add the 5th argument to get the behaviour the example described.

Logging

The loader writes through @ticatec/logger-api, a zero-dependency contract rather than a concrete logging library.

This package runs at the very front of startup - it is what produces the logger's own configuration - so its first records are usually written before setLoggerProvider() has been called. That is expected: they fall back to the console, filtered by LOG_LEVEL, and the logger proxy picks up the real provider as soon as one is registered.

| Event | Level | Payload | |---|---|---| | loadConfig() entry | debug | mode, config file, logger file | | Loader selected | debug | mode | | Local config directory resolved | debug | absolute root | | Connecting to Consul / Nacos | debug | target, and whether auth / TLS is configured | | Configuration loaded | debug | file, source, top-level key count, elapsed ms | | Resolving an include | debug | parent file, included file, mount key | | Optional include not found, skipping | info | parent file, included file | | Source returned no content | warn | file, source | | Consul key / Nacos data ID missing | warn | key or data ID | | Path escapes the config directory | warn | requested path, root |

Configuration values are never logged. A config file is exactly where database passwords and API keys live - the sample config/db.yaml in this repository has a password: line. Records carry file names, the include graph, key counts and timings; never parsed content. CONSUL_TOKEN is likewise reported only as authenticated: true.

DEBUG [ConfigLoader] Loading application and logger configuration {"mode":"local","configFile":"app.yaml","logFile":"logger.yaml"}
DEBUG [ConfigLoader] Selecting configuration loader {"mode":"local"}
DEBUG [LocalFileLoader] Resolved local configuration directory {"root":"/srv/app/config"}
DEBUG [ConfigLoader] Resolving include {"parent":"app.yaml","include":"db.yaml","key":"database"}
DEBUG [ConfigLoader] Configuration loaded {"file":"app.yaml","source":"LocalFileLoader","keys":3,"ms":4}

One summary record is written per load() call, plus one trace per include - not one summary per file, which would make a deep include tree unreadable.

To route the records into a real logger, register a provider once the logger configuration has been loaded:

import { loadConfig } from '@ticatec/config-loader';
import { setLoggerProvider } from '@ticatec/logger-api';
import { initialize, getPinoLogger } from '@ticatec/logger-pino';

const { appConf, loggerConf } = await loadConfig('local', 'app.yaml', 'logger.yaml');
initialize(loggerConf);
setLoggerProvider(getPinoLogger);

API Reference

Core Functions

loadConfig(configMode, configFile, logFile, loggerPostLoader?, appPostLoader?, loaderOptions?)

Load both application and logger configurations.

• configMode - Configuration source type: 'local', 'consul', or 'nacos'
• configFile - Path/key to the main configuration file
• logFile - Path/key to the logger configuration file
• loggerPostLoader - Optional post-processing function for logger config
• appPostLoader - Optional post-processing function for application config
• loaderOptions - Optional configuration options passed to the underlying loader (e.g. ConsulOptions, local root directory)

Returns: Promise<{appConf: any, loggerConf: any}>

getLoader(type, options?)

Factory function to create configuration loaders.

• type - Loader type: 'local', 'consul', or 'nacos'
• options - Optional configuration options passed to the underlying loader

Returns: Promise<BaseLoader>

BaseLoader Class

Abstract base class for all configuration loaders.

Methods

• load(fileName, postLoader?) - Load configuration with includes support
• loadConfig(fileName, postLoader?) - Load and parse single configuration file
• deepMerge(obj1, obj2) - Deep merge two configuration objects

ConfigNotFoundError Class

Exported error class thrown when a configuration file or remote KV key does not exist (code = 'CONFIG_NOT_FOUND').

Properties

• code: 'CONFIG_NOT_FOUND'
• file: string - The configuration file name or KV key that was not found
• loader?: string - The name of the loader that attempted to read the file (e.g. 'LocalFileLoader', 'ConsulLoader')
• cause?: unknown - The underlying error if any (e.g. Node's ENOENT error)

Error Handling Example

Callers can use err instanceof ConfigNotFoundError or duck-typing err.code === 'CONFIG_NOT_FOUND' to differentiate between "configuration does not exist" (e.g. to apply fallback defaults) and "configuration is malformed or broken" (e.g. YAML parse error, network failure, ACL permission denied):

import { loadConfig, ConfigNotFoundError } from '@ticatec/config-loader';

try {
    const { appConf, loggerConf } = await loadConfig('local', 'app.yaml', 'logger.yaml');
    // use appConf ...
} catch (err) {
    if (err instanceof ConfigNotFoundError || (err as any)?.code === 'CONFIG_NOT_FOUND') {
        // Missing file or KV key: fall back to default configuration
        console.warn(`Configuration '${(err as ConfigNotFoundError).file}' not found, falling back to defaults`);
    } else {
        // Malformed YAML, permission denied, or service unavailable: abort startup
        console.error('Failed to load configuration due to an invalid file or service error:', err);
        process.exit(1);
    }
}

Examples

Basic Local Configuration

import { getLoader } from '@ticatec/config-loader';

const loader = await getLoader('local');
const config = await loader.load('app.yaml');
console.log(config);

Consul with Authentication

import { loadConfig } from '@ticatec/config-loader';

process.env.CONSUL_HOST = 'consul.example.com';
process.env.CONSUL_PORT = '8500';
process.env.CONSUL_TOKEN = 'secret-token';
process.env.CONSUL_HTTP_SSL = 'true';

const config = await loadConfig('consul', 'apps/myapp/config', 'apps/myapp/logging', null);

Complex Include Structure

# app.yaml
app:
  name: E-commerce API
  version: 2.1.0

includes:
  - file: database/postgres.yaml
    key: database
    params:
      ssl: true
  - file: services/redis.yaml  
    key: cache
  - file: services/elasticsearch.yaml
    key: search
    params:
      index_prefix: ecommerce_v2

Extending the Library

The configuration loader is designed to be easily extensible. You can create custom loaders for any configuration source by extending the BaseLoader class. Here are practical examples using popular configuration centers.

Creating Custom Loaders

Example 1: etcd Configuration Loader

import BaseLoader from '@ticatec/config-loader';
import { Etcd3 } from 'etcd3';

export default class EtcdLoader extends BaseLoader {
    private client: Etcd3;

    constructor() {
        super();
        this.client = new Etcd3({
            hosts: process.env['ETCD_HOSTS']?.split(',') || ['http://localhost:2379'],
            auth: {
                username: process.env['ETCD_USERNAME'],
                password: process.env['ETCD_PASSWORD']
            }
        });
    }

    /**
     * Load configuration from etcd key-value store
     * @param fileName - The etcd key path
     * @returns Promise that resolves to the configuration content
     * @protected
     */
    protected async loadFile(fileName: string): Promise<string> {
        try {
            const value = await this.client.get(fileName);
            if (!value) {
                throw new Error(`Configuration key '${fileName}' not found in etcd`);
            }
            return value.toString();
        } catch (error) {
            throw new Error(`Failed to load from etcd: ${error.message}`);
        }
    }
}

Usage:

import EtcdLoader from './loaders/EtcdLoader';

// Set environment variables
process.env.ETCD_HOSTS = 'http://etcd1:2379,http://etcd2:2379,http://etcd3:2379';
process.env.ETCD_USERNAME = 'config_user';
process.env.ETCD_PASSWORD = 'config_pass';

const loader = new EtcdLoader();
const config = await loader.load('/myapp/config/production.yaml');

Example 2: Eureka Configuration Loader

import BaseLoader from '@ticatec/config-loader';
import axios from 'axios';

export default class EurekaLoader extends BaseLoader {
    private eurekaUrl: string;
    private appName: string;

    constructor() {
        super();
        this.eurekaUrl = process.env['EUREKA_URL'] || 'http://localhost:8761/eureka';
        this.appName = process.env['EUREKA_APP_NAME'] || 'config-service';
    }

    /**
     * Load configuration from Eureka service instance
     * @param fileName - The configuration file path on the service
     * @returns Promise that resolves to the configuration content
     * @protected
     */
    protected async loadFile(fileName: string): Promise<string> {
        try {
            // Get service instances from Eureka
            const instanceUrl = await this.getServiceInstanceUrl();
            
            // Fetch configuration from service instance
            const response = await axios.get(`${instanceUrl}/config/${fileName}`, {
                timeout: 5000,
                headers: {
                    'Accept': 'application/x-yaml'
                }
            });

            return response.data;
        } catch (error) {
            throw new Error(`Failed to load from Eureka service: ${error.message}`);
        }
    }

    private async getServiceInstanceUrl(): Promise<string> {
        const response = await axios.get(`${this.eurekaUrl}/apps/${this.appName}`, {
            headers: { 'Accept': 'application/json' }
        });

        const instances = response.data.application?.instance;
        if (!instances || instances.length === 0) {
            throw new Error(`No instances found for service: ${this.appName}`);
        }

        // Use the first available instance
        const instance = Array.isArray(instances) ? instances[0] : instances;
        const protocol = instance.securePort?.enabled ? 'https' : 'http';
        const port = instance.securePort?.enabled ? instance.securePort['$'] : instance.port['$'];
        
        return `${protocol}://${instance.hostName}:${port}`;
    }
}

Usage:

import EurekaLoader from './loaders/EurekaLoader';

// Set environment variables
process.env.EUREKA_URL = 'http://eureka-server:8761/eureka';
process.env.EUREKA_APP_NAME = 'config-service';

const loader = new EurekaLoader();
const config = await loader.load('application.yaml');

Advanced Integration Patterns

Pattern 1: Factory with Custom Loaders

import { BaseLoader, LocalFileLoader, getLoader } from '@ticatec/config-loader';
import EtcdLoader from './loaders/EtcdLoader';
import EurekaLoader from './loaders/EurekaLoader';

export const getAdvancedLoader = async (type: string): Promise<BaseLoader> => {
    switch (type) {
        case 'etcd':
            return new EtcdLoader();
        case 'eureka':
            return new EurekaLoader();
        case 'local':
            return new LocalFileLoader();
        default:
            // getLoader imports the Consul / Nacos loader dynamically, so their
            // optional peers are only required when that source is actually used.
            return await getLoader(type);
    }
};

The built-in loaders are reached through the package entry point: LocalFileLoader is exported directly, and ConsulLoader / NacosConfigLoader through getLoader('consul' | 'nacos'). There are no deep import paths - the exports map intentionally publishes only the entry point, so a path such as @ticatec/config-loader/dist/... would fail with ERR_PACKAGE_PATH_NOT_EXPORTED (and there is no dist/ directory either - the builds land in lib/cjs and lib/esm).

Pattern 2: Multi-Source Configuration

import { PostLoader } from '@ticatec/config-loader';
import { getAdvancedLoader } from './advanced-loaders';

export const loadMultiSourceConfig = async (
    sources: Array<{ type: string; configFile: string; key?: string }>,
    postLoader?: PostLoader
): Promise<any> => {
    let finalConfig = {};

    for (const source of sources) {
        const loader = await getAdvancedLoader(source.type);
        const config = await loader.load(source.configFile, postLoader);
        
        if (source.key) {
            finalConfig[source.key] = config;
        } else {
            // Deep merge if no specific key
            finalConfig = { ...finalConfig, ...config };
        }
    }

    return finalConfig;
};

// Usage example
const config = await loadMultiSourceConfig([
    { type: 'etcd', configFile: '/app/database.yaml', key: 'database' },
    { type: 'eureka', configFile: 'services.yaml', key: 'services' },
    { type: 'local', configFile: 'app.yaml' }
]);

Environment Configuration

For etcd Loader

# etcd Configuration
ETCD_HOSTS=http://etcd1:2379,http://etcd2:2379,http://etcd3:2379
ETCD_USERNAME=config_user
ETCD_PASSWORD=config_secret
ETCD_PREFIX=/myapp/config

For Eureka Loader

# Eureka Configuration  
EUREKA_URL=http://eureka-server:8761/eureka
EUREKA_APP_NAME=config-service
EUREKA_TIMEOUT=5000
EUREKA_RETRY_COUNT=3

Testing Custom Loaders

// etcd-loader.test.ts
import EtcdLoader from '../loaders/EtcdLoader';

describe('EtcdLoader', () => {
    let loader: EtcdLoader;

    beforeEach(() => {
        process.env.ETCD_HOSTS = 'http://localhost:2379';
        loader = new EtcdLoader();
    });

    it('should load configuration from etcd', async () => {
        // Mock etcd client or use test container
        const config = await loader.load('/test/config.yaml');
        expect(config).toBeDefined();
    });
});

// eureka-loader.test.ts
import EurekaLoader from '../loaders/EurekaLoader';
import nock from 'nock';

describe('EurekaLoader', () => {
    beforeEach(() => {
        process.env.EUREKA_URL = 'http://localhost:8761/eureka';
        process.env.EUREKA_APP_NAME = 'config-service';
    });

    it('should load configuration from Eureka service', async () => {
        // Mock Eureka API responses
        nock('http://localhost:8761')
            .get('/eureka/apps/config-service')
            .reply(200, {
                application: {
                    instance: {
                        hostName: 'config-service-host',
                        port: { '$': '8080' },
                        securePort: { enabled: false }
                    }
                }
            });

        nock('http://config-service-host:8080')
            .get('/config/app.yaml')
            .reply(200, 'app:\n  name: test\n  version: 1.0');

        const loader = new EurekaLoader();
        const config = await loader.load('app.yaml');
        
        expect(config.app.name).toBe('test');
    });
});

Best Practices for Custom Loaders

1. Error Handling: Always wrap external service calls in try-catch blocks
2. Timeouts: Set appropriate timeouts for network requests
3. Caching: Consider implementing caching for frequently accessed configurations
4. Authentication: Handle authentication tokens and credentials securely
5. Connection Pooling: Reuse connections when possible
6. Health Checks: Implement health checks for external services

// Example with error handling and caching
export default class RobustEtcdLoader extends BaseLoader {
    private client: Etcd3;
    private cache = new Map<string, { content: string; timestamp: number }>();
    private cacheTTL = 60000; // 1 minute

    protected async loadFile(fileName: string): Promise<string> {
        // Check cache first
        const cached = this.cache.get(fileName);
        if (cached && (Date.now() - cached.timestamp) < this.cacheTTL) {
            return cached.content;
        }

        let retries = 3;
        while (retries > 0) {
            try {
                const value = await this.client.get(fileName);
                if (value) {
                    const content = value.toString();
                    // Cache successful result
                    this.cache.set(fileName, { content, timestamp: Date.now() });
                    return content;
                }
            } catch (error) {
                retries--;
                if (retries === 0) throw error;
                await new Promise(resolve => setTimeout(resolve, 1000));
            }
        }
        
        throw new Error(`Configuration '${fileName}' not found after retries`);
    }
}

Contributing

This package lives in the Keelson monorepo. Issues and pull requests are welcome there.

Development Setup

git clone https://github.com/ticatec/keelson.git
cd keelson
pnpm install
cd packages/config-loader

pnpm build       # builds both CJS and ESM (lints first)
pnpm test
pnpm typecheck
pnpm lint

From the monorepo root, pnpm verify builds, type-checks and tests every package.

License

Copyright © 2023 Ticatec. All rights reserved.

This library is released under the MIT License. See the LICENSE file for details.

Support

• 📧 Email: [email protected]
• 📦 Source: github.com/ticatec/keelson/tree/main/packages/config-loader
• 🐛 Issues: github.com/ticatec/keelson/issues

Changelog

See CHANGELOG.md for version history and changes.