NAME

    qtools-library-dot-d is system for creating dynamic, modular libraries in Node.js. The dotD (dot-D) pattern provides a powerful way to build extensible applications through automatic module discovery and loading.

What is qtools-library-dot-d?

qtools-library-dot-d is a Node.js library that enables dynamic module loading from directories. It automatically discovers JavaScript files in specified directories and makes them available as methods on a library object, creating a plugin-like architecture without the complexity of traditional plugin systems.

The dotD Pattern

The "dotD" naming convention (short for "dot-D") refers to directories that contain dynamically loadable modules:

• modules.d/ - Directory containing module files
• endpoints.d/ - Directory containing API endpoint modules
• accessPoints.d/ - Directory containing data access modules
• mappers.d/ - Directory containing data mapping modules

Each module, .js file or directory with package.json, in a dotD directory becomes a method on the library object, with the filename determining the method name.

Why Use Dynamic dotD Libraries?

1. Automatic Module Discovery

No need to manually import or register modules. Simply add a .js file to the dotD directory and it's automatically available.

// Traditional approach - manual imports
const module1 = require('./modules/module1');
const module2 = require('./modules/module2');
const module3 = require('./modules/module3');

// dotD approach - automatic discovery
const myLibrary = qtLibDotD({ libraryName: 'myLibrary' }); //libraryName is used as a label in reports
myLibrary.setLibraryPath('./modules.d');
myLibrary.loadModules();
// All modules in modules.d/ are now available as myLibrary.moduleName()

2. Plugin Architecture

Create extensible applications where new functionality can be added by simply dropping files into directories.

// Add new functionality by creating a new file
// modules.d/newFeature.js -> myLibrary.newFeature()

3. Shared Resources

Pass common resources (configuration, database connections, utilities) to all modules through passThroughParameters. These are given, by that name, to the module (along with the 'dotD', the library object) when it is instantiated during the loadModules() process.

const passThroughParameters = {
    config: getConfig(),
    database: dbConnection,
    logger: xLog,
    utilities: commonUtils
};
myLibrary.loadModules({ passThroughParameters });

4. Clean Separation of Concerns

Each module is self-contained and focused on a single responsibility, making code easier to maintain and test.

5. Runtime Flexibility

Modules can be loaded synchronously or asynchronously, and the library can be sealed to prevent modifications after loading.

Real-World Use Cases

API Endpoint Management

// endpoints.d/userEndpoints.js
// endpoints.d/productEndpoints.js  
// endpoints.d/orderEndpoints.js
// Each file defines related API endpoints

Data Access Layer

// accessPoints.d/getUserData.js
// accessPoints.d/saveUserData.js
// accessPoints.d/deleteUserData.js
// Each file handles specific data operations

Business Logic Modules

// processors.d/validateOrder.js
// processors.d/calculateTax.js
// processors.d/sendNotification.js
// Each file contains specific business logic

CLI Command Tools

// commands.d/deploy.js
// commands.d/backup.js
// commands.d/migrate.js
// Each file implements a CLI command

How It Works

1. Library Creation

const qtLibDotD = require('qtools-library-dot-d');
const myLibrary = qtLibDotD({ libraryName: 'myLibrary' });

2. Module Directory Setup

myLibrary.setLibraryPath('./modules.d');

3. Shared Resource Definition

const passThroughParameters = {
    config: appConfig,
    database: dbConnection,
    logger: logger
}; //provided by loadModules() in the initialization object

4. Module Loading

// Synchronous
myLibrary.loadModules({ passThroughParameters });

// Asynchronous
myLibrary.loadModules({ passThroughParameters }, callback);

5. Library Sealing

myLibrary.seal(); // Prevents further modifications

6. Module Usage

// If you have modules.d/processOrder.js
myLibrary.processOrder(orderData);

// If you have modules.d/sendEmail.js  
myLibrary.sendEmail(recipient, message);

Module Structure

Every dotD module follows this pattern:

const moduleName = __filename.replace(__dirname + '/', '').replace(/.js$/, '');

const moduleFunction = ({ moduleName } = {}) => ({ dotD, passThroughParameters } = {}) => {

    const workingFunction = (/* your parameters */) => {
        // Access shared resources
        const config = passThroughParameters.config;
        const logger = passThroughParameters.logger;

        // Your module logic here

        return result;
    };

    // Register the function
    dotD.library.add(moduleName, workingFunction);
};

module.exports = moduleFunction({ moduleName });

Benefits Over Traditional Approaches

| Traditional | dotD Libraries | | --------------------- | --------------------------- | | Manual imports | Automatic discovery | | Tight coupling | Loose coupling | | Hard to extend | Easy to extend | | Configuration in code | Configuration through files | | Monolithic | Modular | | Static structure | Dynamic structure |

Getting Started

1. Install the dependency:

   npm install qtools-library-dot-d

2. Create your own modules:

   • Add .js files to demoModules_DotD/
   • Follow the module pattern shown in the examples
   • Access shared resources via passThroughParameters

Advanced Features

• Async module loading for modules that need initialization
• Error handling during module loading
• Library introspection with toString() and qtdProcessLog()
• Multiple libraries in the same application
• Nested module structures for complex applications

Integration with Larger Systems

The dotD pattern works particularly well in:

• Web frameworks (Express, Koa) for route organization
• API systems for endpoint management
• CLI tools for command organization
• Microservices for feature modularity
• Plugin systems for extensible applications

This pattern is used extensively in the qtools ecosystem for building scalable, maintainable Node.js applications.

HISTORY

v1.12 - clarified error message "Ignoring invalid file". v1.10 - there was a reason I did not return the exception list. Undone. v1.09 - fixed broken error handling when library file path is bad; added demonstration application v1.08 - updated README (several times) v1.05 - cleaned up toString() report, returns exception list (or err if callback) if needed, added qtools-functional-library v1.04 - Now tolerates (and reports) directories and files that are not modules.