🧬 Gene LSTM

A TypeScript implementation of evolutionary LSTM neural networks using genetic algorithms. This library combines Long Short-Term Memory (LSTM) networks with neuroevolution techniques to create adaptive, self-optimizing neural networks for sequence learning tasks.

🎮 Live Demo

Try the interactive demo: https://leoni4.github.io/gene-lstm-js/

The demo showcases Gene LSTM solving various problems including LastBit, parity functions, and classification tasks with real-time visualization of the neural network evolution.

Features

• Neuroevolution: Evolves LSTM architectures and weights through genetic algorithms
• Speciation: Maintains diversity through automatic species clustering
• Dynamic Mutation Pressure: Automatically adjusts mutation intensity based on fitness stagnation
• Adaptive Complexity: Balances network complexity with performance
• Sleeping Block Initialization: Smart initialization strategy for stable training
• TypeScript: Full type safety and modern ES modules

Installation

npm install @leoni4/gene-lstm-js

Usage

Quick Start with fit() Method

The fit() method provides a simple, Keras-style API for training Gene LSTM networks:

import { GeneLSTM } from '@leoni4/gene-lstm-js';

// Create a GeneLSTM instance with 300 clients
const glstm = new GeneLSTM(300);

// Training data (lastBit example - predict the last bit in a sequence)
const lastBit = {
    inputs: [
        [0, 0, 0, 0],
        [0, 0, 0, 1],
        [0, 0, 1, 0],
        [0, 0, 1, 1],
        [0, 1, 0, 0],
        [0, 1, 0, 1],
        [0, 1, 1, 0],
        [0, 1, 1, 1],
        [1, 0, 0, 0],
        [1, 0, 0, 1],
        [1, 0, 1, 0],
        [1, 0, 1, 1],
        [1, 1, 0, 0],
        [1, 1, 0, 1],
        [1, 1, 1, 0],
        [1, 1, 1, 1],
    ],
    outputs: [
        0, // last = 0
        1, // last = 1
        0,
        1,
        0,
        1,
        0,
        1,
        0,
        1,
        0,
        1,
        0,
        1,
        0,
        1,
    ],
};

// Train the network
const history = glstm.fit(lastBit.inputs, lastBit.outputs, {
    epochs: 1000,
    verbose: 2,
});

console.log('Training completed in:', history.epochs, 'epochs');
console.log('Final error:', history.error[history.error.length - 1]);

// Use the trained champion to make predictions
const prediction = history.champion!.calculate([1, 0, 1, 1]);
console.log('Prediction:', prediction);

Manual Library Usage

import { GeneLSTM } from '@leoni4/gene-lstm-js';

// Training data just random example
const trainingData = {
    inputs: [
        [0, 0.5, 0.25, 1],
        [1, 0.5, 0.25, 1],
    ],
    outputs: [0, 1],
};

// Create a population of 300 clients
const glstm = new GeneLSTM(300, {
    INPUT_FEATURES: 4, // Number of input features
    verbose: 1, // Logging level
});

// Training loop
for (let epoch = 0; epoch < 1000; epoch++) {
    // Evaluate each client
    for (const client of glstm.clients) {
        let errorSum = 0;

        for (let i = 0; i < trainingData.inputs.length; i++) {
            const output = client.calculate(trainingData.inputs[i]);
            const error = Math.abs(output[0] - trainingData.outputs[i]);
            errorSum += error;
        }

        const avgError = errorSum / trainingData.inputs.length;
        client.score = 1 - avgError; // Higher score is better
    }

    // Evolve population
    glstm.evolve();

    if (epoch % 100 === 0) {
        console.log(`Epoch ${epoch}`);
        glstm.printSpecies();
    }
}

// Use the best performing network (champion)
const champion = glstm.champion || glstm.clients[0];
const prediction = champion.calculate([0.5, 0.5, 0.25, 1]);
console.log('Prediction:', prediction);

Loading Pre-trained Models

import { GeneLSTM } from '@leoni4/gene-lstm-js';
import { PRE_TRAINED_DATA } from './my-trained-model.js';

const glstm = new GeneLSTM(1, {
    loadData: PRE_TRAINED_DATA,
});

const result = glstm.clients[0].calculate([0, 0.5, 0.25, 1]);
console.log('Result:', result);

Advanced Configuration

import { GeneLSTM, EMutationPressure } from '@leoni4/gene-lstm-js';

const glstm = new GeneLSTM(500, {
    // Input configuration
    INPUT_FEATURES: 10,

    // Species parameters
    CP: 0.15, // Compatibility threshold
    targetSpecies: 8, // Target number of species

    // Evolution parameters
    SURVIVORS: 0.7, // Survival rate (70%)
    MUTATION_RATE: 1.0,

    // Mutation pressure (adaptive)
    mutationPressure: EMutationPressure.NORMAL,
    enablePressureEscalation: true,
    stagnationThreshold: 20,

    // Topology mutations
    PROBABILITY_MUTATE_LSTM_BLOCK: 0.02,
    PROBABILITY_ADD_BLOCK_APPEND: 0.9,
    PROBABILITY_REMOVE_BLOCK: 0.15,

    // Weight mutations
    PROBABILITY_MUTATE_WEIGHT_SHIFT: 0.95,
    WEIGHT_SHIFT_STRENGTH: 0.3,

    // Logging
    verbose: 2,
});

Available Scripts

For development and testing:

# Run demo
npm run demo

# Start interactive demo with Vite
npm start

# Build the project
npm run build

# Run tests
npm test

# Run tests in watch mode
npm run test:watch

# Lint code
npm run lint

# Type checking
npm run typecheck

# Build demo for production
npm run build:demo

API

Core Classes

GeneLSTM

Main class for managing the evolutionary process.

Constructor:

new GeneLSTM(clients: number, options?: GeneLSTMOptions)

Key Methods:

• fit(xTrain: SeqInput[], yTrain: SeqInput, options?: IGlstmFitOptions): IGlstmFitHistory

  High-level training method with automatic evolution and error tracking. Provides a Keras-like API for training.

  Parameters:

  • xTrain: Array of input sequences (can be 1D or 2D arrays)
  • yTrain: Array of target outputs (can be 1D numbers or 2D arrays)
  • options: Training configuration object

  Options (IGlstmFitOptions):

  {
    epochs?: number;              // Maximum number of training epochs (default: Infinity)
    errorThreshold?: number;      // Stop when error below this value (default: 0.01)
    validationSplit?: number;     // Fraction of data for validation (default: 0)
    verbose?: 0 | 1 | 2;         // Logging level (default: 1)
                                  // 0 = silent, 1 = periodic, 2 = detailed
    logInterval?: number;         // Log every N epochs when verbose=1 (default: 100)
  
    loss?: 'mae' | 'mse' | 'bce'; // Loss function (default: 'mae')
                                   // mae = Mean Absolute Error
                                   // mse = Mean Squared Error
                                   // bce = Binary Cross Entropy
  
    antiConstantPenalty?: boolean;     // Penalize constant predictions (default: false)
    antiConstantLambda?: number;       // Penalty strength (default: 0.05)
    shuffleEachEpoch?: boolean;        // Shuffle training data (default: true)
  }

  Returns (IGlstmFitHistory):

  {
    error: number[];              // Training error per epoch
    validationError?: number[];   // Validation error per epoch (if validationSplit > 0)
    epochs: number;               // Total epochs completed
    champion: Client | null;      // Best trained network
    stoppedEarly: boolean;        // True if stopped due to errorThreshold
  }

  Example:

  const history = glstm.fit(xTrain, yTrain, {
      epochs: 1000,
      errorThreshold: 0.01,
      validationSplit: 0.2,
      verbose: 2,
      loss: 'mae',
      antiConstantPenalty: true,
  });
  
  console.log('Final error:', history.error[history.error.length - 1]);
  const prediction = history.champion!.calculate([1, 0, 1]);

• evolve(optimization?: boolean) - Evolve the population for one generation

• printSpecies() - Print current species statistics

• adjustCP(speciesCount: number, generation?: number) - Dynamically adjust compatibility parameter

• updateMutationPressure(currentBestFitness: number, generation?: number) - Update mutation pressure based on progress

• model() - Export the best model's architecture

Properties:

• clients: Client[] - All clients in the population
• champion: Client | null - Best performing client ever seen
• mutationPressure: EMutationPressure - Current mutation pressure level

Client

Represents an individual neural network in the population.

Key Methods:

• calculate(input: SeqInput): number[] - Forward pass through the network
• mutate(force?: boolean) - Mutate the client's genome
• distance(client: Client): number - Calculate genetic distance to another client

Properties:

• genome: Genome - The LSTM architecture and weights
• score: number - Fitness score (0-1)
• species: Species | null - Species membership

EMutationPressure

Enum for mutation pressure levels:

• COMPACT - Minimal mutations, favor simplicity
• NORMAL - Balanced mutation rate
• BOOST - Increased mutations to escape local optima
• ESCAPE - High mutation rate for exploration
• PANIC - Maximum mutations when severely stuck

Data Structures

SeqInput

Input format for LSTM calculations:

type SeqInput = number[] | number[][];

• Scalar mode: number[] - Single input vector
• Vector mode: number[][] - Sequence of input vectors

GeneLSTMOptions

Configuration object for GeneLSTM initialization. See detailed options documentation for all available parameters.

GeneOptions

Pre-trained model data format:

type GeneOptions = LstmOptions[];

Export model data using:

const modelData = glstm.model();

Contributing

Contributions are welcome! Please follow these guidelines:

1. Fork the repository
2. Create a feature branch: git checkout -b feature/my-feature
3. Commit your changes following Conventional Commits:
   • feat: add new feature
   • fix: resolve bug
   • docs: update documentation
   • test: add tests
4. Test your changes: npm test
5. Lint your code: npm run lint
6. Push to your fork: git push origin feature/my-feature
7. Submit a pull request to the main branch

Development Setup

# Clone the repository
git clone https://github.com/leoni4/gene-lstm-js.git
cd gene-lstm-js

# Install dependencies
npm install

# Run tests
npm test

# Start development demo
npm start

License

MIT © Leonid Lilo

See LICENSE for details.

Repository

GitHub: leoni4/gene-lstm-js

Issues: Report bugs or request features

NPM: @leoni4/gene-lstm-js

Further Documentation

• Detailed Options Reference - Complete guide to all configuration options
• Examples - More usage examples and problem implementations