DOCSEARCHbyAlgolia
Submit Docs

neat-python

repository·master·Indexed 23 days ago

https://github.com/codereclaimers/neat-python

A pure-Python implementation of the NeuroEvolution of Augmenting Topologies (NEAT) algorithm used for evolving arbitrary neural networks. Version 2.1.0 is designed to be dependency-free, supporting reproducible experiments through random seeding and parallel evaluation across CPU cores. It includes capabilities for evolving controllers for environments like MuJoCo and Inverted Double Pendulum, as well as Continuous-Time Recurrent Neural Networks (CTRNN) for tasks such as Lorenz attractor prediction.

Tokens
56K
Snippets
90
Records
247
Agent score
81%

What's inside neat-python

  1. Explore XOR 'Hello World' examples

    master

    The XOR examples in the examples/xor/ directory serve as 'Hello World' style samples for neat-python. They demonstrate the minimal amount of code required to evolve networks that implement the 2-input XOR function. These examples are useful for learning the library's API or as a debugging tool to step through NEAT-specific logic without the complexity of larger applications.

    Available example types:

    • Minimal Feed-forward: evolve-minimal.py shows the absolute bare minimum for evolving a feed-forward network with sigmoidal neurons.
    • Standard Feed-forward: evolve-feedforward.py implements the same logic as the minimal version but uses better coding practices and provides prettier output.
    • Parallel Feed-forward: evolve-feedforward-parallel.py demonstrates how to utilize multiple processors to evaluate networks in parallel.
    • Spiking Neurons: evolve-spiking.py demonstrates evolving a network of spiking neurons using Izhikevich's neuron model.

  2. Use the iznn module for spiking neural networks

    master

    The iznn module implements spiking neural networks based on the Izhikevich (2003) model. It provides classes for simulating neurons and entire networks with specific spiking behaviors.

    Spiking Behavior Parameter Sets

    The module provides predefined parameter sets (a, b, c, d) for various behaviors:

    • REGULAR_SPIKING_PARAMS
    • INTRINSICALLY_BURSTING_PARAMS
    • CHATTERING_PARAMS
    • FAST_SPIKING_PARAMS
    • THALAMO_CORTICAL_PARAMS
    • RESONATOR_PARAMS
    • LOW_THRESHOLD_SPIKING_PARAMS

  3. Explore neat-python example use cases

    master

    The examples/ directory contains several scripts demonstrating different capabilities of the neat-python library. These range from basic logic problems to complex continuous control in Gymnasium environments.

    Basic & Logic

    • xor: A "hello world" sample for the 2-input XOR problem.
    • export: Demonstrates training an XOR network and exporting the resulting network to a framework-agnostic JSON format.

    Neural Models & Signal Processing

    • neuron-demo: Plots outputs for simple CTRNN and Izhikevich neuron models.
    • lorenz-ctrnn: Evolves CTRNN networks to predict the Lorenz attractor, utilizing per-node evolvable time constants (available in v2.0+).
    • signal-tracking-gpu: GPU-accelerated CTRNN signal tracking (requires neat-python[gpu]).
    • spike-timing-gpu: GPU-accelerated Izhikevich spiking network spike timing (requires neat-python[gpu]).

    Control & Reinforcement Learning (Gymnasium)

    • lunar-lander: Solves the LunarLander-v3 environment.
    • bipedal-walker: Controls the BipedalWalker-v3 environment using continuous-action policies.
    • inverted-double-pendulum: Evolves a controller for InvertedDoublePendulum-v5 using MuJoCo.
    • hopper: Evolves a controller for Hopper-v5 using MuJoCo.
    • single-pole-balancing: Balances a pole on a movable cart.

    Advanced Features

    • picture2d: Uses Compositional Pattern-Producing Networks (CPPN) to generate 2D images.
    • parallel-reproducible: Demonstrates using ParallelEvaluator with deterministic seeding for reproducible parallel evolution runs.

  5. How NEAT handles crossover and structural mutations

    master

    NEAT-Python uses specific mechanisms to manage structural changes and crossover between networks with different topologies:

    Crossover and Homology

    To perform crossover between networks of differing structures, NEAT tracks the origin of nodes using an identifying number (key).

    • Homologous genes: Nodes or connections derived from a common ancestor are matched up for crossover.
    • Disjoint/Excess genes: Nodes or connections that do not share a common ancestor are treated as non-homologous.

    Species and Fitness Sharing

    Structural mutations (adding nodes/connections) can be disruptive. To protect promising new structures while they fine-tune, NEAT uses speciation:

    • Genomic Distance: Measures similarity based on the number of non-homologous nodes/connections and the divergence of homologous ones.
    • Fitness Sharing: Individuals are grouped into species based on genomic distance. Competition is most intense within a species rather than between different species, allowing new structural innovations to survive long enough to be optimized.

  6. Understand how node outputs are calculated

    master

    In NEAT-Python, a node's output is determined by its specific attributes: activation function, bias, response, and aggregation function. The mathematical relationship is:

    activation(bias + (response * aggregation(inputs)))

    • activation function: Determines the final non-linear output (e.g., identity).
    • aggregation function: How the inputs are combined (see aggregations module).
    • bias: A constant value added to the aggregated input.
    • response: A multiplier applied to the aggregated input.

  7. Core NEAT concepts

    master

    Understanding the fundamental abstractions of NEAT:

    • Fitness Function: A user-defined function that measures how well a genome performs a specific task. NEAT uses this score to drive evolution.
    • Genome: The genetic encoding of a neural network, consisting of nodes, connections, and weights.
    • Species: Groups of similar genomes. Species protection ensures that innovative structural changes are not immediately lost due to low initial fitness.
    • Complexification: The process where networks start simple and gradually add complexity (nodes or connections) only when it provides a fitness advantage.

  8. How NEAT-Python evolves neural networks

    master

    NEAT (NeuroEvolution of Augmenting Topologies) is an evolutionary algorithm that evolves artificial neural networks by manipulating genomes.

    Core Components

    • Genomes: A collection of individuals in a population. Each genome consists of two types of genes:
      1. Node genes: Specify individual neurons.
      2. Connection genes: Specify connections between neurons.

    The Evolutionary Process

    1. Fitness Function: The user must provide a function that returns a single real number representing the quality of a genome. Higher scores indicate better performance.
    2. Generations: The algorithm runs for a user-specified number of generations. Each generation is created through the reproduction (sexual or asexual) and mutation of the most fit individuals from the previous generation.
    3. Complexity Growth: Mutations can add new nodes or connections, allowing the neural networks to increase in complexity over time.
    4. Termination: The algorithm stops when the preset number of generations is reached or when an individual meets a user-specified fitness threshold (depending on the configured fitness criterion).

  9. Identify Network-Specific Node Fields

    master

    Depending on the network_type, nodes may contain additional specialized fields:

    • CTRNN: Includes time_constant (number).
    • IZNN (Izhikevich Spiking Network): Includes a (time scale of recovery), b (sensitivity of recovery), c (after-spike reset of membrane potential), and d (after-spike reset of recovery).

  10. Understand parameter relationships and trade-offs

    master

    Tuning NEAT requires understanding how different parameters interact:

    • Population Size vs. Generations: Larger pop_size explores more of the search space but requires fewer generations to find a solution; smaller populations are faster per generation but require more generations.
    • Mutation Rates vs. Complexity: Higher conn_add_prob and node_add_prob increase network complexity faster. If networks become overly complex, reduce these values.
    • Compatibility Threshold vs. Species Count: A lower compatibility_threshold results in more species (higher diversity, slower convergence). A higher threshold results in fewer species (faster convergence, higher risk of premature convergence).
    • Survival Threshold vs. Selection Pressure: Lower values (e.g., 0.1) increase selection pressure (only elites reproduce). Higher values (e.g., 0.5) decrease selection pressure (more genomes can reproduce).
    • Feed Forward vs. Problem Type:
      • Use feed_forward = True for classification, function approximation, or stateless problems.
      • Use feed_forward = False for sequences, control with memory, or temporal problems.

  11. Understand CTRNN time constant limitations in neat-python

    master

    When using Continuous Recurrent Neural Networks (CTRNN) in neat-python, be aware that the implementation uses a single fixed time constant applied uniformly to all nodes in the network.

    This differs from other implementations (like the Julia NeatEvolution) that evolve independent time constants ($\tau_i$) for each node. Because all nodes share the same temporal response, the network cannot easily represent multiple timescales (e.g., fast oscillations vs. slow integration) simultaneously.

    If your task requires modeling complex dynamical systems with varying temporal scales, the fixed time constant may act as a performance bottleneck.

  12. How Continuous-Time Recurrent Neural Networks (CTRNN) are implemented

    master

    The CTRNN implementation in neat-python models the network as a system of ordinary differential equations where neuron potentials are the dependent variables.

    Time evolution is computed using the exponential Euler (ETD1) method. This method is unconditionally stable for the linear decay part, regardless of the ratio between the timestep $\Delta t$ and the time constant $\tau_i$. This is a significant improvement over the previous Forward Euler method, which required $\Delta t < 2 \tau_i$ to avoid divergent trajectories in nodes with small time constants.

    Key components of the model include:

    • $\tau_i$: Time constant of neuron $i$.
    • $y_i$: Potential of neuron $i$.
    • $f_i$: Activation function of neuron $i$.
    • $\beta_i$: Bias of neuron $i$.
    • $w_{ij}$: Weight of the connection from neuron $j$ to neuron $i$.