Neuroevolution

A modular Python framework implementing Artificial Neural Networks (ANN) trained using Genetic Algorithms (GA via PyMOO) to solve OpenAI Gymnasium control and navigation environments without traditional gradient-based backpropagation.

Solution Demos

CartPole (CartPole-v1)

Pole balancing control with continuous action adjustments.

MountainCar (MountainCar-v0)

Building momentum back and forth to climb a steep hill.

Maze (HardMaze-v0)

Complex robot navigation through a maze to reach the goal.

CarRacing (CarRacing-v3)

Full lap track completion and steering control (Reward 900+).

Key Features

• Gradient-Free Optimization: Evolves PyTorch neural network weights using genetic operators (selection, crossover, mutation) from PyMOO.
• Gymnasium Benchmarks: Out-of-the-box support for:
  • CartPole (CartPole-v1) - Balance a pole on a moving cart
  • MountainCar (MountainCar-v0) - Build momentum to drive up a steep hill
  • Maze (HardMaze-v0) - Complex robot maze navigation
  • CarRacing (CarRacing-v3) - Race around procedurally generated tracks
• High Performance: Multi-core parallel fitness evaluation across CPU cores using multiprocessing.
• Checkpointing & Warm Start: Ability to save, resume, and fine-tune evolutionary runs.
• Evaluation & Demo Recording: Built-in utilities to evaluate and record animated GIFs of trained policies.

Installation

# Clone the repository
git clone https://github.com/carloshkayser/neuroevolution.git
cd Neuroevolution

# Install dependencies using Poetry
poetry install

# Or install via pip
pip install -e .

Quickstart

Command Line Interface

# Train on CartPole
neuroevolution --train --env CartPole

# Train on CarRacing in parallel across 8 CPU cores
neuroevolution --train --env CarRacing --generations 35 --population 50 --n-workers 8

# Evaluate trained model
neuroevolution --test --env CarRacing

Python API

from neuroevolution import PyTorchGeneticTrainer

# Create trainer
trainer = PyTorchGeneticTrainer(
    env_name="MountainCar-v0",
    network_architecture=[16, 8],
    model_prefix="mountaincar_ga",
)

# Train with Genetic Algorithm
best_network, best_fitness, generations = trainer.train(
    n_generations=50,
    population_size=40,
    crossover_prob=0.9,
    mutation_prob=0.1,
)

# Evaluate the trained policy
avg_reward, std_reward = trainer.evaluate(n_episodes=10)
print(f"Average Reward: {avg_reward:.2f} +/- {std_reward:.2f}")