An illustrated reinforcement-learning agent navigating a gridworld toward a glowing goal

I released version 1.0.2 of gym-classics2, a Python package designed to make classic reinforcement learning easy to inspect, teach, and experiment with. It is based on Brett Daley’s gym-classics and uses the modern Gymnasium API.

The package includes:

  • finite Markov decision processes such as random walks, gridworlds, mazes, cliff walking, four rooms, and windy gridworld;
  • direct access to transition and reward models for planning algorithms;
  • readable implementations of dynamic programming, Monte Carlo, temporal-difference, function-approximation, eligibility-trace, and policy-gradient methods; and
  • plotting and animation helpers for exploring learned values, policies, and agent behavior.

The implementations follow the presentation in Sutton and Barto’s Reinforcement Learning: An Introduction. They emphasize clear code and inspectable intermediate results, making the package especially useful for coursework, demonstrations, and small experiments.

Install the package directly from GitHub:

python -m pip install "gym-classics2 @ git+https://github.com/mhahsler/gym-classics2.git"

See the documentation for the environment catalog, tutorials, and API reference. Companion slides, examples, and exercises are available in my Introduction to Reinforcement Learning course.