An illustrated reinforcement-learning agent navigating a gridworld toward a glowing goal

I released gym-classics2 version 1.0.3 on PIPy. gym-classics2 is a Python package designed to make classic reinforcement learning easy to learn, experiment, and teach with. Inspired by Brett Daley’s gym-classics, it uses the popular Gymnasium API, and adds model access and learning algorithms.

The package includes:

  • finite Markov decision processes such as random walks, gridworlds, mazes, cliff walking, four rooms, and windy gridworld;
  • direct access to transition and reward models for planning algorithms;
  • readable implementations of dynamic programming, Monte Carlo, temporal-difference, function-approximation, eligibility-trace, and policy-gradient methods; and
  • plotting and animation helpers for exploring learned values, policies, and agent behavior.

The implementations follow the presentation in Sutton and Barto’s Reinforcement Learning: An Introduction. They emphasize clear code and inspectable intermediate results, making the package especially useful for coursework, demonstrations, and small experiments.

Install the package from PiPy:

python -m pip install gym-classics2

See the documentation for the supported environments, tutorials, and API reference. Companion slides, examples, and exercises are available in my Introduction to Reinforcement Learning course.