View on GitHub

Introduction to Reinforcement Learning

Install the Course Environment

The code examples require an installation of Python 3.10 or later and packages like numpy, pandas, Gymnasium, gym-classics2. See the repository’s environment.yml for a complete list.

Deep reinforcement learning examples requires PyTorch, Stable-Baselines3.

The setup depends on your IDE.

Google Colab

Colab runtimes are temporary. Colab has many packages already preinstalled. You need to install the additional course-specific dependencies once at the beginning of each new session by running a notebook cell with:

%pip install "gymnasium[box2d,classic-control,other]>=1.0,<2"
%pip install pyvirtualdisplay
%pip install gym-classics2

To install the latest development version gym-classics2 of run a notebook cell with:

%pip install -q "gym-classics2 @ git+https://github.com/mhahsler/gym-classics2.git"

Restart the Colab runtime if prompted after installation.

You can test this using the following example notebook

Local Installation with Conda

This is the suggested method for Windows. Here are the steps:

  1. Install a Conda distribution such as Miniconda.
  2. Download the repository’s environment.yml file.
  3. Create and activate the environment from a terminal (e.g., a terminal in VS Code) in the directory you downloaded environment.yml to.
  conda env create --file environment.yml
  conda activate reinforcement-learning

In VS Code, create or open a Notebook and use Python: Select Interpreter or Notebook: Select Notebook Kernel to choose the reinforcement-learning environment.

Installation Without Conda

Conda is preferred because it also manages SWIG and FFmpeg. If Conda is not available, use Python 3.12, create a virtual environment and use pip for installation:

python3 -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade pip
python -m pip install jupyterlab ipykernel numpy pandas scipy matplotlib tqdm
python -m pip install "gymnasium[box2d,classic-control,other]>=1.0,<2" pyvirtualdisplay
python -m pip install gym-classics2
python -m pip install "stable-baselines3[extra]>=2.4,<3"

On Windows PowerShell, activate the environment with .venv\Scripts\Activate.ps1. A non-Conda installation may require system installations of SWIG, FFmpeg, and (on Linux/WSL) Xvfb (see below).

Video Capture on Linux and WSL

The Conda environment supplies FFmpeg and the Python visualization packages. Headless recording with pyvirtualdisplay also requires the X virtual framebuffer on Linux or WSL. In the terminal run:

sudo apt-get update
sudo apt-get install xvfb

These operating-system packages are not needed to run non-rendering examples.

Updating gym-classics2 for Hotfixes

Conda can update packages:

conda activate reinforcement-learning
conda update --all

Pip can do this as well.

Some fixes for gym-classics2 may not be already included in the PyPI release version. You can install the development version from Github via the (make sure to activate your virtual environment first):

python -m pip install "gym-classics2 @ git+https://github.com/mhahsler/gym-classics2.git"

Or run the following code in a notebook cell:

%pip install "gym-classics2 @ git+https://github.com/mhahsler/gym-classics2.git"

Troubleshooting

License

© 2026 Michael Hahsler. All code and documents in this repository are provided under the Creative Commons Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) License.

CC BY-SA 4.0