Install the Course Environment
The code examples require an installation of Python 3.10 or later and packages like
numpy, pandas, Gymnasium, gym-classics2. See the repository’s environment.yml
for a complete list.
Deep reinforcement learning examples requires PyTorch, Stable-Baselines3.
The setup depends on your IDE.
Google Colab
Colab runtimes are temporary. Colab has many packages already preinstalled. You need to install the additional course-specific dependencies once at the beginning of each new session by running a notebook cell with:
%pip install "gymnasium[box2d,classic-control,other]>=1.0,<2"
%pip install pyvirtualdisplay
%pip install gym-classics2
To install the latest development version gym-classics2 of run a notebook cell with:
%pip install -q "gym-classics2 @ git+https://github.com/mhahsler/gym-classics2.git"
Restart the Colab runtime if prompted after installation.
You can test this using the following example notebook
Local Installation with Conda
This is the suggested method for Windows. Here are the steps:
- Install a Conda distribution such as Miniconda.
- Download the repository’s
environment.ymlfile. - Create and activate the environment from a terminal (e.g., a terminal in VS Code) in the
directory you downloaded
environment.ymlto.
conda env create --file environment.yml
conda activate reinforcement-learning
In VS Code, create or open a Notebook and use Python: Select Interpreter or
Notebook: Select Notebook Kernel to choose the reinforcement-learning environment.
Installation Without Conda
Conda is preferred because it also manages SWIG and FFmpeg. If Conda is not available, use Python 3.12, create a virtual environment and use pip for installation:
python3 -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade pip
python -m pip install jupyterlab ipykernel numpy pandas scipy matplotlib tqdm
python -m pip install "gymnasium[box2d,classic-control,other]>=1.0,<2" pyvirtualdisplay
python -m pip install gym-classics2
python -m pip install "stable-baselines3[extra]>=2.4,<3"
On Windows PowerShell, activate the environment with
.venv\Scripts\Activate.ps1. A non-Conda installation
may require system installations of SWIG, FFmpeg, and (on Linux/WSL) Xvfb
(see below).
Video Capture on Linux and WSL
The Conda environment supplies FFmpeg and the Python visualization packages.
Headless recording with pyvirtualdisplay also requires the X virtual
framebuffer on Linux or WSL. In the terminal run:
sudo apt-get update
sudo apt-get install xvfb
These operating-system packages are not needed to run non-rendering examples.
Updating gym-classics2 for Hotfixes
Conda can update packages:
conda activate reinforcement-learning
conda update --all
Pip can do this as well.
Some fixes for gym-classics2 may not be already included in the PyPI release version. You can
install the development version from Github via the (make sure to activate your virtual environment first):
python -m pip install "gym-classics2 @ git+https://github.com/mhahsler/gym-classics2.git"
Or run the following code in a notebook cell:
%pip install "gym-classics2 @ git+https://github.com/mhahsler/gym-classics2.git"
Troubleshooting
- If a notebook reports a missing package after installation, confirm that its
selected kernel is
reinforcement-learning. - If video capture reports that no display is available on Linux or WSL, install Xvfb as shown above and restart the kernel.
- Keep dependency names containing extras in quotes, for example
"gymnasium[box2d]", so shells do not interpret the brackets.
License
© 2026 Michael Hahsler. All code and documents in this repository are provided under the Creative Commons Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) License.
