Setting Up the Environment
Why Environment Management Matters
Scientific Python code depends on dozens of packages with complex interdependencies — NumPy, SciPy, PyTorch, CuPy, Matplotlib, and more. A single version mismatch can produce silent numerical errors or outright crashes. Proper environment management is not overhead; it is the foundation on which reproducible research rests.
This section sets up the toolchain you will use for the rest of the book.
Definition: Virtual Environment
Virtual Environment
A virtual environment is an isolated directory tree containing a
Python interpreter and a set of installed packages. Activating a
virtual environment redirects python and pip to use this isolated
tree, so that packages installed in one project do not interfere with
another.
Python provides venv in the standard library:
python -m venv myenv
source myenv/bin/activate # macOS/Linux
myenv\Scripts\activate # Windows
Definition: Conda Environment
Conda Environment
Conda is a cross-platform package and environment manager that handles both Python packages and non-Python dependencies (C libraries, CUDA toolkit, MKL). A conda environment is created with:
conda create -n myproject python=3.11 numpy scipy matplotlib
conda activate myproject
Conda resolves binary compatibility automatically, which makes it the preferred tool for scientific computing stacks that include GPU libraries (CuPy, PyTorch with CUDA).
virtual environment
An isolated Python installation that keeps project dependencies separate from the system Python and from other projects.
Related: conda
conda
A package and environment manager that handles Python packages and non-Python dependencies (C/C++ libraries, CUDA). Distributed via Anaconda or Miniconda.
Related: virtual environment
Example: Creating a Reproducible Environment
Create a conda environment for this book with pinned dependencies, export it, and recreate it on another machine.
Create the environment
# Terminal commands:
conda create -n ferkans-sci python=3.11 -y
conda activate ferkans-sci
pip install numpy==1.26.4 scipy==1.13.0 matplotlib==3.9.0
pip install ipython pytest pyyaml
Export with pinned versions
# For conda environments:
conda env export > environment.yml
# For pip-only environments:
pip freeze > requirements.txt
The environment.yml captures the exact versions of every package,
including transitive dependencies.
Recreate on another machine
# From environment.yml:
conda env create -f environment.yml
# From requirements.txt:
python -m venv myenv && source myenv/bin/activate
pip install -r requirements.txt
Environment Setup Script
# Code from: ch01/python/setup_environment.py
# Load from backend supplements endpointCommon Mistake: Never Use System Python for Scientific Work
Mistake:
Installing scientific packages directly into the system Python
(sudo pip install numpy) or using the macOS/Linux system Python
for project work.
Correction:
Always use a virtual environment or conda environment. System Python
is managed by the OS and modifying it can break system tools. Use
pyenv to manage multiple Python versions if needed:
pyenv install 3.11.8
pyenv local 3.11.8
Common Mistake: Mixing pip and conda Carelessly
Mistake:
Alternating between pip install and conda install in the same
environment without a clear strategy, leading to dependency conflicts
that are difficult to diagnose.
Correction:
Use conda to install packages that have complex binary dependencies
(numpy, scipy, pytorch, cupy) and pip for pure-Python packages.
Always install conda packages first, then pip packages. If using
pip inside conda, prefer pip install over conda install for
the same package — never install the same package with both.
venv vs. conda
| Feature | venv | conda |
|---|---|---|
| Scope | Python packages only | Python + C/C++/CUDA libraries |
| Speed | Fast (no solver) | Slower (SAT solver for deps) |
| Reproducibility | requirements.txt | environment.yml |
| GPU support | Manual CUDA setup | Automatic CUDA toolkit |
| Disk usage | Lightweight | Heavier (copies of libraries) |
| Best for | Pure Python projects | Scientific computing stacks |
Python Version Feature Timeline
Explore which Python features became available in each version. This helps you understand minimum version requirements for modern Python idioms used throughout the book.
Parameters
pyenv
A tool for managing multiple Python versions on a single machine.
Allows switching between Python versions per-project via
.python-version files.
Related: virtual environment
pip
Python's default package installer. Installs packages from the
Python Package Index (PyPI). Command: pip install package_name.
Key Takeaway
Every project gets its own environment. Whether you use venv or
conda, the rule is the same: never install scientific packages
into the system Python, and pin your dependencies for reproducibility.
Quick Check
Which command creates a conda environment with Python 3.11 and NumPy?
conda create -n myenv python=3.11 numpy
pip install python=3.11 numpy
python -m venv myenv --with numpy
conda install python=3.11 numpy
conda create -n myenv python=3.11 numpyThis creates a new environment named 'myenv' with the specified Python version and NumPy.