Python

Python is a general-purpose programming language, aimed at striking a balance between performance and code interpretability. Python has some high-performance numerical libraries, but is not as performant as languages like C++. However, it is widely used, the code can be easy to read, and it has a large collection of packages.

Installation

Do not use the Python that came with your operating system for research work. Install and manage your own, so that each project gets an isolated, reproducible environment.

Two reasonable approaches:

• conda (via Miniforge), which handles non-Python dependencies well. This matters for the geospatial and climate stack (GDAL, PROJ, netCDF), which is why it remains our default.
• uv, a much faster pure-Python package and version manager. Good for projects that are Python-only.

Pick one per project and commit its lockfile or environment file to the repository. Whichever you use, avoid installing into a base environment.

Some packages may not be available through conda; in that case use pip from within the activated conda environment, and record them in your environment file.

Usage

A good place to start if you don’t know any Python is the Python Fundamentals chapter from Earth and Environmental Data Science. There are also a number of other resources available online, and many specific questions can be or already are answered on Stack Overflow.

Python is a dynamically-typed language, meaning that it automatically assigns types to variables and functions. This means that, by default, it lacks type checking, which can result in bugs. Dynamic typing is also one reason why Python is more user-friendly than a statically-typed language like C++, but is also slower.

Development Environments

• VS Code with the Python extension is the group default. It is free, handles notebooks and remote work over SSH (useful for Hopper), and is the same editor we recommend for Julia, so you only have to learn one.
• PyCharm: fully-featured IDE. The Community edition is free, and students can get the full version free through JetBrains’ academic licensing.
• Spyder: lightweight, free, open-source, with a MATLAB-like layout that some people coming from MATLAB prefer.

NoteAtom

Earlier versions of this page recommended Atom. GitHub sunset Atom in December 2022 and it no longer receives updates, including security updates. If you are still using it, switch.

Recommended Packages

You’ll usually want to include the following in any research conda environment:

• numpy: high-performance library for numerical computing.
• scipy: contains functionality for a wide variety of scientific computing tasks, including optimization and statistics.
• matplotlib: library for visualization and plotting.

Other packages may make sense depending on your workflow and tasks:

• pandas: data analysis and manipulation.
• polars: an alternative dataframe library, considerably faster than pandas on large tables. Worth knowing about if pandas becomes your bottleneck.
• xarray: allows for structured datasets and interfacing with netCDF.
• seaborn: adds some additional plotting functionality on top of matplotlib, and integrates well with pandas.
• PyMC: probabilistic programming using a variety of methods, including Hamiltonian Monte Carlo and Metropolis-Hastings. (This was called pymc3 for versions 3 and earlier; the project renamed to pymc at version 4, so older tutorials using import pymc3 need translating.)
• CmdStanPy: a Python interface to Stan, now the recommended way to drive Stan from Python.
• ArviZ: diagnostics, summaries, and plots for Bayesian inference. Pairs with both of the above and does the convergence checking you should be doing anyway.
• PyTorch: machine learning. Now the default in most research settings; TensorFlow is still around but you are less likely to need it.
• mpi4py: bindings for parallelization across multiple processors and compute nodes.
• mypy: optional static type-checking, can help reduce errors by clarifying expected inputs and outputs and make it easier for others to work with your code.
• ruff: linting and code formatting in one very fast tool. It replaces the black + flake8 + isort combination that older guides recommend.

Learn More

• Beyond PEP 8 – Best Practices for Beautiful Intelligible Code by Raymond Hettinger from PyCon 2015.
• Livecoding Madness - Let’s Build a Deep Learning Library by Joel Grus
• Software Testing and Testing Automation with Python
• Python posts from Water Programming.

When You’re Stuck

1. Read the traceback from the bottom up. The last line is what broke; the lines above are how you got there. The first frame in your code, rather than in a library, is usually where the problem is.
2. Official docs and the library’s own documentation before a search engine. Library docs are generally good, and a search result is often three versions out of date.
3. Check the version you actually have. A large fraction of “the docs are wrong” turns out to be a different version installed than the one you are reading about. pip show <package> or conda list.
4. Stack Overflow for error messages. Paste the exact message, minus your paths and variable names.
5. Make a minimal reproducer. Strip the problem to the smallest script that still fails. This solves it outright often enough to be worth doing before asking anyone, and when it does not, you now have something small enough for someone else to look at.
6. Ask. With the reproducer, the traceback, and what you already tried.