still don't have a title

The State of Python Packaging in 2022

Every year or so, I revisit the current best practices for Python packaging. This was my summary for 2021 – here’s the update for 2022.

PyPA

PyPA is still the place to go for information, best practices and tutorials for packaging Python projects. My only criticism from last year, namely that PyPA was heavily biased towards their own tooling (e.g. pipenv), has been addressed: the tool recommendations section lists now several tools for the same purpose with their own ones not necessarily being the first anymore.

setup.py, setup.cfg, requirements.txt, Pipfile, pyproject.toml – oh my!

This is the reason why I’m revisiting the documentation every year, to see what’s the current way to go. Good progress has been made since last year:

Bye setup.py and setup.cfg – hello pyproject.toml

pyproject.toml finally got mature enough to replace setup.py and setup.cfg in most cases. Recent versions of setuptools and pip now fully support pyproject.toml and even PyPA’s packaging tutorial completely switched their example project from away setup.py towards pyproject.toml, making it an official recommendation.

So, now you can replace your setup.py with pyproject.toml. If you had already some kind of declarative configuration in setup.cfg you can move that as well into pyproject.toml. Most tools, like pypy or pytest also support configuration in pyproject.toml (flake8 being a notable exception…) so there’s no reason to keep setup.cfg around anymore. Actually, if you migrate to pyproject.toml it is best to do it properly and remove setup.py and setup.cfg as setuptools behaves a bit buggy when building a package that has either of them and the pyproject.toml.

requirements.txt

requirements.txt are still needed if you develop a “deployable” application (vs. a library) and want to provide pinned dependencies, i.e. with the specific versions that you’ve tested your application with. Usually, the list of requirements in requirements.txt is the same as defined in pyproject.toml, but with pinned versions.

Pipfile + Pipfile.lock

I still do completely ignore Pipfile and Pipfile.lock as they are only supported by pipenv and not backed by any standard.

Summary

The major change this year was the proper support of pyproject.toml. I am slowly replacing all setup.py and setup.cfg in my projects with pyproject.toml and haven’t discovered any issues yet. Even packaging those packages as Debian packages is well-supported by Debian’s tooling.

I’m still running a quite boring stack based on pyproject.toml and requirements.txt, ignoring more advanced tools like poetry or such for dependency management. My build-system defined in pyproject.toml requires setuptools and I’m using build for building and twine for uploading.

Since PyPA changed the packaging tutorial towards pyproject.toml and away from setup.py, I think we will slowly see setup.py and setup.cfg go away over the years. Speaking of PyPA, I’m happy that they changed their attitude towards a more unbiased recommendation of tooling.