Practical Python Packaging (2023)
It has taken me years to have an opinion on what it takes to publish a python project to PyPI. There have been lots of developments in the past few years, but I want to document my current process so that I can track how it evolves over time.
Contents
src Layout #
I like src layout for python projects. There primary reason is that I can standardize workflows: if I know code is in src and tests are in test, my tools can act on this knowledge more easily. More generally, I try to follow this structure for other languages as well.
Abstract vs Concrete Dependencies #
I don't love the conventional terminology on this topic, but here are some pragmatic, if imprecise, definitions:
- Dependency: a package your project needs to be able to run. Dependencies can be optional or grouped by context.
- Abstract Dependency: when you specify the package and, optionally, some version information, but you don't specify where the packages are obtained from.
- Concrete Dependency: when you specify the package, version, and where it comes from.
Practically, most libraries are published on PyPI, so this difference seems foreign to most people.
Pipfile #
Explaining the locations of your project's dependencies
A Pipfile is supposed to enumerate your project's dependencies in an abstract way. That is, it is supposed to declare that you need library foo, but it doesn't say where to obtain it from.
Pipfile is the future replacement for requirements.txt that pip is supposedly . Practically, it is only supported by pipenv which is incredibly slow.
pyroject.toml #
setup.py #
setup.cfg #
https://setuptools.pypa.io/en/latest/userguide/pyproject_config.html