How to build Python packages reproducibly with Poetry

Milan Unicsovics · February 19, 2024

For handling dependencies and creating Python packages, Poetry is a great choice. Poetry’s build command can generate source and wheel distributions. The wheel is a pre-built distribution format containing files and metadata, which only need to be moved to the target system to be installed. On the other hand, source (or sdist) distribution still requires a build step before it is usable. But can these formats be directly utilized in production?

header

Poetry’s design issue

It turns out there is a problem with Poetry’s package building functionality; it does not take into account the lock file while executing the build command. This design flaw leads to multiple issues with the built wheel:

  • No exact dependency versions are specified in the distribution for explicit dependencies if no exact requirements are used.
  • Implicit dependencies are missing from the distribution’s dependency specification.

Unsafe dependencies

Ultimately, these issues make the built wheel distribution unreliable and practically unusable in production as explicit and implicit dependencies are installed non-deterministically.

Locked versions in distributions

The Poetry team is aware of the issue, but making this feature available in a maintainable way is not easy. Until this long-awaited feature is ready, I created a plugin to solve the problem. The plugin extends the building process by reading up the lock file and putting the locked versions into the distribution’s metadata:

Correct dependencies

Installation & usage

  1. The easiest way to install the lockedbuild plugin is via Poetry:
    poetry self add poetry-plugin-lockedbuild
    
  2. Now, the plugin provides a new command to build a wheel file with locked packages:
    poetry lockedbuild
    

For further details, check out the project: