Python Packaging User Guide (PyPUG)

This document provides a summary of various Python-related packaging, build, and management tools that are not part of the PyPA (Python Packaging Authority). These tools cover a wide range of use cases, including:

• Package Managers: Tools like conda, pdm, poetry, spack, and uv for managing dependencies, environments, and Python versions.
• Build Backends & Systems: Tools such as maturin (Rust extensions), meson-python (Meson build system), scikit-build-core (CMake), and buildout for assembling applications.
• Package Index Servers: Solutions for hosting private or cached package indexes, including devpi, pypiserver, simpleindex, and dumb-pypi.
• Deployment & Distribution: Tools like pex and shiv for creating standalone Python executables (zipapps), and zest.releaser for automating the release process.

distlib is a library implementing low-level functions for Python packaging and distribution. It is useful for developers of third-party packaging tools to:

• Make and upload binary and source distributions
• Achieve interoperability
• Resolve dependencies
• Manage package resources

Unlike the stricter packaging project, distlib provides fallback behaviors for handling legacy packages and metadata that predate modern interoperability standards.

virtualenv is a tool for creating isolated Python Virtual Environments.

Comparison to venv:

• Unlike venv, virtualenv can create environments for other Python versions by locating them via the PATH environment variable.
• It provides advanced features for configuring, maintaining, duplicating, and troubleshooting virtual environments.

The Version Specifiers specification defines a standardized scheme for two primary purposes:

1. Identifying versions of Python software distributions.
2. Declaring dependencies on specific versions of software.

This specification is intended to be used by Automated tools, which is a collective term for:

• Build tools: Tools that run on development systems to produce source and binary distribution archives.
• Index servers: Distribution registries that publish version/dependency metadata and enforce constraints.
• Publication tools: Tools that upload archives to index servers.
• Installation tools: Tools that run on deployment targets to consume archives and deploy them.

Pipenv is a toolchain that combines Pipfile, pip, and virtualenv to manage environments and dependencies.

Capabilities:

• Auto-imports requirements.txt.
• Checks for CVEs in Pipfile using safety.
• Manages environments and dependencies via CLI.
• Makes and checks file hashes to ensure compliance with hash-locked dependency specifiers.
• Eases uninstallation of packages and dependencies.

Downstream packaging refers to efforts by ecosystems like Linux distributions (e.g., Gentoo), Conda, Homebrew, and MacPorts to redistribute Python packages. These ecosystems often aim for native OS integration or compatibility with specific non-Python software versions.

To make your project easier for downstream maintainers to package, consider following these guidelines:

• Provide complete source distributions.
• Ensure builds do not require Internet access.
• Support building against system dependencies.
• Support downstream testing.

Implementing these practices helps downstream packagers audit code, run test suites, apply patches, and build on specific platforms or against specific system library versions.

Beyond the BigQuery console and the tools mentioned above, the following projects can be used to analyze download statistics:

• pandas-gbq: Allows accessing BigQuery query results directly via Pandas.
• ClickPy: A public application that visualizes download statistics with free direct SQL access to the underlying open-source ClickHouse database (updated daily).

Hatch is a unified command-line tool for managing dependencies and environment isolation.

For Package Developers:

• Use Hatch and its build backend Hatchling to configure, version, specify dependencies, and publish packages to PyPI.
• Use its plugin system to extend functionality.