Poetry: Python Dependency Management and Packaging

repository·main·Indexed 12 days ago

https://github.com/python-poetry/poetry

A tool for Python packaging and dependency management that uses the pyproject.toml format to replace legacy configuration files like setup.py and requirements.txt. It provides deterministic installs via a lockfile, manages dependency groups and optional extras, and supports building projects for distribution. Requires Python 3.10 or higher.

Tokens
58K
Snippets
231
Records
271
Agent score
97%

What's inside Poetry

  1. What is Poetry and what does it do?

    main

    Poetry is a tool for dependency management and packaging in Python. It allows you to declare the libraries your project depends on and manages their installation and updates. Key features include:

    • Dependency Management: Declare and manage project libraries.
    • Lockfile: Provides a lockfile to ensure repeatable, deterministic installs across different environments.
    • Packaging: Builds your project for distribution.
  2. Managing the poetry.lock file in libraries

    main

    When developing a library, you can choose whether or not to commit the poetry.lock file to your version control (e.g., Git).

    • Committing poetry.lock: Helps your development team test against the exact same dependency versions.
    • Not committing poetry.lock: Recommended if you want to avoid locking versions for downstream consumers. The lock file only affects the main project it is in; it has no effect on other projects that depend on your library.
  3. Include or exclude files and packages in distributions

    main

    You can control which files and modules are included in your final distribution using packages, include, and exclude within [tool.poetry].

    Including Packages

    Use packages to specify modules that are not automatically detected. Poetry automatically detects a single module/package matching the project name in the root or src/ directory. If you use packages, you must explicitly include your main package.

    • include: The module/package name.
    • from: The directory where the package is located (e.g., lib).
    • to: The relative destination path upon installation.
    • format: Restrict to sdist or wheel.

    Include and Exclude Patterns

    • exclude: A list of patterns (globs) to ignore. Defaults to both sdist and wheel.
    • include: A list of patterns to include. include has priority over exclude. Defaults to sdist if no format is specified.

    Warning: When installing a wheel, include files are unpacked into site-packages. Avoid including top-level files like CHANGELOG.md or tests in wheels.

    [tool.poetry.packages]
    # Explicitly include packages
    packages = [
        { include = "my_package" },
        { include = "extra_package/**/*.py" },
        { include = "my_package", from = "lib", to = "target_package" },
        { include = "my_other_package", format = "sdist" }
    ]
    
    # Include/Exclude specific files
    exclude = ["my_package/excluded.py"]
    include = [
        { path = "tests", format = "sdist" },
        { path = "my_package/for_sdist_and_wheel.txt", format = ["sdist", "wheel"] }
    ]
  4. Include one dependency group in another

    main

    You can aggregate dependencies by including one group inside another. This is useful for creating meta-groups like a dev group that contains all test and lint dependencies.

    In the [dependency-groups] (PEP 735) format, use the { include-group = "group-name" } syntax. In the [tool.poetry.group] format, use the include-groups key.

    # PEP 735 style
    [dependency-groups]
    test = ["pytest"]
    lint = ["ruff"]
    dev = [
        { include-group = "test" },
        { include-group = "lint" },
        "tox",
    ]
    
    # Poetry style
    [tool.poetry.group.dev]
    include-groups = [
        "test",
        "lint",
    ]
    
    [tool.poetry.group.dev.dependencies]
    tox = "*"
  5. When is the Poetry build script executed?

    main

    If your project defines a build script via [tool.poetry.build].script, it is executed automatically in the following scenarios:

    1. poetry install: Executed prior to installing the project's root package.
    2. poetry build: Executed prior to building distributions.
    3. PEP 517 build: Executed when another build frontend triggers a PEP 517 build from a source or sdist.
  6. Supported package source types

    main

    Poetry supports several types of package repositories:

    • PyPI: The default Python Package Index. Accessed via JSON API.
    • Simple API Repository (PEP 503): Custom public or private repositories. Note that PEP 503 compliant URLs should typically end in /simple/.
    • Simple API Repository (PEP 658): Supported for faster dependency resolution by providing metadata without downloading full distributions.
    • Single Page Link Source: Projects that release binary distributions via a single page link structure.
  7. Listen to Poetry Events with Plugins

    main

    Plugins can intercept execution by listening to events fired by the Cleo console. These events are accessible via the cleo.events.console_events module. Common events include:

    • COMMAND: Fired before any command is executed.
    • SIGNAL: Fired when command execution is interrupted.
    • TERMINATE: Fired after the command finishes.
    • ERROR: Fired when an uncaught exception occurs.

    You can attach listeners to the application.event_dispatcher within your plugin's activate method.

    from cleo.events.console_events import COMMAND
    from cleo.events.console_command_event import ConsoleCommandEvent
    from cleo.events.event_dispatcher import EventDispatcher
    from dotenv import load_dotenv
    from poetry.console.application import Application
    from poetry.console.commands.env_command import EnvCommand
    from poetry.plugins.application_plugin import ApplicationPlugin
    
    class MyApplicationPlugin(ApplicationPlugin):
        def activate(self, application: Application):
            application.event_dispatcher.add_listener(
                COMMAND, self.load_dotenv
            )
    
        def load_dotenv(
            self,
            event: ConsoleCommandEvent,
            event_name: str,
            dispatcher: EventDispatcher
        ) -> None:
            command = event.command
            if not isinstance(command, EnvCommand):
                return
    
            io = event.io
            if io.is_debug():
                io.write_line("<debug>Loading environment variables.</debug>")
    
            load_dotenv()
  8. How Poetry manages project isolation

    main

    Poetry provides project isolation by managing virtual environments.

    1. Detection: Poetry first checks if it is already running inside an active virtual environment. If so, it uses that environment directly.
    2. Creation: If no environment is active, Poetry will either use an existing one it previously created for the project or create a brand new one.
    3. Python Selection: By default, Poetry uses the Python version that was used to install Poetry itself. If that version is incompatible with the python range defined in your project's metadata, Poetry will attempt to find a compatible version on your system. If it cannot find one, you must explicitly activate a version using poetry env use.

    Integration with pyenv: If you use pyenv, you can switch the current Python version in your shell, and Poetry will respect that version when creating a new environment.

    pyenv install 3.9.8
    pyenv local 3.9.8
    poetry install
  9. Understand the `pyproject.toml` project format

    main

    Poetry uses a single pyproject.toml file to replace legacy files like setup.py, requirements.txt, setup.cfg, MANIFEST.in, and Pipfile. This file centralizes build system requirements, project metadata, dependencies, and scripts.

    Key sections include:

    • [build-system]: Defines the build backend (e.g., poetry.core.masonry.api).
    • [project]: Standard PEP 621 metadata (name, version, description, etc.).
    • [project.dependencies]: Standard dependency declarations.
    • [project.optional-dependencies]: Extras that can be installed via poetry install -E <extra>.
    • [project.scripts]: CLI entry points.
    • [tool.poetry.dependencies]: Poetry-specific dependency configurations (e.g., allowing prereleases).
    • [dependency-groups]: Organized groups for development or documentation tools.
    [build-system]
    requires = ["poetry-core>=2.0.0,<3.0.0"]
    build-backend = "poetry.core.masonry.api"
    
    [project]
    name = "my-package"
    version = "0.1.0"
    # ...
    
    [project.scripts]
    my_package_cli = "my_package.console:run"
  10. Switch between Package and Non-Package modes

    main

    Poetry operates in two modes:

    1. Package mode (Default): Used when you want to build (sdist/wheel) and publish your project. Requires name and version metadata. Running poetry install installs your project itself in editable mode.
    2. Non-package mode: Used when you only want to manage dependencies (e.g., for applications or services) and do not intend to publish the project. Metadata like name and version are optional. Running poetry install only installs dependencies (equivalent to poetry install --no-root).

    To enable non-package mode, add the following to pyproject.toml:

    [tool.poetry]
    package-mode = false
  11. Organize dependencies using Dependency Groups

    main

    Poetry allows you to organize dependencies into logical groups (e.g., test, docs, lint) to separate runtime requirements from development tools.

    • The main group is implicit and contains dependencies declared in project.dependencies (PEP 621) or tool.poetry.dependencies. These are required for the project to run.
    • Custom groups can be declared using the PEP 735 [dependency-groups] section or the Poetry-specific [tool.poetry.group.<group>] section.

    Important: All dependencies across all groups must be compatible with each other because Poetry resolves the entire dependency tree regardless of which groups are being installed.

    # PEP 735 style
    [dependency-groups]
    test = [
        "pytest (>=6.0.0,<7.0.0)",
        "pytest-mock",
    ]
    
    # Poetry style
    [tool.poetry.group.test.dependencies]
    pytest = "^6.0.0"
    pytest-mock = "*"
  12. Versioning requirements for libraries

    main

    Poetry requires all project versions to be PEP 440-compliant. While Poetry supports semantic versioning constraints, it does not enforce a specific release convention.

    Invalid Version Example: 1.0.0-hotfix.1 is NOT PEP 440 compliant. Valid Alternatives: Use 1.0.0-post1 or 1.0.0.post1 instead.