pip-tools

repository·main·Indexed 27 days ago

https://github.com/jazzband/pip-tools

A suite of command-line utilities, pip-compile and pip-sync, designed to manage Python dependencies deterministically. pip-compile resolves dependencies from pyproject.toml, setup.cfg, setup.py, or requirements.in to generate pinned requirements.txt files, while pip-sync synchronizes virtual environments to match these pinned files exactly.

Tokens
2.9K
Snippets
10
Records
26
Agent score
92%

What's inside pip-tools

  1. Overview of pip-tools

    main

    pip-tools provides a set of command-line tools designed to keep pip-based packages fresh while maintaining predictable and deterministic builds. It primarily consists of two tools:

    • pip-compile: Used to resolve dependencies and generate a pinned requirements file.
    • pip-sync: Used to synchronize your virtual environment with the pinned requirements file.
  2. Configure pip-tools using configuration files

    main

    You can define project-level defaults for pip-compile and pip-sync by writing them to a configuration file. This allows you to avoid repeating command-line flags.

    Configuration files are looked up in the following order:

    1. Any path explicitly passed via the --config flag.
    2. .pip-tools.toml in the directory of the input file.
    3. pyproject.toml in the directory of the input file.

    If you are piping input from stdin, the current working directory is used for lookup.

    To bypass configuration files entirely, use the --no-config flag.

  3. Compile dependencies with `pip-compile`

    main

    Use pip-compile to generate a pinned requirements.txt file from dependency specifications in pyproject.toml, setup.cfg, setup.py, or requirements.in.

    Important: Run pip-compile within the same virtual environment as your project to ensure environment markers and conditional dependencies resolve correctly relative to your project's environment.

    If an existing requirements.txt is found that satisfies the dependencies, pip-compile will make no changes. To compile from scratch, delete the existing file or use the --upgrade flag.

    pip-compile requirements.in
  4. Set a custom compile command name

    main

    To change the command name displayed in the autogenerated header of the requirements file, set the CUSTOM_COMPILE_COMMAND environment variable.

    $ CUSTOM_COMPILE_COMMAND="./pipcompilewrapper" pip-compile requirements.in
  5. Sync virtual environment with `pip-sync`

    main

    Use pip-sync to synchronize your virtual environment with a requirements.txt file. It will install, upgrade, or uninstall packages to ensure the environment matches the file exactly.

    Warning: pip-sync is intended to be used only with files generated by pip-compile. It will not upgrade or uninstall packaging tools like setuptools, pip, or pip-tools itself.

  6. Update requirements with `pip-compile`

    main

    To update your pinned dependencies, use the following flags:

    • --upgrade / -U: Upgrade all packages in the existing requirements.txt to the latest versions that satisfy the constraints.
    • --upgrade-package <package> / -P <package>: Upgrade only a specific package (or a specific version of a package).

    You can combine these to upgrade everything while constraining certain packages.

  7. Install pip-tools

    main

    To use pip-tools, you must install it within your project's virtual environment. This ensures that the tools are available to manage the specific dependencies of that environment.

    1. Activate your virtual environment.
    2. Install the package using python -m pip install pip-tools.
    $ source /path/to/venv/bin/activate
    (venv) $ python -m pip install pip-tools
  8. Add change notes for pull requests

    main

    To contribute change notes to the pip-tools changelog, add a new markdown file to the changelog.d/ directory. This project uses Towncrier to compile these fragments into the final changelog during a release.

    Naming Convention

    Files must be named using the format: $NUMBER.$CATEGORY.md.

    • $NUMBER: The PR number or issue number addressed (e.g., 404).
    • $CATEGORY: One of the valid categories (see Categories).
    • Multiple issues: If a PR addresses multiple issues, create a symlink to the change notes with another issue number in the name.

    Content Requirements

    • Use the simple past tense or constructions with "now".
    • Include a byline at the end of the file: -- by {user}github-username .

    Previewing Changes

    You can preview how the changelog will look by running:

    tox run -e build-docs

    Then view the changelog in the generated documentation.

  9. Define configuration sections in TOML

    main

    Configuration is organized into three distinct sections within a TOML file. Use [tool.pip-tools] for global settings, [tool.pip-tools.compile] for settings specific to pip-compile, and [tool.pip-tools.sync] for settings specific to pip-sync.

    Note: The documentation shows [tool.pip-tools.pip-sync] in the example, but the section definition lists [tool.pip-tools.sync]. Follow the section definitions for correct scoping.

    [tool.pip-tools]
    # configuration for pip-compile and pip-sync
    
    [tool.pip-tools.compile]
    # configuration specific to pip-compile
    
    [tool.pip-tools.sync]
    # configuration specific to pip-sync
  10. Use the pip-compile CLI

    main

    The pip-compile command generates a pinned requirements.txt file from source dependency files such as requirements.in, pyproject.toml, setup.cfg, or setup.py.

    By default, it looks for requirements.in. If multiple source files are provided, you must specify an --output-file.

    Common usage patterns:

    • Basic compilation: pip-compile (compiles requirements.in to requirements.txt)
    • Upgrade all packages: pip-compile --upgrade
    • Upgrade specific packages: pip-compile -P django -P requests
    • Include security hashes: pip-compile --generate-hashes
    • Compile with extras: pip-compile --extra dev pyproject.toml
  11. Deprecation notices for `pip-tools`

    main

    The following behaviors are changing in future major releases:

    • --allow-unsafe will be enabled by default. Use --no-allow-unsafe to maintain current behavior.
    • --strip-extras will be enabled by default. Use --no-strip-extras to maintain current behavior.
    • The legacy resolver is deprecated and will be removed.