delvewheel

repository·master·Indexed 41 days ago

https://github.com/adang1345/delvewheel

A command-line tool for creating self-contained Python wheels for Windows by vendoring required DLL dependencies into the package. It serves as the Windows equivalent to auditwheel on Linux or delocate on macOS, providing functionality to repair wheels, inspect external DLL dependencies, and manage DLL name-mangling to avoid version conflicts.

Tokens
3.1K
Snippets
12
Records
19
Agent score
99%

What's inside delvewheel

  1. Custom patching with delvewheel

    master

    By default, delvewheel injects patches into __init__.py files to manage the DLL search path. If you need precise control over where these patches are placed, use the --custom-patch option.

    When --custom-patch is enabled, delvewheel will look for lines in your .py files containing the unindented comment # delvewheel: patch and replace those lines with the actual patch code.

  2. How name mangling works in delvewheel

    master

    To prevent "DLL hell" (where different Python modules require different versions of the same DLL name), delvewheel performs name mangling.

    The Process:

    1. Hashing: delvewheel computes a hash based on the DLL's contents and the hashes of its direct dependencies.
    2. Renaming: The DLL is renamed by appending this hash (e.g., C.dll becomes C-a55e90393a19a36b45c623ef23fe3f4a.dll).
    3. Import Table Editing: delvewheel modifies the binary's import directory table so the loader looks for the new, mangled filename instead of the original name.

    This ensures that multiple versions of the same DLL can coexist in a single Python process because they have unique filenames.

  3. Handling runtime-loaded DLLs

    master

    Because delvewheel relies on reading DLL file headers, it cannot automatically detect DLLs loaded dynamically via ctypes, cffi, LoadLibrary (C/C++/Rust), or libloading (Rust).

    To include these, use one of the following strategies:

    • Manual Inclusion: Use the --include option to specify additional DLLs to vendor into the wheel.
    • Analyze Existing: Include the runtime-loaded DLL in the wheel yourself, then use the --analyze-existing option.

    Note: You are responsible for ensuring these DLLs are discoverable at load time.

  4. Use delvewheel to repair Python wheels for Windows

    master

    Use delvewheel repair to make a Python wheel self-contained by copying its external DLL dependencies into the wheel and patching it so they are loaded correctly at runtime. This is useful for distributing wheels to users who may not have the required DLLs installed on their systems.

    Basic usage:

    delvewheel repair WHEEL

    delvewheel uses the PATH environment variable to find DLLs. You can also use the --add-path option to specify additional search directories.

    delvewheel repair WHEEL
  5. Handling Delay-Load DLL dependencies

    master

    If your project uses delay-load DLL dependencies, you must ensure the directory containing the vendored DLLs is in the search path when they are loaded.

    For Microsoft Visual C/C++ projects, delvewheel provides a custom delay-load import hook located at delayload/delayhook.c. You should add this file to your C/C++ project during the build process.

  6. Generate architecture-specific DLL lists

    master

    The dll_lists/ directory contains lists of .dll and .drv files found in C:\Windows\System32 and C:\Windows\SysWOW64 for various vanilla Windows installations. To identify the intersection of DLLs present across all lists for a specific architecture, run the generate.py script with the corresponding architecture flag.

    Note: Files without an export directory table are excluded from these lists. For Windows arm64, non-arm64 DLLs in C:\Windows\System32 are also excluded.

    generate.py x86
    generate.py x64
    generate.py arm64
  7. Resolving Name-Mangling errors with overlays

    master

    If delvewheel cannot mangle a DLL because the dependent binary contains an overlay (extra data at the end) and lacks sufficient internal padding to fit the new mangled names, an exception will be raised.

    Solutions:

    1. Remove the overlay (Recommended): Strip debug symbols or extra data using the GNU strip utility or the delvewheel flag:
      strip -s EXAMPLE.dll
      # OR
      delvewheel repair --strip
    2. Skip mangling: If the overlay is required for functionality, use the --no-mangle or --no-mangle-all flag to bypass the renaming process.
    delvewheel repair --strip
  8. Configure delvewheel search paths and inclusion/exclusion

    master

    When using show or repair, you can control how delvewheel finds and selects DLLs using the following mechanisms:

    Search Paths

    Use --add-path to provide additional directories to search for DLLs. These paths are prepended to the system PATH during execution.

    Inclusion and Exclusion

    • --include / --add-dll: Forces specific DLLs to be included. Use os.pathsep (e.g., ; on Windows) to provide multiple names.
    • --exclude / --no-dll: Forces specific DLLs or patterns (using *) to be excluded.

    Note: You cannot force both inclusion and exclusion of the same DLL. If a pattern in --exclude matches a name in --include, an error will be raised.

  9. Configure delvewheel repair options

    master

    When running delvewheel repair, you can use several options to control how DLLs are handled:

    • --add-path PATHS: Additional path(s) to search for DLLs (path-separator-delimited). Searched before PATH.
    • --include DLLS: Force specific DLLs to be vendored. Note: delvewheel won't automatically search for their dependencies unless they are required by other included DLLs.
    • --exclude DLLS: Specifically exclude certain DLLs (supports * wildcard).
    • --ignore-existing: Do not search for or vendor DLLs already present in the wheel. Use this for simpler integration with other bundling tools.
    • --analyze-existing: Analyze and vendor dependencies of DLLs already in the wheel (these are name-mangled by default).
    • --analyze-existing-exes: Analyze and vendor dependencies of EXEs already in the wheel.
    • --no-mangle-all: Disable name-mangling for all DLLs.
    • --no-mangle DLLS: Disable name-mangling for specific DLLs.
    • --with-mangle: Used in conjunction with --ignore-existing to mangle dependencies of existing DLLs.
    • --strip: Strip DLLs containing an overlay during name-mangling (requires GNU strip in PATH).
    • -w TARGET / --wheel-dir TARGET: Specify the output directory for the repaired wheel (default is wheelhouse/).
    • -L LIB_SDIR / --lib-sdir LIB_SDIR: Specify the directory suffix for vendored DLLs (default is .libs).
    • --include-symbols: Include .pdb symbol files if they are in the same directory as the DLL.
    • --include-imports: Include .lib import library files if they are in the same directory as the DLL.
    • --namespace-pkg PKGS: Specify namespace packages (dot-notation, delimited by ; or :) to use an alternate patching strategy that avoids modifying __init__.py.
  10. Important delvewheel limitations

    master

    Be aware of the following constraints when using delvewheel:

    • Digital Signatures: Any DLL containing an Authenticode signature will have its signature cleared if dependencies are name-mangled or if built with a non-0 value for the /DEPENDENTLOADFLAG linker flag.
    • Multiple Architectures: You cannot repair a single wheel containing modules for different architectures (e.g., mixing win32 and win_amd64). Create and repair separate wheels for each architecture.
    • Windows Version Compatibility: delvewheel excludes system libraries present in Windows 7 SP1+. If you need to support older systems requiring specific system DLLs, use the --include flag.
    • Python Version Compatibility: The Python version running delvewheel must be able to parse the __init__.py or root .py files in the wheel. For example, do not use Python 3.9 to repair a wheel containing syntax from Python 3.10.