DOCSEARCHbyAlgolia
Submit Docs

venvstacks

repository·main·Indexed 43 days ago

https://github.com/lmstudio-ai/venvstacks

A tool for packaging Python applications into layered, portable, and deterministic virtual environment stacks. It is optimized for AI/ML workflows to share large dependencies like PyTorch across multiple application layers to save space and bandwidth. Version 0.8.0 utilizes a three-layer architecture consisting of runtime, framework, and application layers, orchestrating tools such as pdm, uv, and python-build-standalone.

Tokens
34.5K
Snippets
109
Records
165
Agent score
53%

What's inside venvstacks

  1. Use the venvstacks.stacks Python API

    main

    The venvstacks.stacks module provides the core Python API for managing virtual environment stacks. It is organized into several functional areas: high-level interfaces for defining stacks and environments, metadata components for layers, publishing and exporting results, layer specifications, build environments, and build configuration.

    Warning: The Python API is NOT YET STABLE. Function, class, and method names may change between releases without any deprecation period.

  2. Understand the venvstacks Layer Hierarchy

    main

    venvstacks uses a layered model to define Python environments. Layers are definitions of pinned requirements that are consolidated into an archive for deployment. There are three distinct types of layers:

    • Base Runtime Layers: Specify the foundation Python runtime. Requirements here are installed directly into the base runtime without a virtual environment.
    • Framework Layers: Contain large, shared dependencies (e.g., torch) to avoid redundant publishing. They depend on a specific runtime layer. Applications using these layers are constrained to the versions provided by the framework.
    • Application Layers: Specify the actual deployed Python applications. These depend on one or more framework layers.

    A Stack is the complete combination of an application layer plus its required framework and base runtime layers.

  3. Define environment stacks using venvstacks.toml

    main
    Environment stacks are defined using a venvstacks.toml file. This file serves as the stack specification (or stack definition), describing one or more environment stacks by composing the necessary layers (runtime, framework, and application).

  4. Define environment stacks in venvstacks.toml

    main

    Environment layers are defined in a venvstacks.toml specification file. The file uses separate arrays of tables for different kinds of layers (e.g., Runtimes, Frameworks, Applications).

    Layers are organized hierarchically:

    1. Runtimes: The base Python environment (e.g., cpython-3.11).
    2. Frameworks: Shared layers built on top of runtimes (e.g., scikit-learn).
    3. Applications: The final environments that depend on specific frameworks and runtimes.

  5. How dynamic linking works across layers on Linux and macOS

    main

    To enable binary extension modules in one layer to find dynamically linked libraries (like CUDA libraries) provided by a different layer, venvstacks implements a symlink and wrapper mechanism on Linux and macOS:

    1. Symlink Generation: During environment building, all shared object files that are not Python extension modules are symlinked into a share/venv/dynlib folder within the built environment.
    2. Runtime Wrapper: The symlink to the base environment's Python runtime is replaced with a wrapper script. This script ensures that the share/venv/dynlib folders of all linked layers are added to the shared object loading path (LD_LIBRARY_PATH on Linux, DYLD_LIBRARY_PATH on macOS) before the Python runtime is invoked.

    Note: Library path entries are appended to existing entries, so existing environment variables still take precedence.

    Handling Conflicts: If multiple packages in the same layer provide dynamic libraries with the same name, a BuildEnvError is raised. You can resolve this using the dynlib_exclude option in your layer definition.

    # Example of resolving a dynamic library conflict in a layer definition
layer_definition = {
    "name": "my-layer",
    "dynlib_exclude": ["libpng16.*"]
}

# Using path matching to be more specific
layer_definition_specific = {
    "name": "my-layer",
    "dynlib_exclude": ["PIL/**/libpng16.*"]
}

# Disabling all dynamic library symlinks for a layer
layer_definition_none = {
    "name": "my-layer",
    "dynlib_exclude": ["*"]
}

  6. How venvstacks handles package builds and reproducibility

    main

    To ensure that published layer archives are reproducible (byte-for-byte identical when built with the same environment), venvstacks requires that all Python packages be provided as pre-built binary wheels.

    This approach allows layer lock files to record exact binary hashes and avoids variations introduced by the installation process (such as build-location dependencies). Because of this design, venvstacks does not natively support building components directly from source references. If you have dependencies without binary wheels, you must build them into wheels externally before including them in your stack.

  7. Understand venvstacks metadata file locations and formats

    main

    The build process generates several metadata files in different formats and locations depending on the context:

    Build Output (Local/Publishing)

    When publishing artifacts or exporting environments, metadata is stored in a directory named __venvstacks__:

    • Platform-specific layer summaries: __venvstacks__/<platform>/venvstacks.json
    • Per-layer metadata: __venvstacks__/<platform>/env_metadata/<layer_metadata>.json

    Deployed Environments

    In a deployed environment, the layer configuration metadata is located at:

    • share/venv/metadata/venvstacks_layer.json

    File Formats

    • Input files: Human-edited configuration files (like venvstacks.toml) use TOML.
    • Output files: All generated metadata files use JSON.

  8. Understand the purpose of injected files in venvstacks

    main

    Files located in the src/venvstacks/_injected/ directory are specialized assets used during the deployment lifecycle. They serve two primary purposes:

    1. Deployment Injection: These files are automatically injected into environments when you publish artifacts or perform local environment exports.
    2. Build-time Integration: They are designed to be importable by the build process. This allows the build system to access specific functionality without duplicating implementation logic, and ensures these components can be unit tested like standard code.

  9. Layer depth and performance considerations

    main

    While venvstacks does not enforce a hard limit on the depth of layer dependency chains (how many framework layers exist between the application and the runtime), there are practical implications to consider:

    • sys.path overhead: Each framework layer adds an extra sys.path entry on all platforms.
    • Loading path overhead: On Linux and macOS, each layer adds an extra dynamic library loading path entry.

    Best Practice: To avoid hitting Python runtime or platform limitations, it is recommended to keep dependency chains relatively short (ideally no more than 5-10 total layers) rather than creating excessively granular layers.

  10. Understand the venvstacks build and packaging toolchain

    main

    To use or extend venvstacks, it is important to understand the underlying tools it orchestrates:

    • Runtime Management: Base runtime environment layers are installed with pdm using runtimes from the python-build-standalone project.
    • Environment Creation: Layered framework and app environments are created using the standard library's venv module.
    • Dependency Locking: Platform-specific locking for each layer is performed using uv pip compile. The locked requirements from lower layers are passed as constraints files when locking higher layers.
    • Package Installation: Packages are installed using uv pip install. An override mechanism is used to prevent re-installing packages already provided by lower layers.
    • Distribution Format: venvstacks expects precompiled wheel archives for all included Python distribution packages. If these are not available, tools like wagon or fromager can be used to generate them.

  11. Define virtual environment stacks using TOML

    main

    Virtual environment stacks are defined in a TOML file using three top-level arrays of tables. These arrays represent the different types of layers available in a stack:

    • [[runtimes]]: The base Python runtime layer.
    • [[frameworks]]: Intermediate layers providing specific libraries or frameworks.
    • [[applications]]: The top-level layer containing the application logic and launch configuration.
    [[runtimes]]
[[frameworks]]
[[applications]]