Pixi Package Manager

repository·main·Indexed 11 days ago

https://github.com/prefix-dev/pixi

A cross-platform, multi-language package manager and workflow tool built on the Conda ecosystem. Pixi provides a Cargo-like experience for managing dependencies and tasks across languages including Python, C++, and R. It supports hybrid packaging, conditional dependencies via manifest expressions, and solve-groups for managing environment-specific installation modes (e.g., editable vs. production).

Tokens
264.3K
Snippets
946
Records
1.2K
Agent score
84%

What's inside Pixi

  1. What is Pixi?

    main

    Pixi is a package manager designed to bring a modern developer experience (similar to cargo, npm, or yarn) to the Conda packaging ecosystem. It is built on top of Conda, allowing users to leverage the vast collection of packages available on conda-forge.

    Pixi is designed to be:

    • Fast: Rapid environment solving.
    • User Friendly: Intuitive commands with minimal surprises.
    • Isolated: Reproducible, platform-agnostic environments.
    • A Single Tool: Handles dependency management, command management, building, and uploading packages in one place.
    • Language Agnostic: Supports any language available via Conda, including Python, C++, Rust, and Zig. It is particularly effective for multi-language projects (e.g., C++ mixed with Python) by managing everything from application dependencies to system-level packages.
  2. Use deno_task_shell syntax and built-in commands

    main

    Pixi uses deno_task_shell to provide a cross-platform, Bourne-shell-like interface for tasks. This allows you to write shell scripts that work on Windows, macOS, and Linux.

    Built-in Commands

    • cp, mv, rm, mkdir, pwd, sleep, echo, cat, exit, unset, xargs

    Shell Syntax

    • Logic: && (continue if success), || (continue if fail), ; (sequential execution).
    • Variables: export VAR=val to set, $VAR to use, unset VAR to remove. Shell variables (set via VAR=val) are not exported to spawned commands.
    • Pipelines: | (stdout), |& (stdout + stderr).
    • Command Substitution: $(command)
    • Negation: ! command negates the exit code.
    • Redirects: > (overwrite stdout), >> (append stdout), 2> (stderr), &> (both).
    • Globbing: * (standard), **/*.py (recursive).
  3. How Python dependencies are mapped to Pixi

    main

    Pixi automatically translates standard Python [project] fields into Pixi workspace configurations:

    1. requires-python: The version specified in requires-python is automatically added as a python dependency in the Pixi workspace.
    2. dependencies: Standard PyPI dependencies listed in [project.dependencies] are automatically added to the Pixi workspace as [pypi-dependencies].

    Overriding PyPI dependencies with Conda: If you want to use a Conda package instead of a PyPI package for a specific dependency, add it to the [tool.pixi.dependencies] section. Pixi prioritizes Conda dependencies over PyPI dependencies.

    [project]
    name = "my_project"
    requires-python = ">=3.9"
    dependencies = [
        "numpy",
        "pandas",
    ]
    
    # To force numpy to be a Conda dependency instead of PyPI:
    [tool.pixi.dependencies]
    numpy = "*"
  4. Opt out of `CONDA_PREFIX` for global tools

    main

    By default, Pixi sets CONDA_PREFIX to the environment's path when running a globally exposed executable. Some tools may behave unexpectedly if they see this variable.

    To prevent this, package authors can include a marker file at: etc/pixi/<executable>/global-ignore-conda-prefix

    Inside the package's recipe.yaml, you can create this file during the build script:

    build:
      script:
        - mkdir -p $PREFIX/etc/pixi/borg
        - touch $PREFIX/etc/pixi/borg/global-ignore-conda-prefix

    When this file exists, Pixi will remove CONDA_PREFIX from the environment variables for that specific executable.

  5. Understand the C ABI and cross-language communication in Polyglot Particles

    main

    The system relies on a shared C ABI defined in particle_core to allow different languages to interact without using dlopen between packages.

    The Core Interface

    particle_core provides:

    • Two vtable types: pc_emitter_vtable_t and pc_modifier_vtable_t.
    • A runtime pc_pool_t that manages particles, runs modifiers, and pulls from emitters.

    Cross-Language Flow

    1. Implementation: particle_cpp and particle_rs implement the vtable interfaces and export discovery functions (e.g., particle_cpp_get_emitter("cone")).
    2. Python Bindings: particle_cpp_py (via pybind11) and particle_rs (via PyO3) expose these implementations as Python objects. These objects expose vtable_addr and state_addr as integers.
    3. Execution: The particle_view module (a pybind11 module) provides a View(w, h).run(emitters=[...], modifiers=[...]) function. It accepts the raw integer addresses provided by the Python objects, builds a pc_pool_t internally, and executes the SDL loop.
    4. Orchestration: particle_kit acts as the high-level orchestrator, importing the binding modules and the view to build and run scenes.
  6. Use wildcard platform selectors for shared configuration

    main

    To avoid repeating configuration for multiple platforms that share common traits, you can use the * wildcard in a target selector. This is most effective when using custom platform names in your workspace.platforms definition.

    Rules for Wildcards:

    • * is the only supported metacharacter and matches any run of characters.
    • Patterns are matched in full and are case-sensitive (e.g., cuda-*, *-64, *cuda*).
    • Precedence: If multiple selectors match a platform, the one defined later in the pixi.toml manifest wins. Always place specific overrides (e.g., [target.cuda-win-64]) after the wildcard block (e.g., [target."cuda-*"]).
    • Restrictions: Wildcards are only allowed on workspace and feature targets. They are not supported in [package.target] or [package.build.target].
    [workspace]
    platforms = [
      { name = "cuda-win-64", platform = "win-64", cuda = "12" },
      { name = "cuda-linux-64", platform = "linux-64", cuda = "12" },
      "win-64",
      "linux-64",
    ]
    
    [target."cuda-*".tasks]
    test = "python test.py --cuda"
    train = "python train.py --cuda"
  7. Understand Environment Variable Priority in Pixi

    main

    Pixi follows a strict hierarchy when resolving environment variables. If a variable is defined at multiple levels, the level with the highest priority wins.

    Priority Order (Highest to Lowest):

    1. task.env (Defined in [tasks.<name>.env] in pixi.toml)
    2. activation.env (Defined in [activation.env] in pixi.toml)
    3. activation.scripts (Variables exported in files listed in [activation.scripts])
    4. Activation scripts of dependencies
    5. Outside environment variables (Variables set in your shell/system before running Pixi)

    Example: task.env overriding everything

    If you define APP_CONFIG in a task, it will override values in activation settings, dependencies, and your system shell.

    [tasks.start]
    cmd = "echo Config: $APP_CONFIG"
    env = { APP_CONFIG = "task-specific" }
    
    [activation.env]
    APP_CONFIG = "activation-env"
    
    [activation]
    scripts = ["app_setup.sh"]
    
    [dependencies]
    config-loader = "*"  # Sets APP_CONFIG="dependency-config"
    export APP_CONFIG="activation-script"

    If you run pixi run start while export APP_CONFIG="system-config" is set in your shell, the output will be: Config: task-specific

    [tasks.start]
    cmd = "echo Config: $APP_CONFIG"
    env = { APP_CONFIG = "task-specific" }
    
    [activation.env]
    APP_CONFIG = "activation-env"
    
    [activation]
    scripts = ["app_setup.sh"]
    
    [dependencies]
    config-loader = "*"  # Sets APP_CONFIG="dependency-config"
    export APP_CONFIG="activation-script"

    Running the command:

    pixi run start

    Output: Config: task-specific

  8. Use `[constraints]` to restrict transitive dependencies

    main

    The [constraints] table allows you to restrict the versions of packages that may be installed without explicitly requiring them as dependencies. A constraint is only enforced if the package is pulled in by another dependency.

    This is useful for:

    • Preventing known-bad versions: Ruling out a specific version range of a transitive dependency that has a regression.
    • Coordinating optional packages: Ensuring that if an optional library (like CUDA) is installed, it meets a specific version requirement.

    Constraints use the same VersionSpec and MatchSpec syntax as [dependencies]. They can also be made platform-specific using the [target.<platform>.constraints] table.

    [dependencies]
    requests = ">=2.28"
    
    [constraints]
    # Ensures openssl is at least 3.0 if it is pulled in transitively
    openssl = ">=3.0"
    
    [target.linux-64.constraints]
    # Tighten the constraint specifically for Linux
    openssl = ">=3.0.7"
  9. Configure per-environment editability with solve-groups

    main

    You can use Pixi solve-groups to ensure that different environments (e.g., default for development and prod for production) use identical dependency versions while allowing different installation modes.

    Specifically, you can use the editable flag in pypi-dependencies to install a local package as editable in development environments for faster iteration, while installing it as non-editable in production environments for a clean deployment. This is achieved by defining different features for each environment and assigning them to the same solve-group.

    [tool.pixi.feature.dev.pypi-dependencies]
    docker-project = { path = ".", editable = true }
    
    [tool.pixi.feature.prod.pypi-dependencies]
    docker-project = { path = ".", editable = false }
    
    [tool.pixi.environments]
    default = { features = ["test", "dev"], solve-group = "default" }
    prod = { features = ["prod"], solve-group = "default" }
  10. Understand package hooks as code execution surfaces

    main

    Be aware that package installation and activation can trigger arbitrary code execution:

    • post-link scripts: These run during installation. Pixi disables these by default. Do not enable them unless you have explicitly reviewed the package behavior.
    • Activation scripts: These run during environment activation (e.g., when using pixi shell, pixi run, or pixi shell-hook). These are enabled by default as part of standard conda behavior.
    • Resolution-time execution: Commands like pixi lock, pixi install, and pixi update can execute code if your project contains source dependencies, as Pixi must invoke the package's build backend. Only run these commands on trusted projects or within a sandbox.

    Security Tip for direnv users: If you use watch_file pixi.lock, a change to the lock file will automatically trigger pixi shell-hook, which can execute malicious activation scripts. Consider using require_allowed pixi.toml pixi.lock (once supported by direnv) to force a manual approval step.

  11. Understand the Global Manifest

    main

    The Global Manifest is a TOML file that tracks all globally installed environments, their dependencies, channels, and exposed binaries. It can be edited manually, synced, or shared via version control.

    Key components of an environment entry in the manifest:

    • channels: The Conda channels used for downloading packages (ordered by priority).
    • dependencies: The Conda packages to be installed (supports version constraints and wildcards).
    • exposed: A mapping of package binaries to the names they will be available under in your system PATH.
    • platform: The target platform for the environment (defaults to the current machine).
    • shortcuts: (Optional) List of application shortcuts (e.g., for Start Menus).
    version = 1
    
    [envs.python]
    channels = ["conda-forge"]
    dependencies = { python = "3.12.*" }
    exposed = { py3 = "python" }
  12. How skills are defined

    main

    A skill is a directory containing a SKILL.md file with YAML frontmatter. The agent reads this file to understand the skill's purpose.

    • name: (Optional) The name of the skill. Defaults to the directory name.
    • description: (Required) A description of what the skill does.

    Skill instructions must be written in Markdown.

    ---
    name: my-skill
    description: "Does something useful for the agent"
    ---
    
    Skill instructions go here as Markdown.
    The agent reads this file to understand what the skill does.