DOCSEARCHbyAlgolia
Submit Docs

nvfetcher

repository·master·Indexed 43 days ago

https://github.com/berberman/nvfetcher

A tool for automating Nix package updates that integrates nvchecker and the shake build system to generate Nix expressions and SHA256 hashes. It uses TOML configuration files to define package sources, determine latest version numbers, and fetch sources.

Tokens
1.5K
Snippets
4
Records
8
Agent score
5%

What's inside nvfetcher

  1. Overview of nvfetcher

    master

    nvfetcher is a tool designed to automate Nix package updates. It is built on top of the shake build system and integrates nvchecker to manage package sources.

    Users provide a TOML configuration file defining package sources. nvfetcher then:

    1. Determines the latest version numbers.
    2. Fetches the sources and calculates SHA256 sums.
    3. Generates _sources/generated.nix (ready-to-use Nix expressions) and _sources/generated.json (metadata used for version change tracking).

    Versions are checked during every run, but expensive prefetch operations are only performed when a version change is detected.

  2. Configure nvfetcher with a TOML file

    master

    To use nvfetcher, create a TOML configuration file (e.g., nvfetcher.toml) where each section defines a package. You specify how to find the version (e.g., via src.github) and how to fetch the source (e.g., via fetch.github).

    You can also specify cargo_lock files to automatically compute outputHashes for git dependencies within a Cargo.lock file.

    # nvfetcher.toml
[rustdeck]
src.github = "rustdesk/rustdesk"
fetch.github = "rustdesk/rustdesk"
# Compute outputHashes for git dependencies in cargo lock
cargo_lock = ["Cargo.lock"]

  3. Use nvfetcher as a Haskell library

    master

    For developers wanting to integrate nvfetcher into Haskell projects, the package is available on Hackage.

    You can use a Nix Flake to set up a development environment with GHC and the nvfetcher library pre-installed using the ghcWithNvfetcher development shell.

    nix develop github:berberman/nvfetcher#ghcWithNvfetcher
runghc Main.hs

  4. Use nvfetcher via CLI

    master

    To run nvfetcher as a CLI program, you must provide package source definitions in a TOML configuration file (defaulting to nvfetcher.toml). The CLI generates Nix sources for the latest versions of packages defined in your config.

    Common Commands:

    • nvfetcher build: (Default) Generates Nix sources.
    • nvfetcher clean: Removes all generated files.
    • nvfetcher purge: Removes the Shake database.

    Key CLI Flags:

    • -o|--build-dir DIR: Directory for artifacts (default: _sources).
    • -c|--config FILE: Path to the TOML config.
    • -f|--filter REGEX: Regex to filter which packages to update.
    • -k|--keyfile FILE: Path to an nvchecker keyfile.
    • -j NUM: Number of threads (0 for auto-detection).
    • --keep-going: Do not stop if some packages fail to fetch.
    Usage: nvfetcher [--version] [--help] [-o|--build-dir DIR] [--commit-changes] 
                 [-l|--changelog FILE] [-j NUM] [-r|--retry NUM] [-t|--timing] 
                 [-v|--verbose] [-f|--filter REGEX] [-k|--keyfile FILE] 
                 [--keep-old] [--keep-going] [TARGET] [-c|--config FILE]

  6. Extract source files and Cargo locks

    master

    You can instruct nvfetcher to extract specific files from the fetched source into the build directory.

    File Extraction: Use extract = [ "glob" ] (relative to source root). For each matching file, nvfetcher copies it to the build directory and generates an extract.<file_name> = ./. + "<path>" entry in the Nix expression.

    • Supports recursive globs like **/*.
    • Directories are ignored.

    Rust/Cargo Support: To automate cargoLock.outputHashes for rustPlatform.buildRustPackage, use:

    • cargo_lock = [ "cargo_lock_glob" ]: Globs for lock files relative to the source root. These will be pulled into the build directory.

  7. Configure version sources with nvchecker

    master

    In your nvfetcher.toml, each package is a TOML table. The src key defines how to track upstream version updates using nvchecker logic.

    Common src options:

    • src.github = owner/repo: Latest GitHub release.
    • src.pypi = pypi_name: Latest PyPI release.
    • src.git = git_url: Latest commit of a repo (use src.branch for specific branches).
    • src.archpkg = archlinux_pkg_name: Latest Arch Linux package.
    • src.aur = aur_pkg_name: Latest AUR package.
    • src.repology = project:repo: Latest version from Repology.
    • src.webpage = web_url: Version found via src.regex on a webpage.
    • src.cmd = cmd: Version from a shell command output.
    • src.container = owner/name: Latest tag from a Docker registry.

    Global src transformations: You can tweak the obtained version string using:

    • src.prefix: Add a prefix.
    • src.from_pattern / src.to_pattern: Regex-based transformations (e.g., stripping a v).

  8. Configure Nix fetchers

    master

    Once a version is determined, the fetch key defines how to retrieve the source code. The variable $ver can be used in strings to represent the version found by nvchecker.

    Common fetch options:

    • fetch.github = owner/repo: Fetch via GitHub.
    • fetch.pypi = pypi_name: Fetch via PyPI.
    • fetch.git = git_url: Fetch via Git.
    • fetch.url = url: Fetch via a direct URL.
    • fetch.tarball = tarball_url: Fetch a specific tarball.
    • fetch.docker = owner/name: Fetch a Docker image.

    Special Fetch Behaviors:

    • Force Fetching: Set fetch.force = true to always download the file even if the version hasn't changed. This is useful when the URL is static but the content changes.
    • Git Metadata: If using src.git, you can configure commit dates using git.date_format (strftime) and git.date_tz (IANA timezone or local).