lychee

repository·master·Indexed 23 days ago

https://github.com/lycheeverse/lychee

A high-performance, asynchronous link checker written in Rust that identifies broken links and email addresses in websites and files such as Markdown and HTML. It is available as a CLI tool, a Rust library (lychee-lib), and a GitHub Action. Features include support for custom User-Agents, Basic Auth, per-host throttling, and various output formats including JSON.

Tokens
11.4K
Snippets
20
Records
87
Agent score
84%

What's inside lychee

  1. Overview of lychee features

    master

    lychee is a fast, asynchronous link checker written in Rust. Key features include:

    • File Support: Checks Markdown, HTML, and text files, as well as entire websites.
    • Link Selection: Supports file globbing, include/exclude patterns, and filtering by scheme.
    • HTTP Capabilities: Supports custom User-Agents, Basic Auth, custom headers, custom timeouts, HEAD requests, redirects, per-host throttling, and retry/backoff logic.
    • URL Handling: Handles relative URLs, URL anchor fragments, URL text fragments, and email addresses.
    • Output Formats: Provides colored output, progress bars, summaries, and JSON output.
    • Advanced Usage: Can be used as a library, supports configuration files, and can skip private domains.
  2. Overview of lychee

    master
    lychee is a fast, async, stream-based link checker written in Rust. It is designed to find broken hyperlinks and mail addresses within websites and various file formats, including Markdown and HTML. It is available as a command-line utility, a Rust library (lychee-lib), and a GitHub Action.
  3. Configure lychee as a pre-commit hook

    master

    Lychee can be integrated into your development workflow using pre-commit.

    Run on staged files only

    Add the following to your .pre-commit-config.yaml:

    repos:
      - repo: https://github.com/lycheeverse/lychee.git
        rev: v0.15.1
        hooks:
          - id: lychee
            # Optionally include additional CLI arguments
            args: ["--no-progress", "--exclude", "file://"]

    Run against the entire repository

    To run lychee against the whole repository instead of just staged files, set pass_filenames to false and provide the directory path (e.g., .):

    - id: lychee
      args: ["--no-progress", "."]
      pass_filenames: false
  4. Install lychee via pre-built binaries or Cargo

    master

    Pre-built binaries

    Download binaries for Linux, macOS, and Windows from the releases page. You can also use cargo-binstall:

    cargo binstall lychee

    Build from source with Cargo

    First, install build dependencies (for APT/dpkg-based Linux distros):

    curl -sSf 'https://sh.rustup.rs' | sh
    apt install gcc pkg-config libc6-dev libssl-dev

    Then, compile and install:

    cargo install lychee
  5. Install Elvish completions for lychee

    master

    To enable tab-completion in Elvish, generate the completion script using lychee --generate complete-elvish.

    1. Create the library directory:
      dir="${XDG_CONFIG_HOME:-$HOME/.config}/elvish/lib"
      mkdir -p "$dir"
    2. Generate and save the file:
      lychee --generate complete-elvish > "$dir/lychee.elv"
    3. Add the following to your ~/.elvish/rc.elv:
      use lychee
    dir="${XDG_CONFIG_HOME:-$HOME/.config}/elvish/lib"
    mkdir -p "$dir"
    lychee --generate complete-elvish > "$dir/lychee.elv"
  6. Set up Lychee pre-commit hooks

    master

    You can integrate Lychee link checking into your workflow using pre-commit. Add the following configuration to your .pre-commit-config.yaml file.

    Important: You must use the lychee-v* format for the rev field (e.g., lychee-v0.20.1) rather than the standard v* format to ensure compatibility with the repository's tag structure.

    repos:
      - repo: https://github.com/lycheeverse/lychee
        rev: lychee-v0.20.1
        hooks:
          - id: lychee
  7. Install lychee via Docker

    master

    You can pull the lychee Docker image from Docker Hub. Image tags include latest (recommended), specific versions (e.g., 0.20.0), nightly, and master. Each tag also has an -alpine variant (e.g., latest-alpine).

    docker pull lycheeverse/lychee
  8. Run Lychee via Docker

    master

    You can run Lychee in a containerized environment. By default, a Debian-based image is used. To use an Alpine-based image, use the latest-alpine tag (e.g., lycheeverse/lychee:latest-alpine).

    When running via Docker, it is recommended to use the following flags:

    • --init: Allows Lychee to be stopped from the terminal.
    • -it: Enables interactive terminal mode to show the progress bar.
    • --rm: Automatically removes the container after the run.
    • -w /input: Sets the default workspace to /input.
    • -v <local_path>:/input: Mounts your local directory into the container.
    # Linux/macOS
    docker run --init -it --rm -w /input -v $(pwd):/input lycheeverse/lychee README.md
    
    # Windows PowerShell
    docker run --init -it --rm -w /input -v ${PWD}:/input lycheeverse/lychee README.md
  9. Configure GitHub Token to Avoid Rate Limiting

    master

    To prevent being rate-limited when checking github.com links, you can provide a GitHub personal access token. A token with no extra permissions is sufficient for public repositories.

    You can provide the token via:

    1. The GITHUB_TOKEN environment variable.
    2. The --github-token <TOKEN> CLI option.
    3. The configuration file.
  10. Install Fish completions for lychee

    master

    To enable tab-completion in Fish, generate the completion script using lychee --generate complete-fish.

    User Installation

    1. Create the completions directory:
      dir="${XDG_CONFIG_HOME:-$HOME/.config}/fish/completions"
      mkdir -p "$dir"
    2. Generate and save the file:
      lychee --generate complete-fish > "$dir/lychee.fish"
      Fish will automatically load these completions on the next shell start.

    System-wide Installation

    • On Linux:
      sudo lychee --generate complete-fish > /usr/share/fish/vendor_completions.d/lychee.fish
    • On macOS with Homebrew:
      lychee --generate complete-fish > $(brew --prefix)/share/fish/vendor_completions.d/lychee.fish
    dir="${XDG_CONFIG_HOME:-$HOME/.config}/fish/completions"
    mkdir -p "$dir"
    lychee --generate complete-fish > "$dir/lychee.fish"