DOCSEARCHbyAlgolia
Submit Docs

zsh-bench

repository·master·Indexed 21 days ago

https://github.com/romkatv/zsh-bench

A benchmarking tool for measuring user-visible latency in interactive zsh shells, including input lag, command lag, and first prompt lag. It includes a companion tool, human-bench, to simulate specific latencies for testing human perception. The tool supports benchmarking current configurations or predefined setups via Docker isolation and provides analysis on the performance impact of various zsh plugins, frameworks, and themes like Powerlevel10k.

Tokens
6.5K
Snippets
12
Records
22
Agent score
27%

What's inside zsh-bench

  1. Compare Git Prompt performance

    master

    Different git prompt implementations have significantly different performance profiles, especially in large repositories:

    • git-branch: Shows only the branch name; latency is constant regardless of repository size.
    • agnoster: Scans the entire repo for untracked files/changes; latency is linear to the number of files and can cause high command lag (🔴).
    • starship: An external binary prompt; suffers from high latency due to multiple fork+exec calls and high number of clones per command.
    • powerlevel10k: Uses gitstatus to scan repositories without invoking the git binary directly. It does not block the zsh prompt during scans, keeping command lag constant even in giant repositories.

  2. Understand zsh-bench latency measurement and thresholds

    master

    The benchmark results use normalized latency values based on a threshold where a 25ms first prompt lag is considered 50% and 100ms is 200%. Latencies are categorized using color-coded emojis:

    • 🟢 Green: Latency is under half of the threshold (e.g., <12.5ms for first prompt lag).
    • 🟡 Yellow: Latency is up to 50% or 100% (considered 'really good' and indistinguishable from zero).
    • 🟠 Orange: Latency is up to 200%.
    • 🔴 Red: Latency is above 200%.

    Key metrics measured include:

    • first prompt lag: Time to show the first prompt.
    • first cmd lag: Time to execute the first command.
    • cmd lag: Latency added to every command execution.
    • input lag: Latency added to every keystroke.

  3. Compare zsh configuration building blocks

    master

    The benchmark identifies several basic building blocks that can be composed to create a zsh configuration. Each block adds specific capabilities and associated latencies:

    CapabilityConfig ExampleNotes
    Pure zshno-rcsNo rc files; extremely fast.
    tmuxtmuxEnables tmux integration.
    CompsyscompsysEnables the completion system.
    Syntax Highlightingzsh-syntax-highlightingAdds syntax highlighting.
    Auto Suggestzsh-autosuggestionsAdds autosuggestions.
    Git Promptgit-branchShows current git branch with minimal lag.

    Latencies are additive. For example, the first prompt lag of a tmux+compsys configuration is the sum of the latencies of both tmux and compsys.

  4. Understand the risks of deferred initialization

    master

    Deferred initialization (using tools like zinit turbo mode or zsh-defer) attempts to reduce first cmd lag by loading features when the shell is idle. However, this approach has significant drawbacks:

    • Feature unavailability: You must be prepared to use zsh without syntax highlighting or autosuggestions for the initial period, as the shell is busy processing the first command before it can reach an idle state.
    • Unsafe deferral: Most features (environment variables, command definitions, widget behavior) cannot be safely deferred.
    • Syntax highlighting vs. Autosuggestions: While syntax highlighting is often safe to defer, autosuggestions must be initialized after syntax highlighting. Because autosuggestions change key behaviors, deferring them is considered unsafe.
    • Plugin breakage: Some plugins may fail if loaded from the Zsh Line Editor (ZLE) context. Relying on plugin-specific workarounds increases the risk of breakage during updates.

  5. Understand Powerlevel10k performance and Instant Prompt

    master

    Powerlevel10k is a highly configurable theme. Its configuration significantly impacts latency:

    • Standard config: Shows minimal info (e.g., CWD and git status) for low latency.
    • Full config: Enables many segments (up to 64), which increases command lag and input lag.

    Instant Prompt

    Powerlevel10k uses an Instant Prompt feature to render a functional prompt immediately upon terminal startup, while zsh continues to initialize in the background. This makes first prompt lag nearly constant and independent of your configuration size.

    Important Constraints:

    • While initializing, commands that read from the TTY or write to stdout/stderr can interfere with the prompt or capture user input intended for the shell.
    • Powerlevel10k handles this by redirecting standard input to /dev/null and standard output/error to a temporary file during initialization. This content is printed above the prompt once initialization completes.

  6. Understand zsh-bench latency measurements

    master

    zsh-bench measures several types of user-visible latency in milliseconds (ms). The output also includes detected shell capabilities (0 or 1).

    Detected Capabilities

    • creates_tty: Shell invokes tmux or screen to create its own TTY.
    • has_compsys: Shell initializes the completion system via compinit.
    • has_syntax_highlighting: zsh-syntax-highlighting is active.
    • has_autosuggestions: zsh-autosuggestions is active.
    • has_git_prompt: Git branch information is displayed in the prompt.

    Measured Latencies

    MetricDescription
    first_prompt_lag_msTime from shell start to the first prompt appearing. High values cause a delay when opening a terminal.
    first_command_lag_msTime from shell start to the moment the first interactive command starts executing.
    command_lag_msTime from pressing <Enter> on an empty line to the next prompt appearing.
    input_lag_msTime from pressing a key to the character appearing on the command line. High values make typing feel sluggish.
    exit_time_msTime to execute zsh -lic "exit". (Note: This is considered a baseline and not a measure of interactive latency).
    creates_tty=1
has_compsys=1
has_syntax_highlighting=1
has_autosuggestions=1
has_git_prompt=1
first_prompt_lag_ms=14.331
first_command_lag_ms=56.500
command_lag_ms=2.518
input_lag_ms=5.195
exit_time_ms=5.886

  7. Avoid using `time zsh -lic "exit"` as a performance benchmark

    master

    A common mistake when benchmarking zsh startup is using the command:

    time zsh -lic "exit"

    This measures exit_time_ms, which is often optimized by plugin managers using deferred initialization. While this results in a very low number, it does not accurately reflect the user experience. A configuration might have a very fast exit time but still suffer from high first prompt lag (the time you see an empty screen) or high first command lag (the time before you can actually type/run something).

    Focus instead on first prompt lag and first command lag for a meaningful measure of shell responsiveness.

  8. Compare zsh configuration optimization levels

    master

    The project provides several predefined zsh configurations representing different levels of optimization and complexity:

    • diy+: Replaces the prompt with powerlevel10k to minimize first prompt lag and command lag. It is self-bootstrapping (clones plugins automatically).
    • diy++: Adds compilation of large zsh files to wordcode to reduce first command lag.
    • diy++unsafe: Adds three aggressive optimizations to reduce first command lag by an additional 5% of the budget. Not recommended due to potential side effects.

    Risks of diy++unsafe optimizations

    1. Compiling .zshrc to wordcode: Can cause issues when manually editing and reverting .zshrc (e.g., using mv to restore a backup) because zsh uses file modification time to check if wordcode matches the source. It also prevents using aliases defined within the same file.
    2. Invoking compinit with -C: May prevent new tool completions from appearing after installation until the cache is manually deleted.
    3. Printing the first prompt before plugin installation: Can lead to messy terminal output or unexpected behavior if plugins are still being set up.

  9. Benchmark predefined zsh configurations

    master

    You can benchmark specific predefined configurations using Docker isolation. This allows you to test different setups without affecting your host environment.

    Requirements:

    • docker must be installed.

    Usage: Pass --isolation docker followed by -- and the name of the configuration directory found in the configs folder of the repository.

    Alternatively, use --isolation user to run the benchmark as the user zsh-bench on the host machine.

    ~/zsh-bench/zsh-bench --isolation docker -- <name> [name]..

  10. Structure your .zshrc for Powerlevel10k Instant Prompt

    master

    To use Powerlevel10k's Instant Prompt correctly and avoid issues with terminal output or input redirection, structure your .zshrc into three distinct sections:

    1. Fast/Interactive Section: Commands that read from/write to the TTY or take unpredictable amounts of time (e.g., starting tmux, cloning git repos, or simple print statements). These must be very fast.
    2. Instant Prompt Activation: The exact command to activate the feature.
    3. Bulk Initialization Section: The majority of your config (e.g., compinit, sourcing plugins). These commands must not read from the TTY or write to stdout/stderr. If you must interact with the TTY here, use the $TTY variable for 'invisible' writes (like setting cursor shapes or terminal titles).
    # 1. Fast/Interactive Section
if [[ -z ${TMUX+X}${ZSH_SCRIPT+X}${ZSH_EXECUTION_STRING+X} ]]; then
  exec tmux
fi

if [[ ! -e ~/zsh-autosuggestions ]]; then
  print -r -- 'installing zsh-autosuggestions ...'
  git clone --depth=1 https://github.com/zsh-users/zsh-autosuggestions.git ~/zsh-autosuggestions
fi

print -Pr -- 'Hello, %n. Today is %D{%A}.'

# 2. Activate Instant Prompt (MUST be this exact command)
if [[ -r "${XDG_CACHE_HOME:-$HOME/.cache}/p10k-instant-prompt-${(%):-%n}.zsh" ]]; then
  source "${XDG_CACHE_HOME:-$HOME/.cache}/p10k-instant-prompt-${(%):-%n}.zsh"
fi

# 3. Bulk Initialization (No TTY reading/writing)
autoload -Uz compinit && compinit
source ~/zsh-autosuggestions/zsh-autosuggestions.zsh

  11. Debug and validate zsh-bench results

    master

    The zsh-bench tool includes debugging utilities to help you inspect and validate benchmark runs. To debug effectively, you should first run a single iteration of the benchmark and save the temporary data to a scratch directory.

    Important: Do not resize your terminal after running the benchmark, as this will prevent the TTY replay from working correctly.

    Debugging Workflow

    1. Run a single iteration: Use --iters 1 and specify a --scratch-dir to retain data.
    2. Replay the TTY: Use the dbg/replay tool to watch exactly what happened during the benchmark.
    3. Analyze timing: Use dbg/timeline to see a timestamped table of raw writes to the TTY, or use --pause-at-ms with the replay tool to inspect the state at specific millisecond intervals.
    # 1. Run one iteration and save data
~/zsh-bench/zsh-bench --iters 1 --scratch-dir /tmp/zsh-bench

# 2. Replay the session
~/zsh-bench/dbg/replay --scratch-dir /tmp/zsh-bench

# 3. (Optional) Replay with specific timing/speed
~/zsh-bench/dbg/replay --scratch-dir /tmp/zsh-bench --delay-multiplier 10 --max-delay-ms 1000

# 4. (Optional) View a timestamped timeline of TTY writes
~/zsh-bench/dbg/timeline --scratch-dir /tmp/zsh-bench

# 5. (Optional) Pause replay at a specific millisecond
~/zsh-bench/dbg/replay --scratch-dir /tmp/zsh-bench --pause-at-ms 10.149