whichllm

The tool follows a specific data pipeline:

1. Model fetching: Pulls text-generation and GGUF-filtered models from HuggingFace. Vision models are included when using --profile vision or any.
2. Benchmark sources: Merges live data (LiveBench, Artificial Analysis, Aider) with frozen tiers (Open LLM Leaderboard v2, Chatbot Arena ELO).
3. Benchmark evidence: Uses five resolution levels: direct (exact match), variant (suffix-stripped), base_model, line_interp (size-aware interpolation), and self_reported (heavily discounted).

Cache Locations:

• Default: ~/.cache/whichllm/
• If $XDG_CACHE_HOME is set to an absolute path: $XDG_CACHE_HOME/whichllm/

Cache TTL:

• models.json: 6 hours
• benchmark.json: 24 hours

The whichllm ranker does not simply pick the largest model that fits in memory. Instead, it calculates a composite score (capped at 0..100) to identify the most usable model for your specific hardware. The ranking prioritizes a balance of model quality, benchmark evidence, hardware fit, and estimated speed.

Key factors influencing the score include:

• Model Metadata: Parameters, quantization type, and lineage.
• Hardware Fit: VRAM/RAM usage and estimated tokens per second (tok/s).
• Benchmark Evidence: Strength and source of evaluation data.
• Usability: Speed estimates and quantization penalties.

By default, whichllm includes full-GPU, partial-offload, and CPU-only candidates. To restrict results to models that fit entirely within your GPU VRAM, use one of the following:

• --gpu-only: Only models that fit in GPU.
• --fit gpu: Only models that fit in GPU.
• --fit full-gpu: Only models that fit entirely in GPU.

If no results are shown, try lowering the context length or using a smaller quantization.

whichllm --gpu-only
whichllm --fit gpu
whichllm --fit full-gpu

To simulate a workstation with multiple GPUs, you can pass the --gpu flag multiple times, use a comma-separated list, or use count shorthand.

Note: The --vram flag is only supported for a single simulated GPU. For multi-GPU setups, use known GPU names so whichllm can resolve the VRAM from its internal database.

Use speed filters to remove slow models from your recommendations:

• --speed usable: Filters for models $\ge$ 10 tok/s.
• --speed fast: Filters for models $\ge$ 30 tok/s.
• --min-speed <N>: Sets an exact floor for generation speed (e.g., --min-speed 4).

whichllm --speed usable
whichllm --speed fast
whichllm --min-speed 10

The output of whichllm can be customized based on the desired surface:

• Standard Tables: Renders hardware panels and recommendation tables using Rich. Shows memory, estimated speed, fit type, and published date.
• Detailed View: Use the --details flag to switch from general recommendations to download-oriented metadata.
• JSON Output: Use the appropriate subcommand to get machine-readable JSON for ranking, plan, or upgrade tasks.

Speed Metadata Symbols:

• ~: Indicates an estimate with a range.
• ?: Indicates a low-confidence estimate that is sensitive to the backend used.

If you want to plan for a future purchase or if automatic detection is unavailable, you can simulate specific hardware using the --gpu flag.

• Simulate a single GPU: --gpu "RTX 4090"
• Simulate multiple GPUs: --gpu "RTX 4090" --gpu "RTX 3090"
• Specify VRAM for a simulated GPU: --gpu "RTX 5060 Ti" --vram 16
• Simulate Apple Silicon: whichllm hardware --gpu "Apple M3 Max"

Note: --vram only applies to one GPU. For multi-GPU simulation, use known GPU names and omit --vram.

Overriding iGPU/Unified Memory: For detected integrated graphics or unified memory systems, you can manually override usable memory and bandwidth: whichllm --vram 8 --bandwidth 68

whichllm --gpu "RTX 4090"
whichllm --gpu "RTX 5060 Ti" --vram 16
whichllm --gpu "RTX 4090" --gpu "RTX 3090"
whichllm --vram 8 --bandwidth 68

Use whichllm snippet [MODEL_NAME] to print a standalone Python script for manual use instead of executing it. This is ideal when you want to integrate model loading code into your own project.

Behavior:

• If MODEL_NAME is provided, it searches for the model.
• If no model is provided, it picks the most-downloaded GGUF model from the fetched list (note: this differs from run, which uses hardware ranking).

The output includes a suggested uv run --no-project command containing all necessary --with dependencies for manual execution.

Use the --gpu flag to simulate different hardware configurations, including specific cards or multi-GPU setups. This is useful for planning hardware purchases.

• Single GPU: --gpu "RTX 4090"
• Specific variant: --gpu "RTX 5060 16"
• Multi-GPU (string syntax): --gpu "2x RTX 4090"
• Multi-GPU (multiple flags): --gpu "RTX 4090" --gpu "RTX 3090"
• Multi-GPU (comma separated): --gpu "RTX 4090, RTX 3090"

Use whichllm run to start an interactive chat session with a model. If you provide a MODEL_NAME, it searches for an exact ID, suffix match, or term match. If omitted, it automatically ranks models for your current hardware and selects the top result.

Requirements:

• uv must be in your PATH.

Execution Flow:

1. Loads models from cache or HuggingFace.
2. Selects a model and quantization.
3. Generates a temporary Python script.
4. Runs the script using uv run --no-project with required dependencies added via --with flags.
5. Deletes the script after the session ends.

This method keeps your local project environment clean as dependencies are not added to pyproject.toml.

You can install whichllm using uv, Homebrew, or pip depending on your preferred package manager.

Using uv

For a one-time run:

uvx whichllm@latest

For continuous use:

uv tool install whichllm
uv tool upgrade whichllm

Using Homebrew

brew install andyyyy64/whichllm/whichllm

Using pip

pip install whichllm