llm-checker

LLM Checker is composed of a CLI interface (enhanced_cli.js) that interacts with a main engine (src/index.js). The engine coordinates three primary subsystems:

1. Hardware Detection: Analyzes CPU, RAM, and GPU to determine hardware tiers and performance capabilities.
2. AI Model Selector: Uses a mathematical 5-factor scoring algorithm to rank models based on hardware and user preferences.
3. Model Database: A collection of 15+ models containing technical characteristics and scoring data.

The system also includes an Ollama Integration for client-side interaction and scraping.

You can use calibration routing policies generated by calibrate --policy-out to steer model selection.

• Use the --calibrated [file] flag to specify a policy file. If the file is omitted, it defaults to ~/.llm-checker/calibration-policy.{yaml,yml,json}.
• The --policy <file> flag takes precedence over --calibrated.

Resolution Precedence:

1. --policy (explicit)
2. --calibrated (explicit file or default discovery)
3. deterministic selector fallback

llm-checker recommend --calibrated --category coding
llm-checker recommend --calibrated ./calibration-policy.yaml --category reasoning
llm-checker ai-run --calibrated --category coding --prompt "Refactor this function"
llm-checker ai-run --policy ./calibration-policy.yaml --prompt "Summarize this report"

On the first run, llm-checker automatically copies a packaged snapshot (models.db) to ~/.llm-checker/models.db. To maintain this database after the initial setup, use the following commands:

• Use llm-checker sync to update your local Ollama copy.
• Use llm-checker registry-sync to refresh the multi-source registry in your local copy (which includes artifacts from Hugging Face, Ollama, and GPT4All).

Running llm-checker with no arguments opens an interactive TTY terminal panel. This is useful for manual exploration without remembering specific flags.

Navigation:

• Use up/down arrows to select a command.
• Type / to open the full command list.
• Press Enter to execute the selected command.
• You can add optional extra flags before running a command (e.g., --json --limit 5).

Use architecture-specific flags and environment variables to ensure LLM Checker utilizes the correct backend (Metal for Apple Silicon, CUDA for NVIDIA, or OpenCL for AMD).

# Apple Silicon (M1/M2/M3)
export LLAMA_METAL=1

# NVIDIA GPUs
llm-checker check --gpu-acceleration cuda

# AMD GPUs
export LLAMA_OPENCL=1

LLM Checker includes a built-in Model Context Protocol (MCP) server that allows Claude Code and other MCP-compatible assistants to analyze your hardware and manage local models.

To set it up, you can use one of the following methods:

Method 1: Global Install (Recommended)

1. Install the package globally: npm install -g llm-checker
2. Add it to Claude Code: claude mcp add llm-checker -- llm-checker-mcp

Method 2: Using npx (No global install required) claude mcp add llm-checker -- npx llm-checker-mcp

Method 3: Automatic Setup Command You can generate the exact command for your environment by running: llm-checker mcp-setup

After running the command, restart Claude Code to complete the integration.

# Install globally first
npm install -g llm-checker

# Add to Claude Code
claude mcp add llm-checker -- llm-checker-mcp

To improve the model's accuracy or generate a new model, you can collect performance benchmarks and then run the training pipeline. If your environment uses python3 instead of python, execute the scripts directly from the ml-model directory.

# Using npm scripts
npm run benchmark
npm run train-ai

# Using python3 directly if 'python' is not found
cd ml-model
python3 python/benchmark_collector.py
python3 python/train_model.py

You can install LLM Checker globally via npm or run it directly using npx.

Requirements:

• Node.js 16 or higher
• Ollama installed for running models

Note on optional dependencies: If database commands report sql.js is missing, reinstall with the --include=optional flag to ensure the SQLite-powered catalog is available.

Termux (Android) Installation:

pkg update
pkg install ollama
npm install -g llm-checker

# Install globally
npm install -g llm-checker

# Or run directly with npx
npx llm-checker hw-detect

# If database commands fail due to missing sql.js:
npm install -g llm-checker --include=optional

For developers needing coding assistance, you can compare specific models, browse models in the coding category, or run a quick coding session.

Use the --calibrated flag with ai-run to enable a calibrated routing policy (auto-discovery) which considers model specialization, context length, and performance benchmarks.

# Compare coding-specific models
llm-checker ai-check --models codellama:7b deepseek-coder:6.7b starcoder:7b

# Browse all coding models
llm-checker list-models --category coding

# Quick coding session
llm-checker ai-run --prompt "Help me write a Python function to parse JSON"

# Quick coding session with calibrated routing policy (auto-discovery)
llm-checker ai-run --calibrated --category coding --prompt "Help me write a Python function to parse JSON"