llmtrim

The llmtrim-tray is a system-tray / menu-bar application that provides a visual dashboard of llmtrim's compression savings. It displays aggregate savings, a per-agent breakdown (e.g., Claude Code, Codex, Gemini), and savings trends via a popover window.

Supported Platforms:

• macOS
• Windows
• Linux (X11/Wayland via WebKitGTK and AppIndicator).

Linux Runtime Dependencies:

• libwebkit2gtk-4.1
• libayatana-appindicator3

The llmtrim-core engine is written in Rust but is available for other languages:

• Python, Ruby, Swift, Kotlin: Available via UniFFI through the llmtrim-uniffi package.
• JavaScript / TypeScript: Available via WebAssembly as the @llmtrim/js npm package (see llmtrim-wasm).
• CLI: Use the llmtrim crate for a drop-in compressing proxy.

The llmtrim-uniffi package provides UniFFI bindings over llmtrim-core, allowing you to perform LLM request compression natively within your application's process. Unlike the llmtrim proxy which intercepts network traffic via HTTPS_PROXY, these bindings require you to explicitly call compress() on your request body and then send the resulting JSON using your own HTTP client.

Use this approach when:

• You are running in sandboxed or serverless environments where setting a process-wide HTTPS_PROXY is difficult.
• You use certificate-pinned clients that reject MITM (Man-in-the-Middle) CA certificates.
• You want to avoid the latency of an extra network hop or the complexity of running a sidecar process.

llmtrim uses a quality-gated approach to compression, meaning it only applies lossy cuts if the model's answer quality is maintained.

Key modes include:

• auto: The default mode. It is quality-gated to ensure high-quality responses.
• aggressive: Accepts more lossy cuts, provided the quality gate still holds.
• safe: A preset for users who require a byte-faithful round-trip (no lossy information reduction).

Note: While 'lossless' stages preserve the information the model reads (e.g., via abbreviation legends), the engine does not transform the model's response back to its original form.

llmtrim differs from other tools by compressing both input and output while leaving the cached prefix untouched to maintain prompt cache stability. It scores based on whether the answer survives the cut rather than just tokens removed.

Input Compression Comparison:

• llmtrim auto: ~25% reduction, ~5ms overhead, high quality.
• llmtrim aggressive: ~28% reduction, ~5ms overhead, high quality.
• Headroom: ~24% reduction, ~0.9s overhead (ML-based).
• leanctx / LLMLingua-2: 52-81% reduction, ~6s overhead, significantly lower quality.
• entroly: 80-89% reduction, <1ms overhead, significantly lower quality.

Output Compression Comparison:

• llmtrim output_terse: ~69% reduction, ~19 tokens overhead.
• caveman: ~80% reduction, ~949 tokens overhead (due to large system prompt).

The tray application consumes data via a specific contract designed to protect privacy. The AgentAggregate type and dashboard queries are designed to drop sensitive columns such as project paths and session_name.

Key data points provided to the UI include:

• % saved: The gross savings percentage (the primary metric).
• bill_micros: The billing amount, provided as an i64 in microseconds (divide by 1e6 for standard currency units).
• trend: An array of data points used to draw an SVG-path sparkline.
• has_savings_data: A boolean flag to handle pre-meter NULLs (if false, the UI should show — instead of 0%).

The llmtrim-tray is designed with a strict security model to protect user privacy:

• Data Minimization: The Tauri IPC contract only carries aggregate numbers. It explicitly excludes sensitive metadata such as project, session_name, mcp_server, or tool_name.
• Error Masking: Errors passed to the webview are stripped of filesystem paths; full error details are sent only to stderr.
• Sandboxing: The Content Security Policy (CSP) sets connect-src 'none'. The capability allowlist excludes shell, fs, http, updater, and clipboard.
• Read-Only Access: The application opens the ledger in read-only mode.

You can use the llmtrim engine directly in your applications without a proxy. Compression runs in-process and does not require extra model calls.

# Rust
cargo add llmtrim-core

# Python
pip install llmtrim

# Ruby
gem install llmtrim

# Kotlin
implementation("io.github.fkiene:llmtrim:0.13.7")

# Swift
SwiftPM fkiene/llmtrim-swift ≥ 0.1.8

# JS / TS
@llmtrim/js (WASM)

You can expose the llmtrim engine to a Model Context Protocol (MCP) client. Running llmtrim mcp starts an MCP server over stdin/stdout providing three tools:

• llmtrim_compress
• llmtrim_compress_text
• llmtrim_stats

Registration:

• To register with Claude Code: llmtrim mcp install
• To get the configuration for other clients: llmtrim mcp install --print