Recursive Language Models (RLM)

Based on benchmark performance and interpretation, choose your approach based on context size and task requirements:

• Direct Completion: Use for short contexts that fit comfortably within the model's native context window.
• Recursive Language Models (RLM): Use when exact computation over a large context is required. RLM keeps the bulk of the context outside of the model prompts, using Python to perform aggregation and management, which prevents context overflow and manages costs.

The max_depth parameter controls the potential for recursive child-model calls. However, RLM does not always force recursion.

In many successful runs, the system may operate entirely at the root REPL (max_depth_reached=0). This demonstrates that RLM's value is derived not just from child-model calls, but from its ability to externalize context and perform local computation (e.g., via Python) to handle large datasets.

RLM operates by storing context as a variable in a Python REPL environment. The root LM receives the query and instructions, then explores the context using Python code.

Key Characteristics:

• Persistence: REPL variables persist between iterations.
• Isolation: Each local step executes in an isolated subprocess. Print output is isolated per step.
• Safety: Uses RestrictedPython and subprocesses for defense-in-depth. Arbitrary module imports are blocked; only re, json, math, datetime, and collections are exposed.
• Timeouts: Non-terminating local code is terminated by repl_timeout. Time spent waiting for model subcalls does not count against the local Python timeout.

RLM enables language models to process extremely long contexts (100k+ tokens) by storing the context as a Python variable in a spawned worker process (REPL) instead of including it directly in the prompt. This approach:

• Allows the LM to recursively explore, search, and partition the context.
• Reduces model token usage on large-context tasks.
• Avoids "context rot" (performance degradation associated with very long prompts).

Important Implementation Note: Because RLM uses a spawned worker process for isolated REPL execution, all executable Python scripts must use the standard if __name__ == "__main__": entry-point guard to ensure compatibility with Python's multiprocessing on spawn-based platforms.

if __name__ == "__main__":
    # Your RLM logic here
    pass

To use RLM, import the RLM class, initialize it with a model name, and call .complete() with your query and the long context document. The context is stored as a variable rather than being injected into the prompt string.

from rlm import RLM

def main():
    # Initialize with any LLM
    rlm = RLM(model="gpt-5-mini")

    # Process long context
    result = rlm.complete(
        query="What are the main themes in this document?",
        context=long_document,
    )
    print(result)

if __name__ == "__main__":
    main()

Since the package is not yet published to PyPI, you must install it directly from the GitHub repository.

1. Clone the repository.
2. Install in editable mode using pip install -e ..
3. Alternatively, install with development dependencies using pip install -e ".[dev]".

Requirements:

• Python 3.9 or higher
• An API key for your chosen LLM provider (OpenAI, Anthropic, etc.) or a local model setup (Ollama, llama.cpp, etc.).

# Clone the repository
git clone https://github.com/grishahq/recursive-llm.git
cd recursive-llm

# Install in editable mode
pip install -e .

# Or install with dev dependencies
pip install -e ".[dev]"

The RunState class manages the per-invocation state shared within a single RLM recursion tree. It tracks usage statistics, budget constraints, iteration counts at specific depths, and a chronological trajectory of events occurring during the run. It is designed to be thread-safe for recording events and node IDs.

from rlm.run_state import RunState
from rlm.budget import RunBudget
from rlm.stats import UsageTracker

# Initialize RunState with required components
state = RunState(
    usage=UsageTracker(),
    budget=RunBudget(),
    event_handler=lambda event: print(f"Event: {event.kind}")
)

RLM supports 100+ providers via LiteLLM. You can configure keys in two ways:

1. Environment Variables: Create a .env file (copy from .env.example) and add your keys:

   OPENAI_API_KEY=sk-...
   DEEPSEEK_API_KEY=...

2. Directly in Code: Pass the api_key argument to the RLM constructor.

Hybrid Models: Use LiteLLM provider prefixes (e.g., deepseek/deepseek-v4-flash) to allow RLM to automatically select the correct API key for different models in a hybrid setup.

# Using environment variables (via .env)
rlm = RLM(model="gpt-5-mini", recursive_model="deepseek/deepseek-v4-flash")

# Passing key directly
rlm = RLM(model="gpt-5-mini", api_key="sk-...")

RLM uses a depth-based model to control the complexity and capability of the agent. The max_depth parameter determines how many levels of recursion are allowed:

1. Depth 0: The root agent has access to a REPL (Python execution environment) but cannot make subcalls to other LLMs via the RLM interface. It can only execute code or return a final answer.
2. Depth 1: The agent can make subcalls to a plain Language Model (LM) using _call_leaf, which returns a direct answer without a REPL loop.
3. Depth 2: The agent can instantiate a child RLM instance. The boundary of this child RLM falls back to a plain LM call.

This hierarchy allows you to control the computational budget and complexity of the reasoning process.