Deep Agents

Deep Agents Evals is an end-to-end behavioral evaluation suite designed for the Deep Agents SDK. It evaluates agents by running them against real LLMs and capturing their full execution trajectory, including tool calls, file mutations, and final responses. The suite scores agents based on two primary metrics: correctness and efficiency.

Key components include:

• EVAL_CATALOG.md: Contains the full list of available evaluations and categories.
• MODEL_GROUPS.md: Contains the model catalog used by the evaluation workflow.
• Harbor Integration: Supports running sandboxed benchmarks, such as Terminal Bench 2.0.

Deep Agents Code (dcode) is a pre-built coding agent for the terminal that works with any LLM supporting tool calling. Key features include:

• Interactive TUI: A rich terminal interface with streaming responses.
• Conversation Resume: Ability to continue previous sessions.
• Web Search: Grounding responses in live web data.
• Remote Sandboxes: Isolated execution environments (supports LangSmith, AgentCore, Daytona, Modal, Runloop, etc.).
• Persistent Memory: Context retention across different conversations.
• Custom Skills: Extension via custom slash commands.
• Headless Mode: Non-interactive execution for CI/CD and scripting.
• Human-in-the-loop: Manual approval/rejection of tool calls.

Deep Agents provides a 'batteries-included' environment for long-horizon, multi-step tasks. Key features include:

• Sub-agents: Delegate tasks to agents with isolated context windows.
• Filesystem: Read, write, edit, or search via pluggable local, sandboxed, or remote backends.
• Context management: Summarize long threads and offload tool outputs to disk.
• Shell access: Run commands in a chosen sandbox.
• Persistent memory: Pluggable state and store backends for cross-session recall.
• Human-in-the-loop: Approve, edit, or reject tool calls before execution.
• Skills: Reusable behaviors that can be loaded on demand.
• Tools: Support for custom functions or any MCP (Model Context Protocol) server.

Deep Agents is model-agnostic and works with any LLM that supports tool calling (frontier APIs, open-weight models, or local models via Ollama, vLLM, etc.).

Deep Agents is an extensible agent harness built on LangChain and LangGraph. It provides long-horizon features like filesystem/shell backends, planning, context management, skills, persistent memory, human approval, and subagents.

The repository is organized as a monorepo of independently versioned Python packages located in libs/. There is no root pyproject.toml; instead, each package manages its own dependencies and lifecycle via its own pyproject.toml, uv.lock, and Makefile.

The cuml-machine-learning skill provides a scikit-learn-compatible API for running machine learning tasks on NVIDIA GPUs. This is ideal for datasets with 10K+ rows where GPU acceleration provides significant speedups.

When to use this skill:

• Classification: Predict categories, detect fraud, or classify text.
• Regression: Forecast values, predict prices, or estimate quantities.
• Clustering: Segment customers or group documents.
• Dimensionality Reduction: Visualize high-dimensional data or compress features.
• Preprocessing: Feature engineering on large datasets.

Deep Agents is an open-source, opinionated agent harness designed for long-horizon, multi-step work. It is built on top of LangGraph and is model-agnostic, working with any LLM that supports tool calling (frontier, open-weight, or local via Ollama, vLLM, etc.).

Core Features:

• Sub-agents: Delegate tasks to agents with isolated context windows.
• Filesystem: Read, write, edit, or search via pluggable local, sandboxed, or remote backends.
• Context management: Summarize long threads and offload tool outputs to disk.
• Shell access: Run commands in a chosen sandbox.
• Persistent memory: Pluggable state and store backends for cross-session recall.
• Human-in-the-loop: Approve, edit, or reject tool calls before execution.
• Skills: Reusable behaviors loaded on demand.
• Tools: Support for custom functions or any MCP server.

The unified evals CI job runs multiple models through a fixed battery of benchmarks to produce a cross-model comparison, including a leaderboard and a radar chart. It evaluates models as 'deep agents' across distinct capability axes using identical tasks, harnesses, and scoring to ensure comparability.

Key capability axes include:

• autonomous
• conversation
• research
• context (available but not in the default set)

Ralph is an autonomous looping pattern designed for simple context management. Instead of managing complex conversation histories or worrying about token limits, each iteration starts with fresh context.

Progress is tracked via the filesystem and git, which act as the agent's memory and worklog. The core logic follows a simple loop: the agent receives a prompt, performs work (creating files, etc.), and then the loop repeats with the same prompt but with the updated filesystem state.