Deep Research Agent

repository·main·Indexed 25 days ago

https://github.com/skyworkai/deepresearchagent

A self-evolution protocol and runtime for LLM-based agent systems. It features the Resource Substrate Protocol Layer (RSPL) for managing agents, tools, and memory, and the Self Evolution Protocol Layer (SEPL) for iterative improvement. The system includes modular components for agents, environments, and optimizers (such as TextGrad, GRPO, and Reinforce++), and supports an iterative loop of Act, Observe, Optimize, and Remember.

Tokens: 27.4K
Snippets: 31
Records: 176
Agent score: 85%

What's inside Deep Research Agent

Overview of Deep Research Agent Architecture
main
Deep Research Agent is a self-evolving protocol and engineering runtime for LLM agent systems. It decouples the definition of resources from the evolution process using two primary layers:
- RSPL (Resource Substrate Protocol Layer): Models prompt, agent, tool, environment, and memory as protocol-registered resources with explicit state, lifecycle, and versioned interfaces.
- SEPL (Self Evolution Protocol Layer): Defines closed-loop operator interfaces for proposing, assessing, and committing improvements, featuring auditable lineage and rollback capabilities.
The system follows a self-evolution loop consisting of:
1. Act: Agent generates actions/outputs using LLM and tools.
2. Observe: Records results, trajectories, intermediate info, and environmental feedback.
3. Optimize: Uses optimizers (e.g., Reflection, GRPO, Reinforce++) to convert feedback into reusable improvements.
4. Remember: Writes session events and insights into the memory system for future reuse.
https://github.com/skyworkai/deepresearchagent/blob/main/README_zh.md
Overview of Deep Research Agent
main
Deep Research Agent is a self-evolution protocol and runtime for LLM-based agent systems. It decouples the evolution of resources from the evolution process itself using two primary layers:
- RSPL (Resource Substrate Protocol Layer): Models prompts, agents, tools, environments, and memory as protocol-registered resources with explicit state, lifecycle, and versioned interfaces.
- SEPL (Self Evolution Protocol Layer): Specifies a closed-loop operator interface to propose, assess, and commit improvements with auditable lineage and rollback capabilities.
The system supports an iterative loop of Act (producing actions), Observe (capturing outcomes/traces), Optimize (updating prompts/solutions via optimizers), and Remember (persisting insights to memory).
https://github.com/skyworkai/deepresearchagent/blob/main/README.md
Core Modules of Deep Research Agent
main
The project is organized into several functional modules located in the src/ directory:
- Agents (src/agent/): Logic for planning, tool calling, and domain-specific execution.
- Tools (src/tool/): Capabilities exposed to agents (workflows and default tools).
- Environments (src/environment/): Stateful interfaces like file systems, backtesting environments, or browsers.
- Memory (src/memory/): Systems for session/event memory, summaries, and long-term state.
- Optimizers (src/optimizer/): Algorithms (Reflection, GRPO, Reinforce++) that transform feedback into updates.
- Tracing & Versioning (src/tracer/, src/version/): Tools for recording trajectories and managing versioned iteration artifacts.
- Configuration (configs/, src/config/): MMEngine-style compositional configuration for assembling components.
https://github.com/skyworkai/deepresearchagent/blob/main/README_zh.md
Core Building Blocks of Deep Research Agent
main
The system is composed of several modular components located in the src/ directory:
- Agents (src/agent/): Runtime logic for planning and tool-calling.
- Tools (src/tool/): Callable capabilities (workflow and default tools).
- Environments (src/environment/): Stateful interfaces (e.g., filesystem, browser, trading backtest).
- Memory (src/memory/): Session and event memory systems for long-term state.
- Optimizers (src/optimizer/): Algorithms for self-improvement (e.g., reflection, GRPO, Reinforce++).
- Tracing & Versioning (src/tracer/, src/version/): Tools for recording trajectories and managing iterative artifacts.
- Config System (configs/, src/config/): MMEngine-style configurations for composing the stack.
https://github.com/skyworkai/deepresearchagent/blob/main/README.md
Configure fonts for the 3D GraphML Viewer
main
To ensure proper font rendering (especially for Chinese characters), you must provide a font file:
1. Place a font file named font.ttf in the assets directory.
2. Alternatively, modify the CUSTOM_FONT constant in the source code to point to a different font file.
https://github.com/skyworkai/deepresearchagent/blob/main/src/tool/esg_tools/lightrag/tools/lightrag_visualizer/README-zh.md
Access LightRAG API documentation
main
When the LightRAG server is running, you can access interactive API documentation via Swagger UI or ReDoc at the following local addresses:
- Swagger UI: http://localhost:9621/docs
- ReDoc: http://localhost:9621/redoc
You can use these interfaces to test endpoints or use the provided curl commands.
https://github.com/skyworkai/deepresearchagent/blob/main/src/tool/esg_tools/lightrag/api/README-zh.md
Deploy LightRAG Server with Docker Compose
main
You can deploy LightRAG using Docker without cloning the full repository by creating a docker-compose.yml file.
Example docker-compose.yml:
```
services:
  lightrag:
    container_name: lightrag
    image: ghcr.io/hkuds/lightrag:latest
    ports:
      - "${PORT:-9621}:9621"
    volumes:
      - ./data/rag_storage:/app/data/rag_storage
      - ./data/inputs:/app/data/inputs
      - ./config.ini:/app/config.ini
      - ./.env:/app/.env
    env_file:
      - .env
    restart: unless-stopped
    extra_hosts:
      - "host.docker.internal:host-gateway"
```
Deployment Steps:
1. Create a working folder and move into it.
2. Create the docker-compose.yml file.
3. Create a .env file from env.example and configure your models.
4. Run docker compose up.
To rebuild after pulling a new image, use docker compose up --build.
```
docker compose up
```
https://github.com/skyworkai/deepresearchagent/blob/main/src/tool/esg_tools/lightrag/api/README.md

Use the TextGradOptimizer class for flexible prompt optimization

main

For scenarios requiring more control or custom optimization logic, use the TextGradOptimizer class. This allows you to manually manage the optimization lifecycle and inspect optimized variables in detail.

from src.optimizers.textgrad_optimizer import TextGradOptimizer

async def main():
    # ... Initialize Agent ...
    agent = acp.get_info("tool_calling").instance
    
    # Create optimizer instance
    optimizer = TextGradOptimizer(
        agent=agent,
        log_dir=config.workdir
    )
    
    # Execute optimization
    await optimizer.optimize(
        task="Your task description here",
        files=[],
        optimization_steps=3,
        optimizer_model="gpt-4o"
    )
    
    # Retrieve and inspect optimized variables
    optimized_vars = optimizer.get_optimized_variables()
    for tg_var in optimized_vars:
        print(f"Variable description: {tg_var.role_description}")
        print(f"Optimized value: {tg_var.value}")
    
    # Run Agent with optimized prompts
    result = await agent.ainvoke(task="Your task", files=[])

import asyncio
asyncio.run(main())

Install LightRAG Server and WebUI

main

You can install LightRAG via PyPI or from source. To enable API support, ensure you include the [api] extra.

Via PyPI:

pip install "lightrag-hku[api]"

From Source:

# Clone the repository
git clone https://github.com/HKUDS/lightrag.git

# Change to the repository directory
cd lightrag

# Create a Python virtual environment if necessary
# Install in editable mode with API support
pip install -e ".[api]"

pip install "lightrag-hku[api]"

Run the LightRAG API server
main
You can start the LightRAG server using the lightrag-server command. The server supports various backends including ollama, lollms, openai, and azure_openai.
Common usage patterns:
- Default (Ollama): Runs with default settings assuming Ollama is running locally. lightrag-server
- With Authentication: Protects the server with an API key. lightrag-server --key my-key
- With OpenAI: Requires LLM_BINDING=openai and EMBIEDDING_BINDING=openai in .env or via CLI.
To see all available options, use lightrag-server --help.
https://github.com/skyworkai/deepresearchagent/blob/main/src/tool/esg_tools/lightrag/api/README-zh.md
Use LightRAG Query Modes via Ollama Emulation
main
LightRAG provides an Ollama-compatible interface, allowing tools like Open WebUI to access it as a chat model (named lightrag:latest).
Query Prefixes: You can control the RAG query mode by prefixing your message with one of the following:
- /local
- /global
- /hybrid (Default)
- /naive
- /mix
- /localcontext
- /globalcontext
- /hybridcontext
- /naivecontext
- /mixcontext
Special Commands:
- /bypass: Passes the query directly to the underlying LLM, including chat history (bypasses RAG).
- /context: Returns only the context information prepared for the LLM without generating a final response.
Adding User Prompts: To guide the LLM on how to process retrieved results (e.g., formatting) without affecting the retrieval phase, append a prompt in square brackets to the prefix:
```
/mix[Use mermaid format for diagrams] Please draw a character relationship diagram for Scrooge
```
https://github.com/skyworkai/deepresearchagent/blob/main/src/tool/esg_tools/lightrag/api/README.md
Use interactive controls in the 3D GraphML Viewer
main
The viewer provides several ways to navigate and interact with the 3D graph:
Camera Movement
Use the following keys to move the camera:
- W: Move forward
- S: Move backward
- A: Move left
- D: Move right
- Q: Move up
- E: Move down
Viewpoint Control
- Right Mouse Button: Hold and drag to rotate the view.
Node Interaction
- Hover: Highlights the node.
- Click: Selects the node.
https://github.com/skyworkai/deepresearchagent/blob/main/src/tool/esg_tools/lightrag/tools/lightrag_visualizer/README-zh.md

Deep Research Agent

What's inside Deep Research Agent

Camera Movement

Viewpoint Control

Node Interaction