DOCSEARCHbyAlgolia
Submit Docs

Auto-Deep-Research

repository·main·Indexed 23 days ago

https://github.com/hkuds/auto-deep-research

An open-source, cost-effective alternative to OpenAI's Deep Research built on the AutoAgent framework. It provides a fully-automated AI assistant for deep research tasks, supporting multiple LLM providers via LiteLLM and utilizing Docker for containerized code and web browser environments. Features include a CLI for interactive agent sessions, a MetaChain API for executing agents and tools, and support for authenticated web automation via cookie JSON imports.

Tokens
3.1K
Snippets
7
Records
17
Agent score
82%

What's inside Auto-Deep-Research

  1. Install Auto-Deep-Research

    main

    You can install Auto-Deep-Research using Conda or Docker.

    Conda Installation

    Create a new environment with Python 3.10, clone the repository, and install in editable mode:

    conda create -n auto_deep_research python=3.10
conda activate auto_deep_research
git clone https://github.com/HKUDS/Auto-Deep-Research.git
cd Auto-Deep-Research
pip install -e .

    Docker Installation

    Auto-Deep-Research uses Docker to containerize the agent-interactive environment. Ensure Docker is installed on your machine. The tool will automatically pull the appropriate pre-built image based on your machine's architecture.

  2. Obtain and prepare cookie JSON files for web automation

    main

    To allow the automated browser to maintain authenticated sessions and skip repeated logins, you must provide cookie JSON files in the cookie_json folder.

    Follow these steps:

    1. Login: Open the Google Chrome browser, navigate to the target website, and log in manually.
    2. Export: Use the Chrome extension "Export cookie JSON file for Puppeteer" to export the cookies as a .json file.
    3. Save: Place the exported .json file into the cookie_json folder within the project.
    4. Convert: Convert the exported JSON files into Python code using the provided utility script.
    cd path/to/MetaChain && python autoagent/environment/browser_cookies.py

  3. Configure API Keys for LLMs

    main

    To use different LLM providers, create an environment variable file (e.g., .env) and set the required API key. You only need to set the keys for the providers you intend to use.

    Common keys include:

    • ANTHROPIC_API_KEY
    • OPENAI_API_KEY
    • MISTRAL_API_KEY
    • GEMINI_API_KEY
    • HUGGINGFACE_API_KEY
    • GROQ_API_KEY
    • OPENROUTER_API_KEY
    • DEEPSEEK_API_KEY

  4. Import browser cookies for enhanced web access

    main
    To allow the agent to better access specific websites, you can import browser cookies into the browser environment. Detailed instructions can be found in the metachain/environment/cookie_json/README.md directory.

  6. Cite Auto-Deep-Research in academic work

    main

    If you use this framework in research, please cite it using the following BibTeX entry:

    @misc{AutoAgent,
      title={{AutoAgent: A Fully-Automated and Zero-Code Framework for LLM Agents}},
      author={Jiabin Tang, Tianyu Fan, Chao Huang},
      year={2025},
      eprint={202502.05957},
      archivePrefix={arXiv},
      primaryClass={cs.AI},
      url={https://arxiv.org/abs/2502.05957},
}

  7. Reference: LLM Provider Startup Examples

    main

    Below are specific examples of how to launch Auto-Deep-Research with various providers by setting the COMPLETION_MODEL and the corresponding API key in your .env file.

    ##### Anthropic
```bash
# .env
ANTHROPIC_API_KEY=your_anthropic_api_key

# Command
auto deep-research
    OpenAI
    # .env
OPENAI_API_KEY=your_openai_api_key

# Command
COMPLETION_MODEL=gpt-4o auto deep-research
    Mistral
    # .env
MISTRAL_API_KEY=your_mistral_api_key

# Command
COMPLETION_MODEL=mistral/mistral-large-2407 auto deep-research
    Gemini (Google AI Studio)
    # .env
GEMINI_API_KEY=your_gemini_api_key

# Command
COMPLETION_MODEL=gemini/gemini-2.0-flash auto deep-research
    Huggingface
    # .env
HUGGINGFACE_API_KEY=your_huggingface_api_key

# Command
COMPLETION_MODEL=huggingface/meta-llama/Llama-3.3-70B-Instruct auto deep-research
    Groq
    # .env
GROQ_API_KEY=your_groq_api_key

# Command
COMPLETION_MODEL=groq/deepseek-r1-distill-llama-70b auto deep-research
    OpenAI-Compatible Endpoints (e.g., Grok)
    # .env
OPENAI_API_KEY=your_api_key_for_openai_compatible_endpoints

# Command
COMPLETION_MODEL=openai/grok-2-latest API_BASE_URL=https://api.x.ai/v1 auto deep-research
    OpenRouter (e.g., DeepSeek-R1)
    # .env
OPENROUTER_API_KEY=your_openrouter_api_key

# Command
COMPLETION_MODEL=openrouter/deepseek/deepseek-r1 auto deep-research
    DeepSeek
    # .env
DEEPSEEK_API_KEY=your_deepseek_api_key

# Command
COMPLETION_MODEL=deepseek/deepseek-chat auto deep-research

  8. Run Auto-Deep-Research via CLI

    main

    Start the research agent using the auto deep-research command. You can customize the behavior using environment variables and CLI flags.

    Command Options

    • --container_name: Name of the Docker container (default: deepresearch)
    • --port: Port for the container (default: 12346)
    • COMPLETION_MODEL: Specify the LLM model name following LiteLLM format (Default: claude-3-5-sonnet-20241022)
    • DEBUG: Enable debug mode for detailed logs (default: False)
    • API_BASE_URL: The base URL for the LLM provider (default: None)
    • FN_CALL: Enable function calling (default: None; usually auto-detected based on model name)
    auto deep-research

  9. Run research in client mode with run_in_client()

    main

    The run_in_client function is an asynchronous entry point for running research tasks using an Agent. It manages a retry loop (up to 3 attempts) and handles logic for escalating to a meta_agent if the initial agent fails to resolve the case.

    If a meta_agent is provided and the initial agent fails after multiple retries, the system attempts to set up a MetaChain environment using docker_config and code_env to allow the meta_agent to develop and run new tools to solve the problem.

    Arguments:

    • agent (Agent): The primary agent performing the research.
    • messages (List): The conversation history/instructions.
    • context_variables (dict, optional): Additional context for the agent.
    • logger (MetaChainLogger, optional): Logger instance.
    • meta_agent (Agent, optional): An agent used for escalation if the primary agent fails.
    • docker_config (DockerConfig, optional): Configuration for the workplace Docker environment.
    • code_env (DockerConfig, optional): Configuration for the code execution environment.

  10. Use the MetaChain API to call tools

    main

    The API exposes registered tools as dynamic POST endpoints at /tools/{tool_name}.

    To call a tool, you must provide a ToolRequest JSON body containing an args dictionary. The keys in args must match the required parameters of the tool function (parameters without default values).

    Request Schema (ToolRequest):

    • args (dict): A dictionary where keys are parameter names and values are the arguments to pass to the tool.

    Response Format:

    • Returns {"status": "success", "result": <tool_output>} on success.
    • Returns a 400 error if required parameters are missing or if the tool execution fails.

  11. Run research in client mode with run_in_client_non_async()

    main
    The run_in_client_non_async function is a synchronous version of the research execution loop. It attempts to resolve the case using the provided agent and messages, with a maximum of 3 retries. If the agent signals 'Case not resolved', the function appends a prompt to try again in a different way and continues the loop.

  12. Use the MetaChain API to run agents

    main

    The MetaChain API provides endpoints to execute registered agents. Agents are invoked via a POST request to /agents/{agent_name}/run.

    To run an agent, you must provide an AgentRequest JSON body containing the model to use, the query string, and optional context_variables.

    Request Schema (AgentRequest):

    • model (str): The model identifier to use for the agent.
    • query (str): The user query or instruction.
    • context_variables (dict, optional): A dictionary of context variables to pass to the execution.

    Response Schema (AgentResponse):

    • result (str): The content of the last message in the agent's response.
    • messages (list): The full list of messages generated during the execution.
    • agent_name (str): The name of the agent that was executed.