DOCSEARCHbyAlgolia
Submit Docs

Marco DeepResearch

repository·main·Indexed 43 days ago

https://github.com/ath-maas/marco-deepresearch

A research-driven framework from Alibaba designed to bridge the gap between general AI agents and human experts in complex, domain-specific tasks. It includes specialized architectures like Table-as-Search and UMEM, as well as benchmarks such as HSCodeComp for agent-based HS Code classification and DeepWideSearch for evaluating agent depth, width, and efficiency in information retrieval.

Tokens
24.9K
Snippets
46
Records
99
Agent score
56%

What's inside Marco DeepResearch

  1. Overview of Marco DeepResearch Series

    main

    The Marco DeepResearch series is a comprehensive suite of benchmarks and frameworks designed to address real-world challenges in intelligent agent systems. The project bridges the gap between fundamental research and practical deployment by focusing on:

    • Hierarchical Rule Application: Handling complex, nested decision logic (e.g., HSCodeComp).
    • Large-scale Information Retrieval: Balancing breadth of entity discovery with depth of multi-hop reasoning (e.g., DeepWideSearch, Table-as-Search).
    • Self-Evolving Memory: Managing long-term, generalizable memory without overfitting to specific instances (e.g., UMEM).
    • Skill Distillation: Using search-driven frameworks to expand an agent's capabilities beyond its internal knowledge (e.g., Search2Skill).

  2. Overview of Search2Skill Framework

    main

    Search2Skill is a search-driven, self-evolving skill framework designed for LLM agents to acquire reusable skills in expert domains. Unlike 'inward-looking' methods that only distill from an agent's existing knowledge, Search2Skill identifies capability gaps and uses external web search to gather evidence, which is then distilled into a structured skill library.

    Key components of the loop include:

    1. Capability-gap identification: The agent recognizes when its parametric knowledge or current skill library is insufficient.
    2. External search: The agent formulates queries and gathers evidence from the web.
    3. Skill distillation: The agent converts evidence into a structured, reusable skill.
    4. Skill reuse: The distilled skill is stored in a persistent library and injected into the context of future related tasks.

  3. Overview of Marco DeepResearch

    main

    Marco DeepResearch is an Alibaba initiative focused on advancing real-world AI agent capabilities. It addresses limitations in domain-specific reasoning, hierarchical rule application, and large-scale information seeking through a suite of benchmarks, frameworks, and models.

    Key components include:

    • HSCodeComp: A benchmark for hierarchical rule application (e.g., tariff/legal/medical).
    • DeepWideSearch: A benchmark for deep-and-wide information seeking.
    • Table-as-Search: A hierarchical multi-agent framework for long-horizon search tasks.
    • UMEM (Unified Memory Extraction and Management): A self-evolving memory system for agents.
    • Search2Skill: A framework that distills reusable expert-domain skills from external search using rubric-based RL.

  4. Overview of Marco DeepResearch sub-projects

    main

    The Marco DeepResearch family consists of several specialized benchmarks and frameworks addressing different challenges in real-world agentic systems:

    • HSCodeComp: Focuses on hierarchical rule application in the e-commerce domain (e.g., HSCode classification).
    • DeepWideSearch: Benchmarks the combination of depth and breadth in agentic information seeking/retrieval.
    • Table-as-Search: A production-grade hierarchical multi-agent framework that formulates long-horizon information seeking as table completion.
    • UMEM: A unified memory extraction and management framework designed for self-evolving agents.
    • Search2Skill: A framework for search-driven self-evolving skill distillation using Rubric-based Reinforcement Learning.

  5. Overview of the Marco DeepResearch Family projects

    main

    The Marco DeepResearch initiative consists of several specialized benchmarks and frameworks:

    • HSCodeComp: A benchmark for hierarchical rule application in the e-commerce domain.
    • DeepWideSearch: A benchmark for deep-and-wide agentic information seeking.
    • Table-as-Search: A hierarchical multi-agent framework designed for production-ready long-horizon information seeking.
    • UMEM: A unified memory extraction and management system for self-evolving agents.
    • Search2Skill: A framework for search-driven, self-evolving skill distillation via rubric-based reinforcement learning.
    • Marco-Agent-DeepResearch: An efficient 8B-scale deep research agent.

  6. Overview of the Marco DeepResearch Family

    main

    The Marco DeepResearch Family is a suite of benchmarks and frameworks designed to address real-world agent challenges. It covers four primary research areas:

    1. Hierarchical Rule Application: Using HSCodeComp to evaluate how agents handle complex, nested rules (e.g., customs/tariff codes).
    2. Deep-and-Wide Information Seeking: Using DeepWideSearch and Table-as-Search to evaluate agents' ability to discover many entities (wide) and perform deep multi-hop reasoning (deep) to produce structured tables.
    3. Self-Evolving Memory: Using UMEM to build memory systems that generalize across queries without falling into the "Rote Memorization" trap.
    4. Skill Distillation: Using Search2Skill to distill capabilities from external search into an agent's intrinsic skills via rubric-based reinforcement learning.

  7. HSCode Classification Rules and Special Cases

    main

    When implementing or prompting the HSCode Classification Agent, the following logic must be applied to handle complex product scenarios:

    Decision Rules for Ambiguous Cases

    • Rule 1: Headings are for reference only; legal classification must be determined by heading/subheading text.
    • Rule 3(a): The most specific description takes precedence.
    • Rule 3(b): Mixtures are classified based on their essential character.
    • Rule 3(c): If two headings are equally applicable, the one appearing last in numerical order is preferred.

    Handling Special Product Types

    • Multi-material products: Analyze all materials; do not rely on a single material.
    • Multi-functional products: Classify based on the primary function/use.
    • Sets/Composite goods: Classify based on the component that provides the essential character.
    • Unfinished/Semi-finished goods: If they possess the essential characteristics of a finished product, classify as the finished product.
    • Parts/Accessories: Use specific headings for parts if they exist; otherwise, classify under the heading for the complete machine/article.

  8. System Prompt for HSCode Classification Expert Agent

    main

    The hscode_system.md file defines the system instructions for the HSCode Classification Expert Agent. This agent is designed to classify products according to US Customs HSCode standards, outputting a precise 10-digit code.

    Agent Role and Knowledge

    The agent acts as a professional assistant that understands the hierarchical structure of HS Codes:

    • First 2 digits: Chapter (Major category)
    • First 4 digits: Heading (Medium category)
    • First 6 digits: Subheading (International standard)
    • 7-8 digits: US Tariff
    • 9-10 digits: Statistical Suffix

    Classification Workflow

    The agent follows a four-stage process to ensure accuracy:

    1. Deep Analysis: Parses product name, material composition, intended use, manufacturing process, and physical properties.
    2. Preliminary Categorization: Filters by primary material, validates with use/function, and applies the principle of "specific over general."
    3. Hierarchical Refinement: Drills down from 2 $\rightarrow$ 4 $\rightarrow$ 6 $\rightarrow$ 8 $\rightarrow$ 10 digits using specific legal rules (e.g., most specific description takes precedence).
    4. Verification: Performs consistency checks and ensures the final output is a complete 10-digit code.

    Output Requirements

    To integrate the agent's output into automated workflows, ensure your parser expects:

    • A 10-digit numeric code.
    • The final answer formatted in LaTeX: \boxed{XXXXXXXXXX}.
    • A detailed reasoning path explaining the decision at each hierarchy level.
    \boxed{XXXXXXXXXX}

  9. How the hierarchical multi-agent system works

    main

    Table-as-Search uses a hierarchical multi-agent architecture to balance search breadth and depth. The workflow is orchestrated by a Main Agent that decomposes tasks and manages information flow between two specialized sub-agents:

    1. Main Agent (Orchestrator): Analyzes the query, decides between tabular or deep search strategies, orchestrates sub-agents, and aggregates results into a final structured output.
    2. Tabular Search Agent (Breadth): Focuses on identifying and collecting a wide list of candidate entities. It optimizes for recall through shallow, wide-reaching searches.
    3. Deep Search Agent (Depth): Focuses on extracting detailed attributes for specific entities. It performs multi-hop reasoning to optimize for precision and completeness.

    These agents interact with a tool ecosystem including Google Search, Jina AI for web access, and a database-backed table manager.

  10. Understand DeepWideSearch evaluation metrics

    main

    DeepWideSearch evaluates agents across three axes: Depth, Width, and Efficiency.

    Depth Evaluation

    Focuses on the ability to identify target entities via multi-hop reasoning:

    • Column-F1: F1 score over unique columns in the target table (core attributes).
    • Core Entity Accuracy (CE Acc.): Accuracy in identifying the correct core entity.

    Width Evaluation

    Focuses on the comprehensiveness of information retrieval:

    • Success Rate: Binary metric; requires an exact match with human-annotated ground truth (all rows, columns, and values).
    • Row-level F1: Precision, recall, and F1 at the entity level.
    • Item-level F1: Precision, recall, and F1 at the individual cell level.

    Evaluation Protocol Statistics

    Because 4 independent runs are conducted per question, the following statistics are reported:

    • Avg@4: Mean performance across all four runs.
    • Max@4: Best performance observed across the four runs.
    • Pass@4: Proportion of questions solved successfully in at least one run (applies to Success Rate).

  11. Configure agent execution modes: efficiency vs accuracy

    main

    The agent.execution_mode setting determines how the agent handles resource limits:

    • efficiency: The agent stops as soon as a limit (token count or round count) is hit. This is ideal for quick testing.
    • accuracy: The agent attempts to use all available rollouts. If tokens run low, it automatically triggers context compression (e.g., using a summarize strategy) to continue. This is ideal for maximizing performance.

    Set this in your profile configuration under the agent key.

    agent:
  execution_mode: "accuracy"
  context_compress_enabled: true
  context_compress_strategy: "summarize"

  12. How the Table-as-Search hierarchical multi-agent workflow works

    main

    Table-as-Search uses a hierarchical multi-agent architecture to perform deep and wide information seeking. The workflow is orchestrated by a Main Agent that decomposes tasks and manages sub-agents.

    There are two specialized sub-agents:

    1. Tabular Search Agent: Optimized for recall. It performs broad, shallow searches to identify and collect a comprehensive list of candidate entities (wide search).
    2. Deep Search Agent: Optimized for precision. It performs multi-hop reasoning and detailed attribute extraction for specific entities (deep reasoning).

    Both agents interact with a Tool Ecosystem that includes Google Search, Web Visit, Table Manager, and Summarization tools.