DOCSEARCHbyAlgolia
Submit Docs

Qwen-Agent Framework

repository·main·Indexed 12 days ago

https://github.com/qwenlm/qwen-agent

A framework for developing LLM applications that leverage the instruction following, tool usage, planning, and memory capabilities of the Qwen model series. It provides building blocks for creating agents such as Browser Assistants, Code Interpreters, and custom assistants, and includes benchmarks for Code Interpreter and DeepPlanning (Shopping and Travel) capabilities.

Tokens
25.6K
Snippets
65
Records
99
Agent score
94%

What's inside Qwen-Agent

  1. Overview of DeepPlanning Benchmark

    main

    DeepPlanning is a benchmark designed to evaluate long-horizon agentic planning by focusing on global constrained optimization rather than just local, step-level reasoning. It tests an agent's ability to handle real-world constraints like time and financial budgets through proactive information gathering and fine-grained reasoning.

    Key evaluation areas include:

    1. Proactive Information Acquisition: Using APIs to discover environment states (e.g., stock levels, opening hours) instead of hallucinating.
    2. Local Constrained Reasoning: Satisfying specific user requirements (e.g., specific brands or amenities).
    3. Global Constrained Optimization: Managing holistic boundaries like total budget caps and multi-day schedule feasibility.

  2. Overview of Qwen-Agent core capabilities

    main

    Qwen-Agent is a modular framework designed for building intelligent LLM applications. Key architectural features include:

    • Unified Agent Interface: Uses a high-level Agent base class with implementations like Assistant and FnCallAgent.
    • Advanced Tool Calling: Supports parallel, multi-step, and multi-turn function calls with automatic parsing.
    • RAG (Retrieval-Augmented Generation): Supports efficient document QA over 1M+ tokens using hybrid RAG and agent-based decomposition.
    • MCP Integration: Supports the Model Context Protocol (MCP) to connect to external services like GitHub, filesystem, or SQLite.
    • Multi-Model Compatibility: Works with Qwen series (Qwen3, Qwen2.5, etc.) via DashScope API or local OpenAI-compatible servers (vLLM, SGLang).
    • Context Management: Automatically manages long-form text to prevent exceeding model context limits while maintaining agent effectiveness.
    • Extensible Architecture: Allows independent swapping of LLMs, tools, memory, and planning strategies.

  4. BrowserQwen core features and modes

    main

    BrowserQwen is a Chrome extension built on Qwen-Agent that provides several advanced capabilities:

    Key Capabilities

    • Content Discussion: Discuss current web pages or PDF documents.
    • History & Summarization: Records browsed pages and materials (PDF, Word, PPT) to help summarize content and automate text-based tasks.
    • Plugin Integration: Includes a Code Interpreter for solving math problems, data analysis, and visualization.

    Operating Modes

    • Workstation - Editor Mode (创作模式): Focused on long-form content creation based on browsed web pages and PDFs, utilizing plugins for rich text assistance.
    • Workstation - Chat Mode (对话模式): Supports multi-webpage Q&A and using the Code Interpreter to generate data charts.
    • Browser Assistant (浏览器助手): Provides direct Q&A for the current webpage or active PDF document.

  5. Understand DeepPlanning benchmark results and metrics

    main

    Results are stored in domain-specific folders and an aggregated file.

    Result Locations

    • Travel: travelplanning/results/{model}_{language}/ (contains evaluation_summary.json, converted_plans/, and trajectories/).
    • Shopping: shoppingplanning/result_report/ (contains summary_report_{model}_{level}_{timestamp}.json and {model}_statistics.json).
    • Aggregated: aggregated_results/{model}_aggregated.json.

    Key Metrics

    DomainMetricDescription
    Shoppingmatch_rate% of expected items correctly matched (Main metric)
    Shoppingweighted_average_case_scoreAverage case completion score (Main metric)
    Travelcomposite_scoreWeighted combination of commonsense and personalized scores (Main metric)
    Travelcase_acc% of cases passing all constraints (Main metric)
    Cross-Domainavg_accAverage of shopping weighted_average_case_score and travel case_acc (Primary metric)

  6. Configure Model Services for Qwen-Agent

    main

    Qwen-Agent supports two main ways to connect to LLMs:

    1. Alibaba Cloud DashScope

    Set the DASHSCOPE_API_KEY environment variable to your unique DashScope API key. In your configuration, use model_type: 'qwen_dashscope'.

    2. OpenAI-compatible Services (vLLM or Ollama)

    Deploy your own service and provide the model_server (base URL) and api_key in the configuration.

    Important Note for Qwen3/QwQ models:

    • When using QwQ and Qwen3 with vLLM, do not add --enable-auto-tool-choice or --tool-call-parser hermes to your vLLM command. Qwen-Agent handles tool parsing internally.
    • When using Qwen3-Coder, it is recommended to enable both --enable-auto-tool-choice and --tool-call-parser hermes in vLLM, and use the use_raw_api parameter in Qwen-Agent.

  7. How the Travel Planning pipeline works

    main

    The benchmark operates in three distinct stages:

    1. Inference (Agent Planning)

    • Action: Loads tasks from data/travelplanning_query_{lang}.json and uses an LLM agent to generate plans using tools (flights, hotels, etc.).
    • Output: Saves trajectories (results/{model}_{lang}/trajectories/) and human-readable Markdown reports (results/{model}_{lang}/reports/).

    2. Conversion (Plan Parsing)

    • Action: Uses an LLM (default: qwen-plus) to parse the Markdown reports into a standardized JSON format.
    • Purpose: Converts human-readable text into structured data required for automated scoring.
    • Output: Structured JSON files in results/{model}_{lang}/converted_plans/.

    3. Evaluation

    • Action: Scores the converted plans based on delivery rate, commonsense (8 dimensions), and personalized constraints.
    • Output: Summary statistics (evaluation_summary.json) and individual task scores (id_{n}_score.json) in results/{model}_{lang}/evaluation/.

    Note: You can skip stages by setting BENCHMARK_START_FROM to conversion or evaluation respectively.

  8. Understand Code Interpreter Benchmark metrics and domains

    main

    The benchmark evaluates LLMs on two primary dimensions:

    1. Code Executability: Measures the ability of the LLM to generate code that can actually be executed. This is measured for the general (General problem-solving) domain.
    2. Code Correctness: Measures whether the generated code produces the correct result. This is divided into two domains:
      • Math: Evaluated using the gsm8k task.
      • Visualization: Evaluated using the visualization task (often judged by a vision model like gpt-4-vision-preview).

  9. DeepPlanning Travel Planning Domain

    main

    The Travel Planning domain tasks agents with acting as personal travel assistants for multi-day trips. The agent must manage tightly coupled time, location, and budget constraints.

    • Input: Natural language queries containing destination, dates, budget, and specific preferences (e.g., "3-star hotel with a dryer").
    • Tools: 9 specialized APIs for searching flights, trains, hotels, restaurants, and attractions.
    • Output: A structured planning report including itemized costs and a minute-by-minute schedule.
    • Core Skill: Spatio-temporal reasoning (aligning flight times, attraction hours, and transit durations).

  10. How context management works in Qwen Agent

    main

    Qwen Agent implements a dynamic context management logic designed to prevent input messages from exceeding a model's maximum context length. This mechanism automatically triggers when calling agent.run(...) or llm.call(...) if the total token count reaches the configured max_input_tokens limit.

    The goal is to truncate input messages while maintaining a rational dialogue structure, allowing the agent to operate within an effectively "infinite context" window by prioritizing the preservation of recent interactions over older ones.

  11. Function Calling and Tool Calling capabilities

    main
    Qwen-Agent supports function calling (tool calling). The LLM classes provide native function calling capabilities, and specific Agent classes like FnCallAgent and ReActChat are built directly upon this functionality. The default tool calling template natively supports Parallel Function Calls.

  12. Use `BaseModelCompatibleDict` for dictionary-like model access

    main

    All core schema classes inherit from BaseModelCompatibleDict, which extends Pydantic's BaseModel. This provides several developer-friendly features:

    • Dictionary-style access: Access fields using msg['key'].
    • Safe retrieval: Use .get('key', default) to avoid KeyError.
    • Clean serialization: model_dump() automatically omits fields with a value of None.
    • Readable representation: str(msg) returns the result of model_dump().
    msg = Message(role='user', content='Hello')
print(msg['role'])  # → 'user'
print(msg.get('non_existent_key', 'default'))  # → 'default'
# fields with value=None are excluded by default
print(msg.model_dump())