JoyAgent-JDGenie

Multimodal Knowledge Management (mRAG) is a comprehensive RAG (Retrieval-Augmented Generation) platform designed for multimodal documents. It provides an end-to-end pipeline for parsing, retrieving, and generating content from complex, heterogeneous data sources.

Core Capabilities

• End-to-End Multimodal Pipeline: Handles the full lifecycle from raw data parsing to cross-modal semantic retrieval and intelligent generation.
• Broad Document Compatibility: Native support for PDF, Office suites, and images, with automatic recognition of complex structures.
• Structured Information Extraction: Integrates the MinerU parsing framework to ensure high-fidelity extraction of document structures and multimodal content.
• Decoupled Architecture: Core modules (parsing, embedding, retrieval, generation) are modular and extensible for enterprise customization.
• Hybrid Retrieval System: Combines keyword, vector, and knowledge graph retrieval techniques for efficient cross-modal positioning.
• Deployment Options: Provides both a Web UI and standard APIs, supporting local and private cloud deployments to ensure data privacy.

JoyAgent-JDGenie is an end-to-end, lightweight, and general-purpose multi-agent product designed to solve the 'last mile' problem of quickly building multi-agent applications. Unlike many open-source projects that only provide SDKs or frameworks requiring further development, JoyAgent-JDGenie is a complete product that can directly answer queries or solve tasks (e.g., generating web-based or PPT-style reports from a single query).

Key characteristics:

• End-to-End Product: Includes frontend, backend, framework, engine, and core sub-agents (Report Generation, Code, PPT, File agents, etc.).
• Extensible: Users can customize new scenarios by mounting relevant sub-agents or tools to the framework.
• Lightweight & Independent: Does not depend on specific cloud ecosystems (unlike SpringAI-Alibaba or Coze).
• High Performance: Achieved 75.15% accuracy on the GAIA Validation set and 65.12% on the Test set, outperforming several industry-known products.

The DGP (Data Governance Protocol) is used to prepare tables for intelligent querying and analysis. To use JoyDataAgent effectively, users should govern their tables according to these principles:

1. Table Design Principles

• Do not mix detail tables (明细表) and metric tables (指标表).
• Do not mix incremental tables (增量表) and full-load tables (全量表).

2. Field Design Principles

• Avoid field name confusion.
• Clearly define the semantics for point-in-time metrics (时点指标) versus period metrics (时期指标).

3. Field Value Design Principles

• Provide semantic descriptions for enumeration values (枚举值).

4. Data Lineage and Semantic Alignment (In Progress)

• The system uses SQLAST parsing to build knowledge graphs of field, table, and operator lineage.
• It performs semantic normalization to resolve conflicts in dimension definitions and ensures precise SQL generation through metric-operator pre-weaving.

JoyAgent-JDGenie is a lightweight, general-purpose multi-agent product designed for end-to-end task completion.

Key Architectural Concepts:

• Multi-level & Multi-pattern Thinking: Combines different design patterns (like react and plan and executor) across multiple levels (work level and task level) to support complex planning.
• Agent Framework Protocol: Supports various agent design patterns, multi-agent context management, and a high-concurrency DAG (Directed Acyclic Graph) execution engine for efficiency.
• Tool Evolution: Instead of generating tools from scratch, the system uses an "auto-disassembly-and-reassembly" approach. It implicitly decomposes existing tools into atomic tools and recombines them using LLMs to create new, more reliable tools.
• Cross-task Workflow Memory: Maintains memory across different tasks to recognize and leverage similar task patterns.

You can use Docker to quickly launch the full service. This method involves cloning the repository, configuring backend and tool environment variables, building the image, and running the container.

Configuration Steps:

1. Backend: Update genie-backend/src/main/resources/application.yml with your base_url, apikey, model, max_tokens, and model_name.
   • Note for DeepSeek users: Set max_tokens to 8192 for deepseek-chat.
2. Tools: Update genie-tool/.env_template with OPENAI_API_KEY, OPENAI_BASE_URL, DEFAULT_MODEL, and SERPER_SEARCH_API_KEY.
   • Note for DeepSeek users: Set DEEPSEEK_API_KEY and DEEPSEEK_API_BASE. Set DEFAULT_MODEL and all occurrences of ${DEFAULT_MODEL} to deepseek/deepseek-chat.

Once configured, build and run the container.

# 1. Clone the repo
git clone https://github.com/jd-opensource/joyagent-jdgenie.git

# 2. (Perform manual config steps mentioned above)

# 3. Build the docker image
docker build -t genie:latest .

# 4. Run the container
docker run -d -p 3000:3000 -p 8080:8080 -p 1601:1601 --name genie-app genie:latest

# 5. Access via browser at http://localhost:3000