TrustRAG Documentation

GLM-4V-Flash is a vision-understanding model provided by the Zhipu AI Open Platform (bigmodel.cn). It is designed for Multimodal Retrieval-Augmented Generation (RAG) tasks, offering a cost-effective alternative for building systems that require both image and text understanding.

Core Capabilities:

• Image captioning (description generation)
• Image classification
• Visual reasoning
• Visual Question Answering (VQA)
• Image sentiment analysis

Key Advantages:

• Free to use: Part of the Flash series of free models on the Zhipu platform.
• High Concurrency: Supports a default of 200 concurrent requests (enterprise-grade).
• Versatile Applications: Useful for OCR (e.g., insurance policy extraction), social media content generation, e-commerce product descriptions, and multimodal data labeling.

TrustRAG is a configurable, modular RAG (Retrieval-Augmented Generation) framework designed for "Reliable input, Trusted output". Key capabilities include:

• Search Engines: Supports websearch with DuckDuckGo and Searx.
• Vector Databases: Supports engines like Milvus and Qdrant.
• File Parsing: Modular support for text, docx, ppt, excel, html, pdf, and md.
• Retrieval & Re-ranking: Includes DenseRetriever (with incremental indexing), HyDE (Rewriter), and BGE (ReRank).
• Advanced RAG Techniques: Supports RAPTOR (Recursive Tree Retriever) and BgeJudge for document utility assessment.
• Multimodal Support: RAG capabilities for multimodal queries (e.g., using GLM-4V-Flash).

DeepResearch implements a multi-step process for deep information processing:

1. Intent Understanding: Parses user queries into multiple sub-queries.
2. Termination Conditions: The process stops if the Token budget or Action depth is exceeded.
3. Recursive Execution: Performs information retrieval, model reasoning, and context processing using depth-first search.
4. Action Type Determination: Based on model reasoning, the system selects one of the following actions:
   • answer: Respond to the user.
   • reflect: Perform self-reflection.
   • search: Execute a search.
   • read: Read retrieved content.
   • coding: Execute code.
5. Result Feedback: Returns the final result to the user based on the terminal action.

TrustRAG is organized into several functional modules that form a complete RAG pipeline. The core modules include:

• citation: Handles answer and evidence citation.
• document: Manages document parsing and chunking across multiple file types.
• generator: The LLM-based generation component.
• judger: Used for document selection.
• prompt: Manages prompt templates.
• refiner: Handles information summarization.
• reranker: Provides re-ranking capabilities to improve retrieval quality.
• retrieval: The core retrieval module.
• rewriter: Handles query rewriting.

Chunking is the process of dividing a complete text into smaller segments. In RAG (Retrieval-Augmented Generation) workflows, effective chunking is critical for optimizing the accuracy of content retrieved from vector databases via LLM embeddings.

Primary Goals:

• Minimize noise within the context of a single segment.
• Ensure content remains semantically relevant.

Key Trade-offs:

• Granularity vs. Efficiency: Creating more chunks increases the number of vectors, which can degrade query efficiency and reduce semantic cohesion.
• Chunk Size vs. LLM Constraints: Chunk size must be balanced against the token limits of the LLM used for generation.
• Strategy: Avoid excessive chunking where possible. When chunking is necessary, aim for larger chunks (up to the maximum effective length) to minimize the total number of chunks and maintain context.

Beyond the sliding window approach, semantic chunking can be further optimized using the following strategies to improve RAG retrieval performance:

• Dynamic Window Size: Adjust the window size based on sentence length (e.g., larger windows for short sentences, smaller windows for long sentences).
• Topic Modeling/TF-IDF: Use TF-IDF or topic modeling to determine if content belongs to the same topic before deciding where to chunk.
• Supervised Learning: Fine-tune a model on existing datasets to automatically learn the optimal splitting points.

The core principle of effective semantic chunking is finding reasonable split points based on semantic similarity rather than relying on fixed lengths or punctuation.

When using TrustRAG's citation features, the system returns a JSON object containing selected document indices and the full document content. This structure allows you to map specific parts of a generated response back to the source material.

Key components of the citation data structure include:

• selected_idx: An array of integers representing the indices of the documents used to support the response.
• selected_docs: An array of objects containing the actual document content. Each object includes:
  • file_name: The name of the source file.
  • content: The full text content of the document.
  • doc_id: A unique identifier for the document.
  • newsinfo: A nested object containing metadata such as title, content (repeated), date, and source.

TrustRAG follows a hierarchical parsing pattern to ensure that information fed to Large Language Models (LLMs) is neither too fragmented nor too overwhelming. The parsing pipeline follows the flow: File -> Document -> Paragraph -> Chunk.

For Markdown files, the parser uses header tags (e.g., #, ##) to identify structure. The strategy involves:

1. Loading and Node Identification: Correctly loading the .md file and identifying node types.
2. Paragraph Extraction: Identifying top-level headers (Level 1) and grouping all content belonging to that header into a paragraph.
3. Chunking: Within each paragraph, the content is further split into chunks using a fixed-window algorithm that prioritizes semantic integrity.

This approach ensures that the context of a header is preserved while maintaining manageable chunk sizes for retrieval.

The DeepResearch framework implements deep information processing through several stages:

1. Intent Understanding: Parses user queries into multiple sub-queries.
2. Condition Checking: Terminates if the Token budget or Action depth is exceeded; otherwise, continues recursion.
3. Recursive Execution: Performs information retrieval, model reasoning, and context processing using depth-first search.
4. Action Type Determination: Based on model reasoning, the system selects one of the following actions:
   • answer: Final response action.
   • reflect: Reflection/self-correction action.
   • search: Web/information search action.
   • read: Reading/content extraction action.
   • coding: Code execution action.

These actions update the system state and context until a final result is returned.

The parsing process is broken down into several specialized model tasks:

• Layout Detection: Uses LayoutLMv3 to detect regions like images, tables, titles, and text.
• Formula Detection: Uses YOLOv8 to detect both inline and block formulas.
• Formula Recognition: Uses UniMERNet to convert detected formulas into LaTeX.
• Table Recognition: Uses StructEqTable for table structure identification.
• OCR: Uses PaddleOCR for text recognition in scanned or garbled documents.