Spring AI Alibaba DataAgent Documentation

DataAgent is an enterprise-grade intelligent data analysis agent built on Spring AI Alibaba Graph. It goes beyond simple Text-to-SQL by providing deep analysis capabilities and automated reporting.

Key Capabilities

• Intelligent Data Analysis: Uses StateGraph for Text-to-SQL conversion, supporting complex multi-table queries and multi-turn dialogue.
• Python Deep Analysis: Executes generated code within task-level containers using the Spring AI Alibaba Sandbox. It supports PEP 723 dynamic dependencies, resource limits, failure retries, and automatic cleanup.
• Smart Report Generation: Automatically summarizes analysis results into HTML/Markdown reports containing ECharts visualizations.
• Human-in-the-loop: Includes a feedback mechanism allowing users to intervene and adjust plans during the generation phase.
• RAG (Retrieval-Augmented Generation): Integrates with vector databases to perform semantic retrieval on business metadata and terminology, improving SQL accuracy.
• Multi-Model Orchestration: Features a model registry that allows runtime switching between different LLMs and Embedding models (compatible with OpenAI specifications, e.g., Qwen, Deepseek).
• MCP Server Support: Implements the Model Context Protocol, allowing DataAgent to act as a Tool Server for ecosystems like Claude Desktop, providing NL2SQL and agent management capabilities.
• API Key Management: Provides lifecycle management and fine-grained permission control for API keys.

DataAgent is organized into several functional layers that facilitate an AI-driven data analysis workflow:

• Clients: Includes the data-agent-frontend-nuxt UI, an Admin Console, and MCP Clients.
• Access Layer: Provides communication via REST API and SSE Stream (Server-Sent Events).
• Management (Spring Boot): The core engine containing controllers (GraphController, AgentController, PromptConfigController, ModelConfigController), services for LLM management (LlmService, AiModelRegistry), vector retrieval (AgentVectorStoreService), and a StateGraph Workflow for agent orchestration.
• Data Storage: Persists business data (Business DB), management metadata (Management DB), vector embeddings (Vector Store), and unstructured knowledge (Knowledge Files).
• LLM Providers: Supports various Chat and Embedding models.
• Python Sandbox Runtime: A secure execution environment using SAA SandboxService and Docker Engine to run generated Python code in task-scoped sandboxes.

The DataAgent operates using a structured workflow (StateGraph) that moves from intent recognition to final reporting. The main lifecycle follows these stages:

1. Context & Intent: Builds multi-turn context and uses an IntentRecognitionNode to determine if analysis is required.
2. Retrieval & Enhancement: Uses EvidenceRecallNode, QueryEnhanceNode, SchemaRecallNode, and TableRelationNode to gather necessary metadata and schema information.
3. Planning: Performs a FeasibilityAssessmentNode check. If feasible, a PlannerNode creates a plan, which is validated by PlanExecutor. A HumanGate allows for manual review/feedback.
4. Execution Loops:
   • SQL Path: SqlGenerateNode $\rightarrow$ SemanticConsistencyNode $\rightarrow$ SqlExecuteNode. If execution fails, it retries generation.
   • Python Path: PythonGenerateNode $\rightarrow$ PythonExecuteNode $\rightarrow$ PythonAnalyzeNode.
5. Reporting: The ReportGeneratorNode synthesizes the results into a final report.

DataAgent workflow nodes are categorized by complexity levels based on their logic and external dependencies. This classification helps in planning testing strategies and resource allocation.

• Very Low: Nodes with pure logic and no external dependencies (e.g., PlanExecutorNode, HumanFeedbackNode).
• Low: Nodes with a single LLM dependency (e.g., IntentRecognitionNode, FeasibilityAssessmentNode, PythonAnalyzeNode, SemanticConsistencyNode).
• Medium: Nodes with 2-3 dependencies (e.g., SqlGenerateNode, PlannerNode, PythonGenerateNode, QueryEnhanceNode, SchemaRecallNode, TableRelationNode, ReportGeneratorNode).
• High: Nodes with approximately 5 dependencies requiring DB mocking (e.g., SqlExecuteNode).
• Very High: Nodes requiring multiple external services (e.g., EvidenceRecallNode, PythonExecuteNode).

| Complexity Level | Nodes | Strategy |
|----------------|-------|----------|
| Very Low | PlanExecutorNode, HumanFeedbackNode | Pure logic, no external deps |
| Low | IntentRecognitionNode, FeasibilityAssessmentNode, PythonAnalyzeNode, SemanticConsistencyNode | Single LLM dependency |
| Medium | SqlGenerateNode, PlannerNode, PythonGenerateNode, QueryEnhanceNode, SchemaRecallNode, TableRelationNode, ReportGeneratorNode | 2-3 dependencies |
| High | SqlExecuteNode | 5 dependencies, DB mocking |
| Very High | EvidenceRecallNode, PythonExecuteNode | Multiple external services |

In DataAgent, OverAllState is the mechanism used to pass data between workflow nodes. When writing tests, you must follow a specific lifecycle for state management to ensure data is correctly accessible and typed.

1. Register Keys

CRITICAL: You must register state keys and their associated replacement strategies before attempting to use them. Failure to do this is a common cause of low test coverage.

2. Update State

Use updateState to inject data into the workflow state, typically using a map of constant keys to values.

3. Retrieve State Values

Use StateUtil to extract values from the state. This utility provides type-specific methods to ensure the data retrieved matches the expected type (e.g., String, Integer, or custom DTOs).

// 1. Register keys before use (CRITICAL)
state.registerKeyAndStrategy(SQL_GENERATE_OUTPUT, new ReplaceStrategy());

// 2. Set state values
state.updateState(Map.of(
    SQL_GENERATE_OUTPUT, "SELECT * FROM users",
    SQL_GENERATE_COUNT, 0
));

// 3. Get state values (via StateUtil)
String sql = StateUtil.getStringValue(state, SQL_GENERATE_OUTPUT);
SchemaDTO schema = StateUtil.getObjectValue(state, TABLE_RELATION_OUTPUT, SchemaDTO.class);

DataAgent follows a multi-layered architecture designed to handle complex data analysis tasks through LLM-driven workflows.

Core Layers:

• Clients: Includes the data-agent-frontend-nuxt UI, an Admin Console, and MCP Clients.
• Access Layer: Provides communication via REST API and SSE Stream (Server-Sent Events).
• Management (Spring Boot): The core engine containing controllers (GraphController, AgentController, PromptConfigController, ModelConfigController), services for LLM orchestration (LlmService), vector retrieval (AgentVectorStoreService), and a Python sandbox execution environment (PythonCodeExecutorService).
• Execution Layer: Uses a Python Sandbox Runtime powered by Docker Engine and SAA SandboxService to safely execute generated code.
• Data Layer: Manages Business DBs, Metadata (Management DB), Vector Stores, and Knowledge Files.
• Observability: Integrates with the Langfuse Platform for monitoring and tracing.

%%{init: {"theme": "base", "flowchart": {"curve": "basis", "nodeSpacing": 35, "rankSpacing": 45}, "themeVariables": {"lineColor": "#475569", "primaryTextColor": "#1F2937"}}}
%%
flowchart LR
  subgraph Clients[Clients]
    UserUI[data-agent-frontend-nuxt UI]
    AdminUI[Admin Console]
    MCPClient[MCP Client]
  end

  subgraph Access[Access Layer]
    RestAPI[REST API]
    SSE[SSE Stream]
  end

  subgraph Management[data-agent-management Spring Boot]
    GraphCtl[GraphController]
    AgentCtl[AgentController]
    PromptCtl[PromptConfigController]
    ModelCtl[ModelConfigController]
    GraphSvc[GraphServiceImpl]
    Context[MultiTurnContextManager]
    Graph[StateGraph Workflow]
    LlmSvc[LlmService]
    ModelRegistry[AiModelRegistry]
    VectorSvc[AgentVectorStoreService]
    Hybrid[HybridRetrievalStrategy]
    CodeExec[PythonCodeExecutorService]
    McpSvc[McpServerService]
    LangfuseSvc[LangfuseService]
  end

  subgraph Observability[Observability]
    Langfuse[Langfuse Platform]
  end

  subgraph Data[Data Storage]
    BizDB[(Business DB)]
    MetaDB[(Management DB)]
    VectorDB[(Vector Store)]
    Files[(Knowledge Files)]
  end

  subgraph LLMs[LLM Providers]
    ChatLLM[Chat Model]
    EmbeddingLLM[Embedding Model]
  end

  subgraph Exec[Python Sandbox Runtime]
    SandboxSvc[SAA SandboxService]
    DockerEngine[Docker Engine]
    TaskSandbox[Task-scoped BaseSandbox]
    PackageIndex[(PyPI / Private Package Index)]
  end

  UserUI --> RestAPI
  UserUI --> SSE
  AdminUI --> RestAPI
  MCPClient --> McpSvc
  RestAPI --> AgentCtl
  RestAPI --> PromptCtl
  RestAPI --> ModelCtl
  SSE --> GraphCtl
  GraphCtl --> GraphSvc
  GraphSvc --> Context
  GraphSvc --> Graph
  Graph --> LlmSvc
  GraphSvc --> VectorSvc
  VectorSvc --> Hybrid
  VectorSvc --> VectorDB
  VectorSvc --> Files
  Graph --> BizDB
  GraphSvc --> ModelRegistry
  ModelRegistry --> ChatLLM
  ModelRegistry --> EmbeddingLLM
  Graph --> CodeExec
  CodeExec --> SandboxSvc
  SandboxSvc --> DockerEngine
  DockerEngine --> TaskSandbox
  TaskSandbox --> PackageIndex
  GraphSvc --> LangfuseSvc
  LangfuseSvc --> Langfuse
  AgentCtl --> MetaDB
  PromptCtl --> MetaDB
  ModelCtl --> MetaDB

The backend architecture follows these core principles to ensure maintainability and testability:

• Single Responsibility: Separate concerns such as environment resolution, Docker client creation, image preparation, and task execution into distinct components.
• Dependency Inversion: Orchestration logic should depend on small, stable contracts (e.g., a Docker gateway contract) rather than constructing infrastructure internally.
• Explicit Side Effects: Constructors should only assign dependencies. They must not connect to services, pull images, or start long-running work.
• Fail-Fast Contracts: Reject invalid configurations immediately with specific, testable exceptions and messages.
• Encapsulation: Extract collaborators only when they represent a coherent policy or an external boundary, not just to reduce class size.
• Compatibility: Maintain existing service interfaces and configuration properties while refactoring implementation details behind new collaborators.