docker-compose-for-agents

The application follows a pipeline where a question is processed through an LLM client that utilizes MCP tools to fetch real-time data.

Data Flow:

1. The QUESTION environment variable is loaded.
2. The Go App initializes a Langchaingo ChatClient.
3. The ChatClient uses an MCP Client to trigger an MCP Tool Callback.
4. The callback queries the Docker MCP Gateway.
5. The Gateway calls the DuckDuckGo Search API.
6. Results flow back through the Gateway $\rightarrow$ Callback $\rightarrow$ ChatClient.
7. The ChatClient uses the Docker Model Runner for LLM inference to synthesize the final answer.

The system uses a hierarchical routing model where the main agent, DevDuck, acts as the coordinator.

• General requests: Handled by DevDuck, who then routes the request to the appropriate sub-agent.
• Explicit routing: You can force a switch to a specific agent by using natural language commands:
  • "I want to speak with Bob" $\rightarrow$ Routes to Bob Agent (Go programming expert).
  • "I want to speak with Cerebras" $\rightarrow$ Routes to Cerebras Agent (Advanced problem-solving).
• Returning to coordinator: If you are stuck in a sub-agent session, use the command go back to devduck to return to the main coordinator.

The project uses a LangGraph-based agent to transform natural language queries into SQL. The workflow is as follows:

1. User sends a natural language query to the Agent.
2. Agent (via LangGraph) decides to make a Tool call to the Docker MCP Server.
3. Docker MCP Server executes the SQL Query against the PostgreSQL database.
4. PostgreSQL returns results to the Docker MCP Server, which provides structured output back to the Agent.
5. Simultaneously, the Agent makes an LLM call to the Docker Model Runner to generate a human-readable answer.
6. The Agent returns the final response to the User.

Note: PostgreSQL is populated from a SQLite dump at runtime.

The demo utilizes the AI Agent component of the Akka SDK. The architecture consists of the following flow:

1. Browser: Interacts with the UI and Agent HTTP API via HTTP.
2. Akka SDK Layer:
   • HttpUi: Serves the user interface.
   • HttpAgent: Processes HTTP requests and acts as an entry point for the Agent API.
   • Agent: The core component that manages session memory (durable storage based on username) and provides streaming interaction with the LLM.
3. LLM Provider (OpenAI): The Agent communicates with OpenAI (e.g., gpt-4o-mini) using streaming to process prompts and generate responses.

The architecture uses the Model Context Protocol (MCP) to bridge the LLM with real-time web data.

Data Flow:

1. The application receives a QUESTION via an environment variable.
2. The Spring Boot App uses a Spring AI ChatClient.
3. The ChatClient utilizes an MCP Tool Callback to interact with a Docker MCP Gateway.
4. The Gateway queries the DuckDuckGo Search API to fetch real-time results.
5. The results are returned through the gateway back to the ChatClient.
6. The ChatClient uses the Docker Model Runner (LLM) to synthesize the search results into a final natural language answer.

The system consists of three specialized agents:

1. DevDuck (Main Agent): The project coordinator. Uses model unsloth/qwen3-gguf:4B-UD-Q4_K_XL. It routes user requests to sub-agents.
2. Bob Agent: A Go programming expert for understanding code, explaining concepts, and generating snippets. Uses model ai/qwen2.5:latest.
3. Cerebras Agent: Specialized in complex problem-solving and advanced computational tasks. Uses model llama-4-scout-17b-16e-instruct via the Cerebras API.

The system uses a collaborative multi-agent workflow orchestrated by an Auditor to verify facts. The architecture consists of three distinct roles:

1. Auditor (Sequential Agent): The orchestrator. It manages the workflow from the initial user question to the final verdict, delegating tasks to the Critic and Reviser.
2. Critic: The researcher. It uses DuckDuckGo via the Model Context Protocol (MCP) to gather real-time external evidence.
3. Reviser: The analyst. It receives the Critic's findings and refines the conclusion using internal reasoning only (no external tools).

All agents perform inference through a Docker Model Runner, allowing the entire reasoning process to be containerized.

The system uses a collaborative multi-agent architecture where a Coordinator orchestrates specialized agents:

The MinionS protocol uses a decompose-execute-aggregate strategy to reduce costs while maintaining high performance. It splits tasks between a Local Model (on-device) and a Remote Model (cloud):

1. Decompose: The remote model breaks complex tasks into simple, parallel subtasks.
2. Execute: The local model processes these subtasks in parallel on document chunks.
3. Aggregate: The remote model synthesizes the local results to provide the final answer.

This approach achieves up to a 5.7× cost reduction compared to remote-only processing while maintaining approximately 97.9% of cloud model performance.

Before running the project, ensure your environment meets these requirements:

• Docker: Docker Desktop 4.43.0+ or Docker Engine installed.
• Docker Compose: If using Docker Engine on Linux, you must have Docker Compose 2.38.1 or later.
• Hardware: A laptop or workstation with a GPU (e.g., a MacBook) for local open models. Alternatively, you can use Docker Offload if a GPU is unavailable.
• Drivers: If using Docker Engine on Linux or Docker Desktop on Windows, ensure GPU support is enabled and necessary drivers are installed per the Docker Model Runner requirements.