DOCSEARCHbyAlgolia
Submit Docs

ai4j Java AI Agentic SDK

repository·main·Indexed 19 days ago

https://github.com/lnyo-cly/ai4j

A Java AI Agentic SDK for JDK 8+ providing unified access to Large Language Models (LLMs), Tool Calling, MCP, A2A, RAG, and an Agent Runtime. It includes a FlowGram workbench for visual workflow orchestration via REST APIs, a modular plugin architecture, and a CLI for managing extension initialization, validation, and execution.

Tokens
247.2K
Snippets
386
Records
1.1K
Agent score
61%

What's inside ai4j

  1. Overview of ai4j-spring-boot-starter

    main

    The ai4j-spring-boot-starter integrates the ai4j Core SDK into the Spring Boot ecosystem by leveraging Spring's configuration, Bean lifecycle, and business layering.

    Key Responsibilities:

    • Configuration Binding: Maps ai.* properties from your application configuration to property objects.
    • Unified Service Entry: Exposes AiService for interacting with AI models.
    • Multi-instance Registry: Uses AiServiceRegistry to manage multiple provider instances.
    • HTTP Client Management: Provides unified OkHttp configuration, including timeouts and connection capabilities.
    • RAG Support: Automatically assembles RAG-related Beans (vector stores, assemblers, rerankers) when conditions are met.

    Note: This starter does not redefine the underlying semantics of Chat, Responses, Tool, MCP, or RAG; those are handled by the Core SDK. If you are not using Spring Boot, use the Core SDK directly.

  2. Overview of Coding Agent

    main

    The Coding Agent is a runtime and host entry point in AI4J designed specifically for local code repository tasks. Unlike a general-purpose agent, it organizes workspaces, tools, sessions, approvals, MCP, Skills, and interfaces (CLI/TUI/ACP) into a cohesive local development workflow.

    Key Value Proposition: It enables models to complete traceable, recoverable, and approvable development tasks within the context of a local code repository by integrating workspace awareness, tool policies, session lifecycles, and host protocols into a single execution model.

  3. Core SDK Overview

    main

    The Core SDK (located in the ai4j/ module) is the foundational layer of AI4J. It provides a continuous engineering model for Java 8+ projects to integrate models, tools, protocols, context, RAG, and extensions. It serves as the capability base for higher-level modules like Spring Boot starters, Agents, Coding Agents, and FlowGram.

    Core Capabilities:

    • Model Access: Chat, Responses, streaming, and multi-modal capabilities.
    • Tools: Exposing local Java functions or controlled capabilities to models.
    • Skills: Providing models with reusable instructions, templates, and task flows.
    • MCP (Model Context Protocol): Connecting to external tools/services via protocol or publishing Java capabilities as an MCP server.
    • Memory: Managing session state, chat history, and context boundaries.
    • Search & RAG: Document ingestion, chunking, embedding, vector search, reranking, and citation.
    • Extension: Implementing new providers, models, services, or network stacks.

  4. Compare AI4J with other Java AI frameworks

    main

    AI4J is positioned as a multi-module AI SDK for Java 8+ designed for progressive upgrades. Use the following comparison to decide if AI4J fits your project requirements:

    SolutionNatureBest For
    AI4JJava 8+ multi-module AI SDK with progressive upgrade capabilitiesLow-cost integration into standard Java/Maven projects, with optional paths to Spring, RAG, MCP, Agent, Coding Agent, or FlowGram
    Spring AIAI abstraction and auto-configuration for the Spring ecosystemTeams already deeply using Spring Boot/Spring Cloud who want to follow official Spring patterns
    LangChain4jMature LLM application framework for the Java ecosystemProjects requiring a large community, many integrations, and high-level abstractions
    AgentScope JavaJava solution for agent research and multi-agent scenariosFocus on agent orchestration, experimental agent runtimes, and platform capabilities
    Pi Agent / Pi SDKJS/TS agent and automation SDKFrontend, Node.js, browser, or JS/TS-based agent scenarios

  5. Pre-production security checklist

    main

    Use this checklist to verify security posture before going live:

    • No real provider keys, MCP tokens, or database passwords are in the repository.
    • Tool/MCP exposure scope is converged to a whitelist based on business scenarios.
    • RAG retrieval inherits business permissions and tenant isolation.
    • Trace, logs, task reports, and session stores undergo sensitive information processing.
    • Coding Agent file, shell, patch, and process tools have approval boundaries.
    • FlowGram task APIs are protected by authentication or a gateway.
    • All external providers, MCP servers, and vector databases have timeout and failure handling implemented.

  6. What is RagQueryPlanner and how does it work

    main

    A RagQueryPlanner is a pre-retrieval processing layer in the RAG pipeline. Its purpose is to decide whether the original user query should be transformed into one or more queries that are better suited for retrieval.

    Key Characteristics:

    • It is not a general text rewriter or an agent planner; it specifically produces retrieval plans.
    • It does not handle multi-path recall, reranking, or answer generation.
    • It preserves the original query to be used later by the Reranker, RagContextAssembler, and the final generation step.
    • If the planner fails or returns no usable queries, the SDK automatically falls back to the original query.

    The RAG Pipeline with a Planner: RagQuery(original + optional history) $\rightarrow$ RagQueryPlanner $\rightarrow$ Retriever $\rightarrow$ Reranker $\rightarrow$ RagContextAssembler

    public interface RagQueryPlanner {
    RagQueryPlan plan(RagQuery query) throws Exception;
}

  7. What is McpGateway and when to use it

    main

    The McpGateway is the core multi-service MCP runtime in AI4J. Unlike a simple wrapper for McpClient, it manages the entire lifecycle of multiple MCP services, including connection states, tool-to-client mappings, and tenant isolation.

    Use McpGateway when you need:

    • To manage more than one MCP service.
    • To handle multiple transport types unified under one entry point.
    • To implement user-level service isolation (multi-tenancy).
    • To support dynamic service start/stop and configuration-driven governance.

    Do NOT use McpGateway if:

    • You only need to connect to a single MCP service (use McpClient instead).
    • You are just testing basic transport connectivity.

  8. Distinguish between ChatMemory and the Tool layer

    main

    A common misconception is that because ChatMemory contains methods like addAssistantToolCalls(...) and addToolOutput(...), it functions as a tool system. This is incorrect.

    ChatMemory only records what happened (the history of interactions) to ensure the model has context. It does not handle:

    • Permission logic (allowing or disallowing tool calls).
    • Approval workflows.
    • Execution of tool side effects.
    • Error compensation or recovery after tool failure.
    • Orchestration of multi-step tool chains.

    The Tool Layer (Core SDK) is responsible for the actual capability surface, including:

    • Exposing specific tool capabilities.
    • Defining parameter schemas via @FunctionParameter.
    • Defining function requests and calls via @FunctionRequest and @FunctionCall.
    • Managing how local tools and remote MCP tools are aggregated.
    • Using ToolUtil and defining functions or mcpServices on the request side.

  9. Manage multiple MCP services with McpGateway

    main

    When you need to manage multiple MCP services simultaneously, use McpGateway. Unlike a simple list of clients, the Gateway acts as a dedicated runtime for multi-service management.

    Key Capabilities:

    • Client Isolation: Supports both global clients (serviceId) and user-specific clients (user_{userId}_service_{serviceId}).
    • Tool Mapping: Uses McpGatewayToolRegistry to pull tool lists from all connected clients and map toolName (global) or user_{userId}_tool_{toolName} (user-specific) to the correct client.
    • Dynamic Configuration: Supports loading configurations via McpConfigSource in addition to the default mcp-servers-config.json.

  10. Reuse `defaultFormMeta` for node property panels

    main

    When developing new nodes, you should prefer reusing defaultFormMeta from src/nodes/default-form-meta.tsx instead of writing a custom formMeta immediately.

    defaultFormMeta provides several critical features out-of-the-box:

    • Basic Validation: Automatically validates that title is present and that inputsValues match the required fields defined in the node's schema.
    • Built-in Effects:
      • syncVariableTitle: Synchronizes titles.
      • provideJsonSchemaOutputs: Derives output schemas.
      • autoRenameRefEffect: Handles automatic renaming of references.
      • validateWhenVariableSync: Validates when variables are synchronized.
      • listenRefSchemaChange: Listens for changes in reference schemas.

    Only implement a custom form-meta.tsx if your node requires specialized UI interactions that the default meta cannot handle.

  11. Key components of the JDBC Agent Memory solution

    main

    When implementing the JDBC Agent Memory solution, the following key objects govern how Agent sessions are persisted, compressed, and recovered:

    • agent/memory/JdbcAgentMemory: The primary implementation for persisting agent memory via JDBC.
    • WindowedMemoryCompressor: Used to apply compression strategies to the session memory to manage context size.
    • DataSource: The Spring-managed data source (e.g., MySQL) used by JdbcAgentMemory for storage.
    • Agent session objects: Objects representing the lifecycle and state of the Agent's current task.

  12. Understand the ChatMemory abstraction

    main

    ChatMemory is the core abstraction for managing multi-turn conversation facts in AI4J. Rather than just storing a list of strings, it organizes conversation history into a unified context that supports projection, clipping, snapshotting, and restoration. It serves as a shared foundation for both Chat and Responses protocols by providing methods to project history into different formats.

    Key capabilities include:

    • Adding various roles: addSystem(...), addUser(...), addAssistant(...).
    • Handling tool interactions: addAssistantToolCalls(...), addToolOutput(...).
    • Bulk operations: addAll(...).
    • Projections: toChatMessages() (for the Chat path) and toResponsesInput() (for the Responses path).
    • State management: snapshot(), restore(...), clear(), and getItems().