Eino LLM Application Development Framework

Eino provides several advanced capabilities for LLM application development:

• Component Ecosystem: Reusable abstractions like ChatModel, Tool, Retriever, and Embedding with official implementations in eino-ext (OpenAI, Claude, Gemini, etc.).
• Stream Processing: Automatic handling of streaming data (concatenating, merging, and copying) as it flows between nodes in a graph or agent.
• Callback Aspects: Ability to inject logic (logging, tracing, metrics) at specific lifecycle points: OnStart, OnEnd, OnError, OnStartWithStreamInput, and OnEndWithStreamOutput.
• Interrupt/Resume (Human-in-the-loop): Support for pausing agent or tool execution to wait for human input and resuming from a checkpoint via state persistence.

Eino provides several advanced capabilities for LLM application development:

• Component Ecosystem: Standardized abstractions for ChatModel, Tool, Retriever, and Embedding. Official implementations are available in eino-ext for providers like OpenAI, Claude, Gemini, and Ollama.
• Stream Processing: Automatic handling of streaming data (concatenation, boxing, merging, and copying) within orchestrated workflows.
• Callback Aspects: Ability to inject logic (logging, tracing, metrics) at specific lifecycle points: OnStart, OnEnd, OnError, OnStartWithStreamInput, and OnEndWithStreamOutput.
• Interrupt/Resume (Human-in-the-loop): Support for pausing agents or tools to wait for human input and resuming from a checkpoint, with built-in state persistence and routing.

The Eino ecosystem is divided into several repositories:

• Eino (core): Contains type definitions, streaming mechanisms, component abstractions, orchestration logic, agent implementations, and aspect mechanisms.
• EinoExt: Provides concrete component implementations (e.g., OpenAI, Ollama), callback handlers, evaluators, and prompt optimizers.
• Eino Devops: Tools for visualized development and debugging.
• EinoExamples: Reference implementations and best practices for common patterns.

A ChatModelAgent is a basic agent that uses a ChatModel and can optionally use tools. It automatically handles the ReAct loop (deciding when to call tools and when to respond).

To use it, configure a ChatModel (e.g., OpenAI), wrap it in an adk.ChatModelAgent, and run it using an adk.Runner.

chatModel, _ := openai.NewChatModel(ctx, &openai.ChatModelConfig{
    Model:  "gpt-4o",
    APIKey: os.Getenv("OPENAI_API_KEY"),
})

agent, _ := adk.NewChatModelAgent(ctx, &adk.ChatModelAgentConfig{
    Model: chatModel,
})

runner := adk.NewRunner(ctx, adk.RunnerConfig{Agent: agent})
iter := runner.Query(ctx, "Hello, who are you?")
for {
    event, ok := iter.Next()
    if !ok {
        break
    }
    fmt.Println(event.Message.Content)
}

When you need precise control over the execution flow, use the compose package to build directed graphs. You can add different types of nodes (like LambdaNode for custom logic or ChatModelNode for LLM calls) and define edges to connect them using compose.START and compose.END.

graph := compose.NewGraph[*Input, *Output]()
graph.AddLambdaNode("validate", validateFn)
graph.AddChatModelNode("generate", chatModel)
graph.AddLambdaNode("format", formatFn)

graph.AddEdge(compose.START, "validate")
graph.AddEdge("validate", "generate")
graph.AddEdge("generate", "format")
graph.AddEdge("format", compose.END)

runnable, _ := graph.Compile(ctx)
result, _ := runnable.Invoke(ctx, input)

For complex workflows that require breaking problems into steps or delegating to specialized sub-agents, use DeepAgent. It can coordinate multiple agents and use tools like shell commands, Python execution, or web search to track and complete progress.

deepAgent, _ := deep.New(ctx, &deep.Config{
    ChatModel: chatModel,
    SubAgents: []adk.Agent{researchAgent, codeAgent},
    ToolsConfig: adk.ToolsConfig{
        ToolsNodeConfig: compose.ToolsNodeConfig{
            Tools: []tool.BaseTool{shellTool, pythonTool, webSearchTool},
        },
    },
})

runner := adk.NewRunner(ctx, adk.RunnerConfig{Agent: deepAgent})
iter := runner.Query(ctx, "Analyze the sales data in report.csv and generate a summary chart")

When you need precise control over the execution flow, use the compose package to build a graph or workflow. You can add different types of nodes (like LambdaNode for custom logic or ChatModelNode for LLM calls) and define edges to connect them. Once compiled, the graph can be invoked directly.

graph := compose.NewGraph[*Input, *Output]()
graph.AddLambdaNode("validate", validateFn)
graph.AddChatModelNode("generate", chatModel)
graph.AddLambdaNode("format", formatFn)

graph.AddEdge(compose.START, "validate")
graph.AddEdge("validate", "generate")
graph.AddEdge("generate", "format")
graph.AddEdge("format", compose.END)

runnable, _ := graph.Compile(ctx)
result, _ := runnable.Invoke(ctx, input)

You can enhance a ChatModelAgent by providing a list of tools via ToolsConfig. The agent will automatically manage tool calls within its reasoning loop.

agent, _ := adk.NewChatModelAgent(ctx, &adk.ChatModelAgentConfig{
    Model: chatModel,
    ToolsConfig: adk.ToolsConfig{
        ToolsNodeConfig: compose.ToolsNodeConfig{
            Tools: []tool.BaseTool{weatherTool, calculatorTool},
        },
    },
})

You can wrap a compiled compose.Graph into a tool using graphtool.NewInvokableGraphTool. This allows an autonomous agent (like a ChatModelAgent) to call a deterministic, precisely controlled business process as if it were a standard tool.

tool, _ := graphtool.NewInvokableGraphTool(graph, "data_pipeline", "Process and validate data")

agent, _ := adk.NewChatModelAgent(ctx, &adk.ChatModelAgentConfig{
    Model: chatModel,
    ToolsConfig: adk.ToolsConfig{
        ToolsNodeConfig: compose.ToolsNodeConfig{
            Tools: []tool.BaseTool{tool},
        },
    },
})

To extend a ChatModelAgent with specific capabilities, pass a ToolsConfig containing a list of tool.BaseTool implementations to the adk.ChatModelAgentConfig.

agent, _ := adk.NewChatModelAgent(ctx, &adk.ChatModelAgentConfig{
    Model: chatModel,
    ToolsConfig: adk.ToolsConfig{
        ToolsNodeConfig: compose.ToolsNodeConfig{
            Tools: []tool.BaseTool{weatherTool, calculatorTool},
        },
    },
})

You can bridge deterministic workflows with autonomous agents by converting a compiled graph into a tool using graphtool.NewInvokableGraphTool. This allows an agent to decide when to trigger a specific domain-specific pipeline.

tool, _ := graphtool.NewInvokableGraphTool(graph, "data_pipeline", "Process and validate data")

agent, _ := adk.NewChatModelAgent(ctx, &adk.ChatModelAgentConfig{
    Model: chatModel,
    ToolsConfig: adk.ToolsConfig{
        ToolsNodeConfig: compose.ToolsNodeConfig{
            Tools: []tool.BaseTool{tool},
        },
    },
})