CloudWeGo Documentation

CloudWeGo is a collection of high-performance, extensible, and reliable microservices middleware projects developed by ByteDance for building enterprise-class cloud-native architectures. It focuses on microservices communication and governance and includes several key projects:

• Kitex: An RPC framework (Go).
• Hertz: An HTTP framework (Go).
• Netpoll: A basic network library.
• shmipc: An inter-process communication library.
• Volo: A lightweight, high-performance, and scalable Rust-based RPC framework.

CloudWeGo is designed to reduce CPU and memory consumption, accelerate application development through pre-built components, and provide robust security through modern design principles and memory-safe languages like Rust.

Eino ADK is a Go-based Agent development framework designed for building autonomous and orchestrated agents. It provides several core primitives:

• ChatModelAgent: A ReAct-based agent that uses an LLM as a decision-maker. It supports tool calls, autonomous reasoning, and runtime enhancement via Middleware.
• Workflow Agents: Primitives for deterministic orchestration, including Sequential, Loop, and Parallel execution.
• Runner / TurnLoop: The execution entry point that supports event streams, checkpoint/resume capabilities, and multi-turn preemption.
• Multi-Agent Collaboration: Patterns for combining agents, primarily through the AgentAsTool approach or Workflow composition.

Kitex is a high-performance, extensible microservice framework for Golang, part of the CloudWeGo ecosystem. It is a comprehensive RPC framework that includes its own network library, serialization library, and framework implementation.

Key characteristics include:

• gRPC Support: Kitex provides an optimized implementation of the gRPC protocol that leverages official gRPC source code but achieves superior performance compared to the standard official gRPC framework.
• Comprehensive Stack: It covers the entire RPC stack, making it suitable for building high-performance microservices.
• Ecosystem: Part of the CloudWeGo suite, which also includes Hertz (Golang HTTP framework) and Volo (Rust RPC framework).

Eino is a comprehensive, Go-based framework for building Large Language Model (LLM) applications. It is designed to cover the entire DevOps lifecycle, providing a structured way to manage the complexities of LLM orchestration.

Key architectural pillars include:

• Components: Individual LLM domain building blocks (e.g., Chat Models, Tools).
• Orchestration: Powerful capabilities to manage complex workflows, such as directed graphs, concurrency, and streaming data.
• Strong Typing: Built on Golang to ensure code readability, maintainability, and reliability.
• DevOps Lifecycle Support: Includes tools for debugging, tracing, and evaluation.

Eino is used internally at ByteDance for services like Doubao, TikTok, and Coze.

CloudWeGo is a high-performance, scalable, and stable set of microservice middleware developed by ByteDance. It is designed for microservice communication and governance, with strong integration support for cloud-native ecosystems including K8s registry, Prometheus, and OpenTracing.

The project is composed of several independent components that can be used separately, allowing developers to pick only what they need without incurring heavy-weight framework overhead.

Eino is an AI application development framework (Agent Development Kit) implemented in Go. It is designed for building extensible and maintainable AI applications by providing:

• Model abstraction: Unifies interfaces across different LLM providers (OpenAI, Ark, Claude, etc.) so you can switch models without changing business logic.
• Capability composition: Uses replaceable Component interfaces for units like conversation, tools, and retrieval.
• Orchestration: Provides abstractions like Agent, Graph, and Chain for complex workflows.
• Runtime support: Includes built-in streaming, state management, and observability (Callbacks).

Kitex is a high-performance, extensible Go RPC framework developed by ByteDance. It is designed for high-scale service governance and features:

• Integrated Network Library: Uses Netpoll, a self-developed network library based on Epoll.
• Protocol Support: Supports multiple message protocols including Thrift and Protobuf.
• Interactive Models: Supports Ping-Pong, Oneway, and Streaming models.
• Extensibility: Provides a flexible code generator for easy integration and customization.

Hertz is a high-performance, highly extensible, and low-latency Golang HTTP framework designed for building microservices at an enterprise scale. It is built on top of the Netpoll networking library, which provides significant performance advantages over the standard Go net/http package, including a 30%-60% reduction in CPU usage for certain service types.

Key characteristics include:

• High Usability: API design inspired by mainstream frameworks, supporting middleware for both Server and Client, and stream processing for proxy scenarios.
• Extensibility: A layered design with interfaces and default implementations that allow users to expand framework capabilities.
• Stability: Developed and tested at ByteDance scale (10,000+ services, 40M+ peak QPS) with rigorous PR and release processes.

To support production-grade orchestration, Eino provides several tools for tracing, debugging, and visualization:

• Tracing: Built-in tracing callbacks are available. Eino integrates with observability platforms like APMPlus and Langfuse.
• Visualization: IDE plugins are available that allow you to visualize orchestrated graphs during coding and debugging. These tools can also support building graphs via UI drag-and-drop and exporting them directly as Eino code.

The Eino ecosystem is divided into several repositories:

• Eino (github.com/cloudwego/eino): The core framework containing type definitions, stream processing, component abstractions, orchestration, and callback mechanisms.
• EinoExt (github.com/cloudwego/eino-ext): Contains concrete component implementations, callback handlers, evaluators, prompt optimizers, and usage examples.

For advanced development, ByteDance provides:

• Eino Devops: For visual development and debugging.
• EinoExamples: A repository of example applications and best practices.