DOCSEARCHbyAlgolia
Submit Docs

ZStack Documentation

repository·master·Indexed 23 days ago

https://github.com/zstackio/zstack

ZStack is an open-source IaaS (Infrastructure as a Service) software designed to automate datacenters by managing compute, storage, and networking resources through APIs. It features an in-process microservice architecture, an event-driven design using CloudBus, and a lock-free architecture based on consistent hashing and work queues. The platform supports various installation methods (Quick, Manual, and Multi-node) and provides comprehensive query APIs for resource management.

Tokens
20.5K
Snippets
25
Records
93
Agent score
80%

What's inside ZStack

  1. Overview of ZStack Cloud Core Architecture

    master

    The core directory contains the fundamental architectural components of ZStack. The system is designed as an IaaS (Infrastructure as a Service) orchestration engine focused on maintaining metadata consistency across physical and virtual resources.

    Key components within the core module include:

    • In-process Microservices: The core architectural pattern.
    • Asynchronous Architecture: For scalability.
    • Stateless Services: To support horizontal scaling.
    • Messaging System: For inter-service communication.
    • Database Interface: For metadata persistence.
    • Logging System
    • Tagging System
    • Error Code System
    • REST Interface
    • Workflow Engine
    • Webhook Implementation
    • Global Configuration
    • Search Functionality

  2. ZStack RESTful API Overview and HTTP Methods

    master

    ZStack provides native RESTful API support (starting from version 1.9) to replace earlier HTTP RPC APIs. All API URLs begin with the /v1 prefix. The API follows standard HTTP verb usage for CRUD and RPC-style operations:

    • GET: Retrieve resource information. Used for all Query APIs and read-only operations.
    • POST: Create a new resource.
    • PUT: Modify an existing resource or perform RPC-style actions (e.g., starting a virtual machine).
    • DELETE: Remove a resource.

  3. Supported Network Services in ZStack

    master

    ZStack supports a wide variety of network services that can be applied to different network types. The available services include:

    • Security: Security Groups (安全组), Firewall (防火墙)
    • Addressing & Routing: DHCP, Host Routing (主机路由), Static Route Table (静态路由表), OSPF, SNAT, Source-In-Source-Out (源进源出)
    • IP Management: Elastic IP (弹性IP)
    • Connectivity & DNS: CentralizedDNS, IPSec, Port Forwarding (端口转发)
    • Traffic Control & Monitoring: VipQos, NetFlow, Port Mirroring (端口镜像), Multicast (组播)
    • Data: UserData

    Note that the availability of specific services depends on the type of network being used (e.g., Flat Network vs. VPC Network).

  4. Use Database-Based Asynchronous Queues for Global Synchronization

    master

    While in-memory queues handle 99% of needs via consistent hashing, certain tasks require global synchronization across different management nodes. This typically happens when creating resources that depend on other uncreated resources (e.g., Virtual Router VMs in an L3 network).

    Characteristics:

    • Scope: Provides global synchronization across all management nodes.
    • Mechanism: Tasks are stored in a database-backed queue.
    • Type: Only supports asynchronous execution (the next task only runs after the previous one signals completion).
    • Performance: Slower than in-memory queues due to database I/O, but used only for specific, critical synchronization needs.

  5. How stateless services work in ZStack

    master

    ZStack achieves high availability and horizontal scalability (scale out) by implementing a stateless service architecture. Instead of services maintaining knowledge of which resources they manage, ZStack uses a consistent hashing algorithm to route messages to the correct service instance.

    Key Concepts

    • Management Node: A single process that packages all ZStack services (in-process microservices). A management node is a complete ZStack software instance.
    • CloudBus: The central message bus used by services to communicate with each other.
    • Consistent Hashing Ring: Every management node maintains a copy of a ring containing all management node UUIDs. This ring ensures that a specific resource UUID always maps to the same service instance.
    • Statelessness: Services do not need to exchange state or maintain resource ownership information. When a node joins or leaves, the ring is updated via lifecycle events broadcasted through the CloudBus.

    Message Routing

    1. Resource-based messages (e.g., StartVmInstanceMsg): The sender uses the resource's UUID to calculate the destination management node via the consistent hashing algorithm. The message is then sent to a service queue identified by the service name and the destination node's UUID.
    2. Non-resource messages (e.g., CreateVolumeMsg): These can be sent to any node or specifically to the local node.
    3. API messages (e.g., APIStartVmInstanceMsg): These use a well-known service ID api.portal. The api.portal queue is shared by all management nodes, providing automatic load balancing and routing to the correct destination using the consistent hashing ring.

  6. Understand the In-Process Microservice Architecture

    master

    ZStack uses an in-process microservice architecture to balance the benefits of microservices (decoupling, scalability) with the operational simplicity of a monolith. Instead of running services in separate processes or containers, ZStack encapsulates all services within a single process called a Management Node.

    Key Characteristics

    • Decoupled Logic: Services (e.g., Compute, Storage, Network) are logically independent and communicate via an external message bus, similar to traditional microservices.
    • Single Process: All services run within the same memory space of the Management Node, simplifying deployment, upgrades, and configuration.
    • Static vs. Dynamic: Unlike traditional microservices which are dynamic (instances scale up/down automatically), ZStack's services are static; their lifecycle is tied to the Management Node process.
    • Thread Pool Model: Services do not own dedicated threads. Instead, they share a global thread pool. A service only consumes a thread when triggered by a message or HTTP request, and returns the thread to the pool once the task is complete. This prevents resource starvation and ensures that inactive services consume no CPU cycles.
    • Scaling: To scale, you add more Management Nodes. While this adds instances of all services, inactive services do not consume resources due to the asynchronous thread pool model.

  7. Understand Synchronous vs Asynchronous APIs

    master

    ZStack APIs are categorized into two types based on their execution model:

    Synchronous APIs

    All GET method APIs are synchronous. The HTTP response contains the requested resource data immediately.

    Asynchronous APIs

    All non-GET methods (except login) are asynchronous. When you call an asynchronous API, ZStack returns an HTTP 202 Accepted status code and a JSON body containing a location field (the polling URL).

    To get the result of an asynchronous API, you must:

    1. Periodically GET the location URL provided in the initial response.
    2. Handle the following responses from the polling URL:
      • 202 Accepted: The job is still processing; continue polling.
      • 200 OK: The job completed successfully; the body contains the result.
      • 503 Service Unavailable: The job failed; the body contains error details.
      • 404 Not Found: The polling URL has expired (usually after 2 days) or is invalid.

  8. Understand L3 Network Types in ZStack

    master

    ZStack supports three primary types of Layer 3 (L3) networks, each serving different connectivity scenarios:

    • Public Network (公有网络): Networks that can communicate directly with the outside of the data center.
    • Flat Network (扁平网络): Networks that can communicate directly with the data center's physical network.
    • VPC Network (VPC网络): Networks used to isolate tenant cloud host traffic. These communicate with the outside of the data center via a VPC Router.

    Note: This section covers business networks for cloud hosts. Other network types like migration networks or management networks are covered in separate sections.

  9. Understand VPC Router Connectivity States

    master

    ZStack management nodes periodically send heartbeat packets to the VPC router to monitor health. The router's connection status is determined as follows:

    • Connecting (连接中): The heartbeat failed, and the router is attempting to reconnect.
    • Connected (已连接): Reconnection was successful; the router is healthy.
    • Disconnected (失联): Reconnection failed; the router is unreachable.

  10. How DHCP service types are determined in ZStack

    master

    ZStack supports two types of DHCP backends, and the selection depends on the L3 network configuration of the VM's network interface:

    • Flat: A distributed DHCP model where each physical host runs its own DHCP server for the VMs residing on it.
    • vrouter: A centralized DHCP model.

    You can identify the service provider type by querying the L3 network and its associated NetworkServiceProvider.

    To check the services attached to an L3 network:

    admin >>>QueryL3Network uuid='<L3_NETWORK_UUID>'

    To check the details of a specific Network Service Provider (to see if it is type Flat or vrouter):

    admin >>>QueryNetworkServiceProvider uuid='<SERVICE_PROVIDER_UUID>'
    admin >>>QueryL3Network uuid='e82f9446c3e34111b8b74cfcf70f4562'

  11. ZStack Core Concepts and Architecture

    master

    ZStack is an IaaS (Infrastructure as a Service) software designed for full automation and stability. Key architectural concepts include:

    • API-Driven Management: Everything in the datacenter (compute, storage, networking) is managed via APIs. There are no scattered configurations or manual software installations required.
    • Plugin System: Built on an Eclipse/OSGI-like architecture where every feature is a plugin. This ensures that adding or removing features does not impact the core orchestration.
    • Workflow Engine: A system capable of rolling back completed changes if an error occurs during an operation.
    • Cascade Framework: A mechanism that spreads operations from a primary resource to all its dependent resources.
    • Network Functions Virtualization (NFV): The default networking model provides each tenant with a dedicated networking node implemented as a virtual appliance VM, making the model self-contained and self-managed.
    • Scalability: Designed to handle hundreds of thousands of physical servers and millions of virtual machines from a single management node.

  12. How the ZStack Workflow Engine ensures stability

    master

    To prevent systems from being left in inconsistent intermediate states during long-running tasks (like VM creation), ZStack uses a Workflow Engine.

    Key Features:

    • Rollback Mechanism: Every operation is encapsulated into a flow. If a step fails, the engine automatically rolls back all previously executed flows in that workflow to restore the system to a consistent state.
    • XML Configuration: Developers can configure critical tasks using XML files. This allows for high component reuse; for example, a sub-workflow for creating an appliance VM can be swapped into a standard VM creation workflow simply by reconfiguring the XML to replace specific flows.