repository·main·Indexed 40 days ago
https://github.com/secretflow/kusciaA lightweight, K3s-based orchestration framework for privacy-preserving computing tasks. Kuscia provides a unified HTTP/GRPC API layer to manage secure, cross-domain collaborative computing across heterogeneous infrastructures, featuring capabilities like DataMesh for data source management and integration with Prometheus and Grafana for performance monitoring.
The Data Mesh API is designed for privacy computing applications (such as SecretFlow and TrustedFlow) to provide capabilities for retrieving DomainData and DomainDataSource information. Privacy computing platforms (like SecretPad) can leverage the Kuscia API to implement broader functionalities.
Unlike the standard Kuscia API which resides on the master, the Data Mesh API is located on the Domain side.
Kuscia (Kubernetes-based Secure Collaborative InfrA) is a lightweight privacy-preserving computation task orchestration framework based on K3s. It is designed to abstract away heterogeneous infrastructure and protocols, providing a unified foundation for privacy-preserving computation.
Key capabilities include:
Kuscia (Kubernetes-based Secure Collaborative InfrA) is a lightweight privacy-preserving computing task orchestration framework built on K3s. It is designed to abstract away heterogeneous infrastructure and protocols, providing a unified foundation for privacy-preserving computing tasks.
Key capabilities include:
The DomainData API allows for the management of data objects within a Kuscia Domain. These APIs are part of the Data Mesh functionality, enabling Domain-side management of data.
Available operations include creating, updating, deleting, querying, and listing data objects. There is also a specialized method for deleting both the data object and its underlying physical data.
Kuscia is a cross-domain privacy computing task orchestration framework designed to solve common industrialization challenges. It acts as a framework layer that sits between product interfaces (like SecretPad) and privacy computing engines (like SecretFlow, TrustedFlow, or SCQL).
The Kuscia Serving API allows you to manage persistent, general-purpose services (such as Secretflow Serving) within the Kuscia ecosystem. The API is defined via protobuf and supports lifecycle management including creation, querying, updating, and deletion of services.
Available methods include:
CreateServing: Creates a new serving instance.QueryServing: Retrieves details of a specific serving instance.UpdateServing: Modifies an existing serving instance.DeleteServing: Removes a serving instance.BatchQueryServingStatus: Performs a bulk query of the statuses of multiple serving instances.Kuscia (Kubernetes-based Secure Collaborative InfrA) is a lightweight privacy computing task orchestration framework based on K3s. It abstracts heterogeneous infrastructure and protocols to provide a unified foundation for resource management, application scheduling, container loading, service discovery, and data security.
The cluster consists of two main parts:
Job Controller: Parses Job DAGs, orchestrates multi-task scheduling, and collects task status.Task Controller: Parses task descriptions and implements multi-party Co-Scheduling (corresponds to vertices in a Job DAG).Kuscia Scheduler: Handles Co-Scheduling of multi-party Pods with an 'All-or-Nothing' effect.Domain Controller: Manages node resources and assigns Namespaces.DomainRoute Controller: Manages routing rules, identity authentication, and authorization policies between nodes and the Master.InterConn Controllers: Enables connectivity between different control planes for multi-party task scheduling.Data Controller (Not Open Source): Manages data authorization.Serving Controller (Not Open Source): Orchestrates and schedules resident task flows.DomainRoute Controller.Nodes are privacy computing nodes composed of working machines (VMs or physical) hosting privacy computing application Pods. They are categorized by networking mode:
Lite Node Sub-components:
CoreDNS (service discovery), DomainRoute (routing/auth), Envoy (traffic proxy), and Transport (message queue communication).Kuscia is a lightweight, cross-domain privacy computing task orchestration framework designed to serve as the runtime foundation for privacy computing engines. It addresses common challenges in the lifecycle of privacy computing tasks, including deployment, execution, and maintenance.
Key capabilities include:
In Kuscia, a Domain represents a privacy computing node. A single Domain can contain multiple K3s worker nodes (Nodes).
Kuscia leverages the K3s Namespace mechanism to manage node permissions and resource isolation; every Domain corresponds to a specific K3s Namespace. This allows users to manage and maintain privacy computing nodes through Kubernetes-native resources.
A KusciaJob represents a task workflow in Kuscia. It is structured as a Directed Acyclic Graph (DAG) of KusciaTask objects. For any edge a -> b in the graph, task a is considered a dependency of task b.
The KusciaJob Controller automatically manages the lifecycle and creation of these tasks based on a specified scheduleMode until all tasks have either completed or failed. Each KusciaTask within the job executes a collaborative computing operator across multiple parties.
# Conceptual DAG Example:
# Privacy Intersection (PSI) -> Random Split -> Model Training -> PredictionKusciaDeployment (KD) is the resource used to manage resident services (常驻服务) in Kuscia, such as federated prediction services.
When you create a KusciaDeployment, the KusciaDeployment Controller automatically creates associated K3s resources (like Service, Deployment, ConfigMap, and Pod) on the participating nodes based on the KD specification. This allows you to manage application configurations, replica counts, resource sizes, images, and update strategies across multiple parties easily.