Distributed Asynchronous Object Storage (DAOS)

The rsvc module provides a generic framework for implementing replicated services, designed to minimize code duplication across different service implementations. It uses a callback-intensive API to extract common logic from the replication process.

Components

• ds_rsvc (daos_srv/rsvc.h): The server-side framework used to implement the service logic, request handling, and leadership event responses.
• dc_rsvc (daos/rsvc.h): The client-side library used by applications (like dc_pool) to discover and communicate with the current service leader.

Note: In some contexts, the client-side library dc_rsvc is still referred to as rsvc; a rename is planned for a future patch.

The DAOS security module centralizes all access and security-related functionality within the DAOS data plane. It manages the lifecycle of authentication and authorization for data plane operations.

Key responsibilities include:

• Facilitating client library requests to the DAOS Agent for authentication credentials.
• Handling server requests to the Control Plane to validate signed client credentials.
• Generating default Access Control Lists (ACLs) for pools and containers.
• Deriving pool and container capabilities by combining ACLs with client credentials.
• Performing access control checks for all pool and container operations.

The DAOS Pipeline API allows for executing queries and aggregation operations directly on the server side. By performing these operations on the server, the API minimizes data movement between the client and the server, as only the matched results are returned to the client.

Warning: The Pipeline API is currently under heavy development and is not intended for production use. The API is subject to breaking changes.

The DAOS Java client provides high-performance access to DAOS storage through two primary submodules:

1. daos-java: A low-level client wrapping DAOS APIs. It is divided into three functional areas:

   • DAOS File (dfs): For POSIX-like file operations.
   • DAOS Object (obj): For direct object storage (Key-Value style).
   • DAOS Event: For managing asynchronous I/O operations.

2. hadoop-daos: A high-level implementation of the Hadoop FileSystem interface based on daos-java, allowing DAOS to be used as a storage backend for Hadoop-based applications.

The DAOS Management Tool (dmg) is a system administration application designed for managing DAOS environments. Unlike standard DAOS client applications that require authentication via a DAOS agent, dmg authenticates directly using local certificates.

Because dmg performs tasks on remote servers via RPC, it has limited software dependencies and is suitable for execution from a login node. It operates as a gRPC client that interacts with multiple daos_server processes in parallel using the DAOS control API.

The DAV (DAOS Allocator for VOS) is an allocator designed for the md_on_ssd phase.

Important Compatibility Note: The current version (Phase 2) supports evictable zones, which introduces a change in the heap layout. Consequently, the Phase 2 allocator is not compatible with the Phase 1 DAV allocator.

To maintain compatibility and support both layouts, the Phase 2 allocator is packaged as a distinct library and is linked to the daos_common_pmem library.

DAOS (Distributed Asynchronous Object Storage) is an open-source software-defined object store optimized for massively distributed Non Volatile Memory (NVM). It is designed to leverage Storage Class Memory (SCM) and NVMe technologies.

Key features include:

• Key-value storage interface
• Transactional non-blocking I/O
• Advanced data protection with self-healing on commodity hardware
• End-to-end data integrity
• Fine-grained data control and elastic storage

DAOS provides several application interfaces and connectors, including:

• Parallel filesystem
• Hadoop/Spark connector
• TensorFlow-IO
• Native Python dictionary bindings
• HDF5
• MPI-IO

The DAOS File System (DFS) provides a POSIX-like API built directly on top of the DAOS API. It encapsulates a namespace within a single DAOS container. To use DFS, you must provide a valid, connected DAOS pool handle and an open container handle where the namespace will reside.

Key characteristics:

• Encapsulation: The entire namespace (directories, files, symlinks) is stored within one DAOS container.
• Structure: Directories and files are represented as DAOS objects within that container.
• Superblock: A reserved object with OID 0.0 stores metadata (SB) about the filesystem, including magic values, versioning, and the root directory OID (1.0).

The common package provides shared utilities used across the DAOS project. It includes tools for:

• File access: Utilities for interacting with the filesystem.
• Collection manipulation: Helpers for managing data structures, including functional programming-style methods.
• Testing: Support for unit tests and mocks.

Note for Developers: To prevent cyclical import dependencies, files within the common package should avoid importing from peer packages unless it is absolutely necessary.

The Data Mover is a collection of tools designed to copy and serialize data across DAOS and POSIX file systems. It supports movement between POSIX, DAOS, and HDF5 containers, as well as serializing DAOS containers into HDF5 files for storage on POSIX filesystems.

Built-in daos command tools:

• daos filesystem copy: Copies between POSIX containers and POSIX filesystems using libdfs.
• daos container clone: Copies any container to a new container using the Object API (libdaos).

External MpiFileutils tools (require MPI support):

• dcp: Copies between POSIX containers/filesystems or between any two DAOS containers.
• dsync: Similar to dcp, but performs incremental copies (only copying differences).
• daos-serialize: Serializes a DAOS container to HDF5 file(s).
• daos-deserialize: Restores a DAOS container from HDF5 files created by daos-serialize.