Persistent Memory Development Kit (PMDK)

repository·master·Indexed 23 days ago

https://github.com/pmem/pmdk

A collection of libraries and tools for managing and programming persistent memory devices on 64-bit Linux. Core libraries include libpmem, libpmem2, libpmemobj for transactional object stores, and libpmempool for pool management. It includes utilities like pmemcheck and pmempool, as well as Ansible playbooks for environment setup and configuration on Intel servers.

Tokens
66.3K
Snippets
141
Records
352
Agent score
78%

What's inside PMDK

  1. Overview of PMDK Libraries and Utilities

    master

    The Persistent Memory Development Kit (PMDK) provides libraries and tools for managing and accessing persistent memory devices.

    Core Libraries:

    • libpmem: Low-level persistent memory support.
    • libpmem2: A newer version of libpmem providing low-level support.
    • libpmemobj: Provides a transactional object store, including memory allocation, transactions, and general persistent memory programming facilities.
    • libpmempool: Supports off-line pool management and diagnostics.

    Utilities:

    • pmempool: Used for managing and analyzing persistent memory pools.
    • pmemcheck: A Valgrind tool used for detecting errors in persistent memory usage.

    Platform Support: Currently, these libraries and utilities are designed for 64-bit Linux.

  2. Transactional object manipulation in libpmemobj

    master

    The libpmemobj library provides a suite of functions and macros for performing transactional object manipulation on persistent memory. This allows developers to group multiple operations into an atomic unit that either succeeds completely or fails without leaving the persistent state inconsistent.

    Key capabilities include:

    • Starting and ending transactions (pmemobj_tx_begin, pmemobj_tx_end).
    • Managing transaction lifecycle via macros (TX_BEGIN, TX_ONABORT, TX_ONCOMMIT, TX_FINALLY, TX_END).
    • Explicitly committing or aborting transactions (pmemobj_tx_commit, pmemobj_tx_abort).
    • Handling locks within transactions (pmemobj_tx_lock, pmemobj_tx_xlock).
    • Tuning the transaction log (pmemobj_tx_log_append_buffer, pmemobj_tx_log_auto_alloc).
    • Attaching user data to the transaction context (pmemobj_tx_set_user_data).
    • Configuring failure behavior (pmemobj_tx_set_failure_behavior).
  3. Use libpmem for low-level persistent memory support

    master

    libpmem is a low-level library for applications using direct access storage (DAX) to interact with persistent memory (pmem) via load/store access. It is designed for applications that manage their own memory allocation and transactions.

    Note: For most applications, it is recommended to use the higher-level libpmemobj library, which provides memory allocation and transactional operations on variable-sized objects.

    To use libpmem, include the header and link against the library:

    #include <libpmem.h>
    // Compile with: cc ... -lpmem
  4. Use pmempool for Persistent Memory pool management

    master

    The pmempool tool is a management utility for Persistent Memory pool files created by PMDK libraries. It is used for offline analysis, manipulation, testing, and debugging of pools. It operates via a set of subcommands that can either be non-destructive (read-only) or modify/create pools.

    $ pmempool [--help] [--version] <command> [<args>]
  5. What is pmreorder and how does it work?

    master

    pmreorder is a collection of Python scripts used to perform persistent consistency checks by replaying and reordering operations logged by pmemcheck.

    It works by performing store reordering between persistent memory barriers (sequences of flush-fence operations). It uses a consistency checking routine to verify if files remain in a consistent state under different reordering scenarios.

    To facilitate debugging, pmreorder sets the PMREORDER_MARKERS environment variable, which contains a subset of markers passed from the application, separated by vertical bars (|).

  6. Warning: libpmem2 persist operations are not transactional

    master

    When using pmem2_get_persist_fn(3) to flush changes to raw, memory-mapped persistent memory, be aware that these operations are not transactional.

    If a program is interrupted during a persist operation, it may result in a partial write (torn update) to persistent memory. To ensure atomicity and avoid torn updates, use a transactional library such as libpmemobj(7) instead of raw libpmem2 calls.

  7. Iterate through objects in a pmemobj pool

    master

    The libpmemobj container operations allow you to traverse the internal collection of objects stored in a persistent memory pool (PMEMobjpool). You can iterate through all allocated objects or filter by a specific type.

    Important: Do not assume any specific order for objects in the internal containers.

    To safely delete objects while iterating, use the _SAFE versions of the macros, which preserve a handle to the next object before the current one is processed.

  8. How type-safe persistent atomic lists work

    master

    The libpmemobj library provides macros to define and operate on type-safe, non-transactional, persistent, circular doubly linked lists. These lists consist of persistent objects of a specific type, ensuring type enforcement by preventing the mixing of different object types in a single list.

    Core Components

    1. List Head: A structure defined by POBJ_LIST_HEAD(HEADNAME, TYPE) that contains a handle to the first element (pe_first) and a PMEMmutex lock.
    2. List Entry: A field within your user-defined structure that enables linking. You must include a field of type POBJ_LIST_ENTRY in your structure. This is declared using POBJ_LIST_ENTRY(TYPE).

    Mental Model

    • Type Safety: When you declare a list head for a specific TYPE, all elements in that list must be of that TYPE.
    • Connectivity: The POBJ_LIST_ENTRY field acts as the glue, containing the pe_next and pe_prev handles used to traverse the circular list.
    • Atomicity: Certain macros (like the INSERT_NEW_* family) are designed to atomically allocate and insert objects.
  9. Configure libpmemobj via environment variables

    master

    You can configure libpmemobj write entry points using external configuration methods instead of direct function calls. This is useful for changing library behavior without recompiling your application.

    There are two primary methods:

    1. Direct Configuration: Set the PMEMOBJ_CONF environment variable to a configuration string.
    2. File-based Configuration: Set the PMEMOBJ_CONF_FILE environment variable to the path of a file containing a sequence of ctl queries.

    For detailed syntax on the queries used in these configurations, refer to the pmem_ctl(5) man page.

  10. Understand the default libpmem logging behavior

    master

    If you do not set a custom function, libpmem uses a default logging function that performs the following:

    1. Syslog: Sinks all logging messages into syslog(3) unconditionally.
    2. Stderr: Sinks messages into stderr(3) only if:
      • The message level is not less severe than the PMEM_LOG_THRESHOLD_AUX threshold.
      • The message level is not PMEM_LOG_LEVEL_HARK.

    Note that the global PMEM_LOG_THRESHOLD still applies: any message less severe than this threshold is discarded before reaching the logging function (default or custom).

  11. Important: Avoid non-transactional allocations inside transactions

    master

    All pmemobj_* allocation and deallocation functions are atomic and fail-safe. However, they should not be used inside an open transaction.

    Why?

    • If executed within a transaction, these operations are considered durable immediately upon completion.
    • They are not rolled back if the transaction is aborted or interrupted.
    • Because these functions have no knowledge of the transactional API's state, mixing transactional and non-transactional updates to the same data within a single transaction will likely lead to data corruption upon a transaction abort.
  12. How delayed atomicity actions work in libpmemobj

    master

    The delayed atomicity mechanism allows you to prepare a set of persistent changes and delay their publication to the persistent state until a specific moment. This is useful for implementing algorithms with relaxed consistency guarantees.

    Key Concepts

    • Atomicity: Publication is fail-safe atomic for the entire collection of actions. If the program exits before publication or if actions are canceled, any reserved resources are automatically released back to the pool.
    • Action Representation: A single action is represented by a struct pobj_action. Functions that create actions populate this structure, while functions that publish actions take an array of these structures and the array size.
    • Thread Safety: Actions can be created in one thread and published in another.
    • Lifecycle: When creating an action, the act argument must be a non-NULL pointer to a struct pobj_action. This structure must not be modified or deallocated until after the actions have been published.

    Workflow Example

    1. Reserve: Use pmemobj_reserve() to reserve space. The object can be modified in memory, but changes are not yet part of the persistent state's atomic view.
    2. Prepare: Use pmemobj_set_value() or pmemobj_defer_free() to queue the intended changes.
    3. Publish: Use pmemobj_publish() to atomically apply all queued actions to the persistent state.
    4. Cancel: If something goes wrong before publication, use pmemobj_cancel() to release reserved resources and invalidate the actions.
    /* reserve, populate and persist the first object */
    PMEMoid tail = pmemobj_reserve(pop, &actv[0], sizeof(struct list_node), 0);
    if (TOID_IS_NULL(tail))
    	return -1;
    D_RW(tail)->value = 1;
    D_RW(tail)->next = OID_NULL;
    pmemobj_persist(pop, D_RW(tail), sizeof(struct list_node));
    
    /* ... prepare other actions ... */
    
    /* atomically publish the above actions */
    pmemobj_publish(pop, actv, 4);