AMD XDNA Driver

The AMD NPU (Neural Processing Unit) is a multi-user AI inference accelerator integrated into AMD client APUs, designed for efficient execution of Machine Learning applications like CNNs and LLMs. It is managed by the amdxdna driver and is based on the AMD XDNA Architecture.

Key hardware components include:

• AMD XDNA Array: A 2D array of compute and memory tiles. Compute tiles contain VLIW processors, while memory tiles act as L2 memory.
• Microcontroller: Runs NPU Firmware responsible for command processing, partition setup, configuration, and workload orchestration.
• Resource Solver: A component of the amdxdna driver that manages the allocation of the 2D array among various workloads using heuristics and workload metadata.

To run AI applications using the XDNA driver, ensure your system meets the following requirements:

• Processor:
  • Test Machine: RyzenAI processor.
  • Build Machine: Any x86 processor (AMD recommended).
• Operating System:
  • Ubuntu >= 22.04
  • Arch Linux
• Linux Kernel:
  • Version v6.10 or above is required. Note that the driver may not support older versions due to Linux API changes.
• XRT Base Package:
  • An installed XRT base package is required. For best compatibility with the plugin package, it is recommended to build XRT from the xrt submodule within this repository (<root-of-source-tree>/xrt).

The repository contains two independent driver source trees for the AMD XDNA NPU:

1. Upstream (Staging) Driver (drivers/accel/amdxdna/): This is the primary driver being upstreamed to the Linux kernel. It is compiled as amdxdna.ko via DKMS and is the recommended driver for most users.
2. Out-of-Tree (OOT/Legacy) Driver (src/driver/amdxdna/): Maintained for compatibility and bring-up. It is compiled as amdxdna_legacy.ko.

When running ./build.sh -release, both trees are built. The plugin package includes the primary amdxdna.ko and the amdxdna_legacy.ko for compatibility.

The library uses a timeline fence synchronization mechanism to manage asynchronous command completion. The flow typically follows these steps:

1. Command Submission: The guest submits an exec_cmd, which returns a sequence number.
2. Synchronization Point: The guest submits a wait_cmd to set a sync point and a timeout.
3. Fence Submission: The guest submits a fence. A dedicated polling thread per hardware context waits on a syncobj.
4. Completion: When the fence signals, the write_context_fence callback is triggered to notify the guest.

Each vxdna_hwctx (Hardware Context) manages the DRM hardware context handle, a timeline syncobj for fence tracking, a polling thread, and a pending fence queue with a condition variable.

An NPU application workload consists of two distinct binaries generated by the NPU compiler:

1. AMD XDNA Array overlay: Used to configure the NPU spatial partition. It contains instructions for stream switch configuration and an ELF for the compute tiles. It is loaded onto the spatial partition by the associated ERT instance.
2. ctrlcode: Used for orchestrating the loaded overlay. It is executed by the ERT running in protected mode on the microcontroller. The ctrlcode is composed of a sequence of opcodes known as XAie_TxnOpcode.

The NPU firmware utilizes two types of isolated contexts to manage operations:

• MERT (Management ERT): A single, isolated, privileged context used to service management commands from the amdxdna driver (e.g., setting up contexts, telemetry, error handling). It is bound to a single mailbox and a single MSI-X interrupt.
• ERT (Execution Runtime): A dedicated, non-privileged context instance created for each workload context. It is used to execute the user-provided ctrlcode. Each ERT is bound to its own dedicated mailbox and its own MSI-X interrupt.

To build the vxdna library, you need CMake >= 3.10, a C++17 compiler, libdrm development headers, and the pthread library. You can build using CMake directly or via the provided build script.

Using CMake

# From xdna-driver root
mkdir build && cd build
cmake -DBUILD_VXDNA=ON ..
make vxdna

Using the build script

./build/build.sh -vxdna

Build Options

• BUILD_VXDNA: Set to ON to enable the vxdna library build (default is OFF).
• BUILD_VXDNA_TESTING: Set to ON to build and run unit tests.

After building, install the library and its associated files using the following command:

cmake --install build --component xrt_plugin

This installs:

• libvxdna.so.1.0.0 (Shared library)
• vaccel.h (Public header)
• vxdna.pc (pkg-config file)

Ubuntu 24.04 requires a kernel update to meet the XDNA driver requirements. You can update to Linux 6.11 by installing the Hardware Enablement (HWE) stack.

sudo apt update 
sudo apt install --install-recommends linux-generic-hwe-24.04
sudo reboot

If you need to manually build a Linux kernel (e.g., for Ubuntu 22.04), ensure you enable CONFIG_DRM_ACCEL and CONFIG_AMD_IOMMU.

Steps:

1. Clone the Linux source (e.g., v6.10).
2. Configure the kernel using your current config or scripts/config.
3. Build the Debian packages.

# Clone Linux source
git clone --depth=1 --branch v6.10 git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git

# Configure kernel (Option step)
cp /boot/config-`uname -r` <your_build_dir>/.config

# Enable required configs
scripts/config --file .config --enable DRM_ACCEL
scripts/config --file .config --enable AMD_IOMMU

# Build kernel packages
make -j `nproc` bindeb-pkg

# Reinstall the generated packages
sudo apt reinstall ../linux-headers-6.10.0_6.10.0-1_amd64.deb ../linux-image-6.10.0_6.10.0-1_amd64.deb ../linux-libc-dev_6.10.0-1_amd64.deb

To allow applications to allocate large Buffer Objects (BO) on the NPU, you must increase the Linux memlock resource limit. It is recommended to use a drop-in file in /etc/security/limits.d/ so settings survive package upgrades.

# Check current limit (in kbytes)
ulimit -l

# Set to unlimited
sudo mkdir -p /etc/security/limits.d
sudo tee /etc/security/limits.d/99-amdxdna.conf > /dev/null << 'EOF'
* soft memlock unlimited
* hard memlock unlimited
EOF

# Log out and log back in, then verify
ulimit -l