hwloc (Hardware Locality)
repository·master·Indexed 20 days ago
https://github.com/open-mpi/hwlocA software project providing a C API and command-line tools to discover and consult the hierarchical map of computing elements in parallel architectures, including NUMA nodes, caches, cores, and processing units. It includes tools like lstopo for topology visualization and hwloc-ps for process and thread binding. The repository provides CMake and MSBuild support for building the library on Windows.
What's inside hwloc
- hwloc (Hardware Locality) is a software project designed to ease the discovery of hardware resources in parallel architectures. It provides a C API and command-line tools to consult the hierarchical map of computing elements within a node, including NUMA memory nodes, shared caches, processor packages, dies, cores, processing units (logical processors or "threads"), and I/O devices. It is primarily used by High-Performance Computing (HPC) applications to exploit code and data locality.
Use the hwloc C API
masterThe hwloc C API allows for manipulating topologies and objects. The primary headers are:
hwloc.h: The basic interface.hwloc/helper.h: Higher-level functions to simplify object manipulation and link traversal.hwloc/inlines.h: Contains inline code for routines inhwloc.h.hwloc/bitmap.h: Extensive interface for manipulating CPU bitmaps used for binding.
Important Note on Portability: Developers should not make hard assumptions about the topology structure (e.g., assuming a PU is always a descendant of a Core) because different operating systems or kernel versions may report different levels of detail. Always use the API to discover the hierarchy dynamically.
Run hwloc-ps on a remote server
masterTo debug process placement on a remote machine, you must ensure
hwloc-psandlstopoare installed on that remote server.Because the web client runs locally, you must:
- Copy the exported SVG file from the remote server to your local machine using
scp. - If the remote port
8888is not directly accessible, use SSH port forwarding to tunnel the traffic to your local machine.
# 1. Copy SVG from remote to local $ scp target:/path/to/filename.svg . # 2. Forward port 8888 if not directly accessible $ ssh target -L 8888:localhost:8888 -Nf- Copy the exported SVG file from the remote server to your local machine using
Install Node.js and dependencies for hwloc-ps.www
masterThe
hwloc-ps.wwwclient requiresnpmto install its dependencies. You can installnpmvia your system's package manager:- Debian/Ubuntu:
apt install npm - RHEL/CentOS:
yum install npm
Once
npmis installed, navigate to thehwloc-ps.wwwdirectory and runnpm installto set up the environment forclient.js.# Install npm $ apt install npm # or $ yum install npm # Install JS dependencies $ cd /path/to/hwloc-ps.www/ $ npm install- Debian/Ubuntu:
Verify hwloc installation
masterTo check if hwloc is working correctly on your machine, build the project and run thelstopoorlstopo-no-graphicscommand-line tools. If the output shows incorrect or missing information (such as cache details), consult the project's issue tracker.Build hwloc on Windows using CMake
masterTo build
hwlocon Windows, use CMake to configure the project, build the binaries, and install them. You can specify the build type (e.g.,ReleaseorDebug) and the installation directory.Note that the
Debugbuild type enables verbose debug messages and assertions, which is useful for development but not recommended for production use.# Configure the project cmake [options] -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=\path\to\install . # Build the project cmake --build . # Install the project cmake --build . --target INSTALLBuild hwloc using MSBuild on Windows
masterYou can build the
hwloc.slnsolution directly from the command line using MSBuild. Ensure you have the%MSBUILD_PATH%environment variable set or provide the full path to the MSBuild executable.Note: This Visual Studio support is currently experimental.
%MSBUILD_PATH%\MSBuild hwloc.sln /p:Configuration=Release /p:Platform=x64Basic usage of hwloc-ps to display process and thread binding
masterTo visualize process and thread binding in an interactive web page, follow these three steps:
- Start the JSON server: Run
hwloc-psin server mode to host the topology data. - Export the topology: Use
lstopoto generate an SVG file of your hardware topology using thenativesvgformat. - Run the client: Use
node client.jspointing to your SVG file to launch the web interface.
If the browser does not open automatically, use the
-uflag with the client to see the URL (typicallyhttp://localhost:3000).# 1. Start the JSON server $ hwloc-ps --json-server # 2. Export topology as SVG $ lstopo filename.svg --of nativesvg # 3. Run the client $ cd /path/to/hwloc-ps.www/ $ node client.js /path/to/filename.svg- Start the JSON server: Run
Visualize topology with lstopo
masterThe
lstopotool can display the hardware topology in several formats:- Graphical mode: Uses X11.
- Textual mode: Human-readable hierarchy.
- Export formats: Plain text, LaTeX tikzpicture, PDF, PNG, and FIG (note that some formats require additional support libraries).
Configure hwloc build options
masterWhen building
hwlocwith CMake on Windows, you can use the following flags to skip specific components:-DHWLOC_SKIP_LSTOPO=1: Skips buildinglstopo.-DHWLOC_SKIP_TOOLS=1: Skips building otherhwloccommand-line tools.-DHWLOC_SKIP_INCLUDES=1: Skips installing header files.
Resolve Platform Toolset errors in Visual Studio
masterIf you encounter an error stating that the specified platform toolset cannot be found (e.g.,
"The builds tools for v142 (Platform Toolset = 'v142') cannot be found."), you can force MSBuild to use an older toolset by passing the/p:PlatformToolsetargument.%MSBUILD_PATH%\MSBuild hwloc.sln /p:Configuration=Release /p:Platform=x64 /p:PlatformToolset=v110Example: Basic topology traversal and binding
masterThe following C example demonstrates how to initialize a topology, walk the tree, count packages, calculate cache sizes, and perform CPU and memory binding. This example is based on
doc/examples/hwloc-hello.c./* * Example hwloc API program. * See other examples under doc/examples/ in the source tree */ #include "hwloc.h" #include <errno.h> #include <stdio.h> #include <string.h> #include <stdlib.h> static void print_children(hwloc_topology_t topology, hwloc_obj_t obj, int depth) { char type[32], attr[1024]; unsigned i; hwloc_obj_type_snprintf(type, sizeof(type), obj, 0, topology); printf("%*s%s", 2*depth, "", type); if (obj->os_index != (unsigned) -1) printf("#%u", obj->os_index); hwloc_obj_attr_snprintf(attr, sizeof(attr), obj, " ", 0, topology); if (*attr) printf("(%s)", attr); printf("\n"); for (i = 0; i < obj->arity; i++) { print_children(topology, obj->children[i], depth + 1); } } int main(void) { int depth; unsigned i, n; unsigned long size; int levels; char string[128]; int topodepth; void *m; hwloc_topology_t topology; hwloc_cpuset_t cpuset; hwloc_obj_t obj; /* Allocate and initialize topology object. */ hwloc_topology_init(&topology); /* Perform the topology detection. */ hwloc_topology_load(topology); /* Get topology depth. */ topodepth = hwloc_topology_get_depth(topology); /***************************************************************** * First example: Walk the topology with an array style *****************************************************************/ for (depth = 0; depth < topodepth; depth++) { printf("*** Objects at level %d\n", depth); for (i = 0; i < hwloc_get_nbobjs_by_depth(topology, depth); i++) { hwloc_obj_type_snprintf(string, sizeof(string), hwloc_get_obj_by_depth(topology, depth, i), 0, topology); printf("Index %u: %s\n", i, string); } } /***************************************************************** * Second example: Walk the topology with a tree style *****************************************************************/ printf("*** Printing overall tree\n"); print_children(topology, hwloc_get_root_obj(topology), 0); /***************************************************************** * Third example: Print the number of packages *****************************************************************/ depth = hwloc_get_type_depth(topology, HWLOC_OBJ_PACKAGE); if (depth == HWLOC_TYPE_DEPTH_UNKNOWN) { printf("*** The number of packages is unknown\n"); } else { printf("*** %u package(s)\n", hwloc_get_nbobjs_by_depth(topology, depth)); } /***************************************************************** * Fourth example: Compute cache size above the first logical processor *****************************************************************/ levels = 0; size = 0; for (obj = hwloc_get_obj_by_type(topology, HWLOC_OBJ_PU, 0); obj; obj = obj->parent) if (hwloc_obj_type_is_cache(obj->type)) { levels++; size += obj->attr->cache.size; } printf("*** Logical processor 0 has %d caches totaling %luKB\n", levels, size / 1024); /***************************************************************** * Fifth example: Bind to only one thread of the last core *****************************************************************/ depth = hwloc_get_type_or_below_depth(topology, HWLOC_OBJ_CORE); obj = hwloc_get_obj_by_depth(topology, depth, hwloc_get_nbobjs_by_depth(topology, depth) - 1); if (obj) { cpuset = hwloc_bitmap_dup(obj->cpuset); hwloc_bitmap_singlify(cpuset); if (hwloc_set_cpubind(topology, cpuset, 0)) { char *str; int error = errno; hwloc_bitmap_asprintf(&str, obj->cpuset); printf("Couldn't bind to cpuset %s: %s\n", str, strerror(error)); free(str); } hwloc_bitmap_free(cpuset); } /***************************************************************** * Sixth example: Memory binding on the last NUMA node *****************************************************************/ n = hwloc_get_nbobjs_by_type(topology, HWLOC_OBJ_NUMANODE); obj = hwloc_get_obj_by_type(topology, HWLOC_OBJ_NUMANODE, n - 1); size = 1024*1024; m = hwloc_alloc_membind(topology, size, obj->nodeset, HWLOC_MEMBIND_BIND, HWLOC_MEMBIND_BYNODESET); hwloc_free(topology, m, size); m = malloc(size); hwloc_set_area_membind(topology, m, size, obj->nodeset, HWLOC_MEMBIND_BIND, HWLOC_MEMBIND_BYNODESET); free(m); /* Destroy topology object. */ hwloc_topology_destroy(topology); return 0; }