Xilinx/AMD Solarflare Onload

You can create container images containing compiled userland components from a mkdist tarball using scripts/onload_mkcontainer.

Note: Kernel modules and the control plane must be installed separately.

Usage Patterns:

• Create userland image: ./scripts/onload_mkcontainer --user onload-user:latest *.tgz
• Create image from existing mkdist contents: ./scripts/onload_mkcontainer --source onload-source:latest *.tgz
• Create both source and user images: Specify both --source and --user flags.

Configuration via Environment Variables:

• Set custom UBI base images: USER_DOCKER_EXTRA_ARGS="--build-arg BUILDER_UBI_BASE=... --build-arg USER_BASE=..."
• Build a debug version: USER_DOCKER_EXTRA_ARGS="--build-arg ONLOAD_BUILD_PARAMS=--debug"

These images are designed for reuse in compatible libc environments (e.g., via COPY --from=) rather than being used as base images (FROM onload-user).

./scripts/onload_mkdist
./scripts/onload_mkcontainer --user onload-user:latest *.tgz

ef_vi is a zero-copy API that provides applications with direct access to the network adapter's datapath. It allows for sending and receiving raw Ethernet frames directly from user-level, bypassing the kernel network stack to achieve minimal CPU overhead and maximum performance.

Use ef_vi when your application requires:

• Very high throughput.
• High-performance packet filtering.
• High-performance data forwarding.

Important Considerations:

• Protocol Implementation: Because ef_vi operates at the Ethernet layer, you must implement your own upper-layer protocols (e.g., IP, UDP, TCP). It is relatively straightforward for UDP, but implementing TCP from scratch is highly complex.
• Coexistence: ef_vi can be used alongside OpenOnload. A common pattern is using ef_vi for performance-critical UDP traffic while using OpenOnload sockets for TCP traffic.
• Zero-Copy: The interface provides direct access to hardware memory buffers, avoiding the copies typically incurred by higher-level socket interfaces.

To use ef_vi, you must first allocate a Virtual Interface (VI), represented by the type ef_vi. A VI consists of three primary components:

1. Receive Descriptor Ring: Used for receiving incoming packets.
2. Transmit Descriptor Ring: Used for sending outgoing packets.
3. Event Queue: A communication channel from the NIC to your software. It notifies you when:
   • Packets arrive from the network.
   • Transmits complete (allowing you to free or reuse buffers).
   • You can retrieve these events by calling ef_eventq_poll().

Memory Requirements:

• Buffers: Packet data buffers must be pinned (to prevent paging) and registered for DMA using an ef_memreg.
• Addressing: The NIC uses ef_addr (a special address space) to identify locations in registered memory.
• Protection Domains (ef_pd): These provide an address space for registered memory. All VIs allocated against the same protection domain can access any memory registered within that domain.

The Onload Cplane Server manages network state (ARP, routes, interfaces, etc.) for Onload-accelerated applications. It operates with the following key behaviors:

Network Namespace Management

If a network namespace file is provided via the command line, the server switches into that namespace before initializing its state. In certain build configurations (CP_SYSUNIT), it may also interact with a 'shim' to manage state across namespaces.

Privilege Dropping

To enhance security, the server can drop privileges after initialization.

• It can drop to a specific UID and GID using --uid and --gid.
• Even when dropping privileges, the server retains CAP_NET_ADMIN to allow for network management tasks like teaming.
• If --track-xdp is enabled, the server also retains CAP_SYS_ADMIN to track XDP programs.

Memory Management

The server uses mmap with MAP_SHARED to create a continuous chunk of memory for the Management Information Base (MIB) tables. This memory is shared between the kernel, the cplane process, and Onloaded user processes, ensuring a consistent view of network state.

The Onload extensions API is accessed via the OnloadExt class, which mimics the documented C interface. All extension methods are static.

Library Loading

When running your Java application, the libOnloadExt.so file must be in your library path. If it is not found, you will encounter a failed to load the jni shared library exception.

You can add the library directory to LD_LIBRARY_PATH or use LD_BIND_NOW to ensure correct location:

export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$ONLOAD_LIB
export LD_BIND_NOW=y

Best Practices and Constraints

• File Descriptors: The wrapper works most effectively when provided with native file descriptors. While some overloads exist for common Java socket classes, you may need to implement your own for custom types.
• Availability Checks: Most calls will throw an exception if the application is not running under Onload. To avoid this, check availability first using:
  • OnloadExt.IsPresent()
  • OnloadZeroCopy.IsZeroCopyEnabled()
• Constants: The API uses public static final int instead of Java enums to maintain compatibility with older Java versions.
• ZeroCopy: The OnloadZeroCopy library requires you to implement a class that provides a Callback function. Note that JNI overhead may reduce the performance gains of ZeroCopy.