LitePCIe Documentation

To achieve zero-copy data transfers, you can mmap the DMA buffers of a LitePCIe channel. The memory layout is determined by DMA_BUFFER_TOTAL_SIZE and DMA_BUFFER_COUNT.

Memory Layout:

• TX Buffers (Host to FPGA): Mapped at offset 0. These correspond to the reader_addr buffers in the kernel.
• RX Buffers (FPGA to Host): Mapped at offset DMA_BUFFER_TOTAL_SIZE. These correspond to the writer_addr buffers in the kernel.

Important Note on Cache Coherency: On certain architectures (like ARM/AArch64), the memory is mapped as cached. Users must explicitly flush CPU caches when performing transfers to ensure data consistency between the CPU and the FPGA.

/* Conceptual mmap usage */
int fd = open("/dev/litepcie0", O_RDWR);
size_t total_size = DMA_BUFFER_TOTAL_SIZE;

// Map RX buffers (FPGA -> Host)
void *rx_ptr = mmap(NULL, total_size, PROT_READ, MAP_SHARED, fd, total_size);

// Map TX buffers (Host -> FPGA)
void *tx_ptr = mmap(NULL, total_size, PROT_WRITE, MAP_SHARED, fd, 0);

The LitePCIe kernel driver exposes DMA channels as character devices in /dev/litepcieN, where N is the channel index. Users can interact with these channels using standard file operations:

• read(): Reads data from the DMA writer (the data sent from the FPGA to the host). It blocks until data is available or returns -EAGAIN if O_NONBLOCK is set.
• write(): Writes data to the DMA reader (the data sent from the host to the FPGA). It blocks until there is space in the DMA buffers or returns -EAGAIN if O_NONBLOCK is set.
• mmap(): Maps the DMA buffers directly into the user's address space for high-performance zero-copy access. The mapping is split into two parts:
  • Offset 0: The TX (Transmit) buffers (DMA Reader buffers).
  • Offset DMA_BUFFER_TOTAL_SIZE: The RX (Receive) buffers (DMA Writer buffers).
• poll(): Allows monitoring the device for readiness (data available to read or space available to write).
• ioctl(): Used for advanced control, such as enabling/disabling DMA, setting loopback mode, or accessing hardware registers.

/* Example conceptual usage of the character device */
int fd = open("/dev/litepcie0", O_RDWR);

// Read data from FPGA
char buffer[DMA_BUFFER_SIZE];
ssize_t bytes_read = read(fd, buffer, sizeof(buffer));

// Write data to FPGA
ssize_t bytes_written = write(fd, buffer, sizeof(buffer));

// Use poll to wait for data
struct pollfd pfd;
pfd.fd = fd;
pfd.events = POLLIN;
poll(&pfd, 1, -1);

close(fd);

The LitePCIe driver provides several ioctl commands to control hardware features and DMA state. Common commands include:

• LITEPCIE_IOCTL_REG: Read or write a specific 32-bit register address on the LitePCIe device. Requires a struct litepcie_ioctl_reg argument.
• LITEPCIE_IOCTL_DMA: Enables or disables DMA loopback mode. Requires a struct litepcie_ioctl_dma argument.
• LITEPCIE_IOCTL_DMA_WRITER: Enables/disables the DMA writer and retrieves current hardware/software buffer counts. Requires struct litepcie_ioctl_dma_writer.
• LITEPCIE_IOCTL_DMA_READER: Enables/disables the DMA reader and retrieves current hardware/software buffer counts. Requires struct litepcie_ioctl_dma_reader.
• LITEPCIE_IOCTL_MMAP_DMA_INFO: Retrieves information about DMA buffer offsets and sizes for correct mmap usage. Requires struct litepcie_ioctl_mmap_dma_info.
• LITEPCIE_IOCTL_MMAP_DMA_WRITER_UPDATE / LITEPCIE_IOCTL_MMAP_DMA_READER_UPDATE: Updates the software-side buffer counts. Requires struct litepcie_ioctl_mmap_dma_update.
• LITEPCIE_IOCTL_LOCK: Manages exclusive access to reader/writer channels. Requires struct litepcie_ioctl_lock.

Note: Some commands like LITEPCIE_IOCTL_FLASH (SPI) and LITEPCIE_IOCTL_ICAP are conditionally available based on hardware configuration.