DOCSEARCHbyAlgolia
Submit Docs

Xilinx SystemC TLM Co-Simulation Demo

repository·master·Indexed 41 days ago

https://github.com/xilinx/systemctlm-cosim-demo

A co-simulation environment for the VPK120 evaluation board that combines QEMU (simulating a Versal APU with Linux) and SystemC/TLM-2.0 models of DCMAC and AXIDMA hardware. It enables software development and verification of the Digital Controller MAC and AXI DMA components without physical hardware, utilizing the AMD-EDF SDK and Remote-Port protocol for communication between the simulated SoC and SystemC models.

Tokens
17K
Snippets
40
Records
47
Agent score
52%

What's inside systemctlm-cosim-demo

  1. Overview of pcie-ats-demo

    master

    The pcie-ats-demo demonstrates an x86 host co-simulating with an attached PCIe MD5 accelerator.

    • Host: Emulated by QEMU.
    • Accelerator: Modeled in SystemC as a PCIe End-point.
    • ATS Mechanism: The accelerator contains an Address Translation Cache (ATC) and issues PCIe Address Translation Service (ATS) requests to the host to obtain Virtual-to-Physical address translations from the host's IOMMU. This allows the accelerator to perform memory transactions using pre-translated addresses.

  2. Overview of QEMU+SystemC DCMAC CoSim Demo

    master

    This project provides a Co-Simulation (CoSim) environment for the DCMAC (Digital Controller MAC) and AXIDMA components on the VPK120 Evaluation board. It uses QEMU to simulate the Aarch64 Versal APU running an AMD-EDF Linux environment, which interacts with SystemC/TLM-2.0 simplified models of the hardware components.

    CoSim Assumptions

    • Network Configuration: Supports single port configurations at 400G, 200G, or 100G.
    • Features: No 1588 support is included.
    • Hardware/Software Stack: Uses DCMAC + AXIDMA, which is supported by the prebuilt Linux kernel for VPK120.
    • Linux Support: Detailed driver features can be found in the Linux AXI Ethernet driver documentation.

  3. SystemC/TLM-2.0 Models in the CoSim Demo

    master

    The simulation relies on simplified SystemC/TLM-2.0 models for the following components:

    • DCMAC: Located in libsystemctlm-soc/soc/net/ethernet/amd/dcmac/.
    • AXIDMA: Located in libsystemctlm-soc/soc/dma/amd/axidma/.
    • AXIGPIO: Implemented in ./versal_dcmac_demo.cc.
    • Top-level Implementation: Defined in ./versal_dcmac_demo.cc.

  4. High Level Architecture of the CoSim Environment

    master

    The architecture bridges a QEMU-simulated SoC with a Host machine via a virtualized network stack:

    1. QEMU Side:
      • Simulates the Aarch64 Versal APU running AMD-EDF Linux.
      • Simulates System Memory and MMIO Targets.
      • Uses Remote-port Bridges to connect QEMU components to SystemC models.
      • Employs SLiRP for DHCP and TCP/UDP translation, allowing the simulated environment to interact with host sockets.
    2. SystemC Side:
      • Implements hardware models like AXIDMA (DMA Initiator, Scatter-Gather DMA Model, S2MM/MM2S AXI-Stream targets/initiators), DCMAC (MAC-RX/TX, FixedE Model, PHY-RX/TX), and AXIGPIO.
    3. Host Side:
      • A Userspace program communicates via TCP/UDP Network Sockets to the simulated environment.
       +----------------------+      +--------------------+                    +-------------------+
   |       <<QEMU>>       |      |      <<QEMU>>      |                    |      <<HOST>>     |
   | VPK120 System Memory | <--> | Aarch64 Versal APU | <-..-..-..-..-..-> |      TCP/UDP      |<-------
   |      Simulation      |      | with AMD-EDF Linux |  (virtual L7 comm) | Userspace program |        |
   +----------------------+      +--------------------+                    +-------------------+        |
              ^                             ^                                                           |
              |                             |                                                            |
              v                             v                                                            |
   +--------------------+        +--------------------+                                                 |
   |      <<QEMU>>      |        |     MMIO Target    |                                                 |
   |   MEM Initiator    |        | Remote-port Bridge |                                                 |
   | Remote-port Bridge |        +--------------------+                                                 |
   +--------------------+               ^                                                                  |
              ^                             |                                                                  |
              |                             +----------------------+---------------------+              |
              |                             |                      |                     |              |
              V                             v                      v                     v              |
   +--------------------+        +--------------------+   +-----------------+   +------------------+    |
   |   <<SYSC.AXIDMA>>  |        |   <<SYSC.AXIDMA>>  |   |  <<SYSC.DCMAC>> |   | <<DEMO.AXIGPIO>> |
   |    DMA Initiator   |        | MMIO Reg Target    |   | MMIO Reg Target |   | MMIO Reg Target  |    |
   +--------------------+        +--------------------+   +-----------------+   +------------------+    |
              ^                             ^                      ^                     ^              |
              |                             |                      |                     |              |
              v                             v                      |                     |              |
   +----------------------------------+   |                      |                     |              |
   |                   <<SYSC.AXIDMA>> |   |                      |                     |              |
   |             Scatter-Gather DMA Model |   |                      |                     |              |
   +----------------------------------+   |                      |                     |              |
                 ^                    |   |                      |                     |              |
                 |                    v   |                      |                     |              |
        +-----------------+   +------------------+                 |                     |              |
        | <<SYSC.AXIDMA>> |   | <<SYSC.AXIDMA>>  |                 |                     |              |
        | S2MM AXI-Stream |   | MM2S AXI-Stream   |                 |                     |              |
        |      Target     |   |    Initiator     |                 |                     |              |
        +-----------------+   +------------------+                 |                     |              |
                 ^                    |                             |                     |              |
                 |                    v                             |                     |              |
      +-------------------+   +-------------------+                |                     |              |
      |   <<SYSC.DCMAC>>  |   |   <<SYSC.DCMAC>>  |                |                     |              |
      | MAC-RX AXI-Stream |   | MAC-TX AXI-Stream |                |                     |              |
      |     Initiator     |   |     Target        |                |                     |              |
      +-------------------+   +-------------------+                |                     |              |
                 ^                    |                             |                     |              |
                 |                    v                             |                     |              |
      +-------------------------------------------+                |                     |              |
      |              <<SYSC.DCMAC>>               |                |                     |              |
      |                 FixedE                    |<---------------/                                    |
      |                 Model                     |                |                                     |
      +-------------------------------------------+                |                                     |
                 ^                    |                             |                                     |
                 |                    v                             |                                     |
      +-------------------+  +--------------------+                 |                                     |
      |   <<SYSC.DCMAC>>  |  |   <<SYSC.DCMAC>>   |                 |                                     |
      | PHY-RX Target     |  | PHY-TX Initiator   |                 |                                     |
      +-------------------+  +--------------------+                 |                                     |
                 ^                    |                             |                                     |
                 |                    v                             |                                     |
      +-------------------+  +--------------------+                 |                                     |
      |      <<QEMU>>     |  |      <<QEMU>>      |                 |                                     |
      |  remote-port-net |  |  remote-port-net   |                 |                                     |
      |      Initiator   |  |      Target        |                 |                                     |
      +-------------------+  +--------------------+                 |                                     |
                 ^                    |                             |                                     |
                 |                    v                             |                                     |
      +-------------------+  +--------------------+                 |                                     |
      |   <<QEMU.SLiRP>>  |  |   <<QEMU.SLiRP>>   |                 |                                     |
      |    Ethernet RX    |  |   Ethernet TX      |                 |                                     |
      +-------------------+  +--------------------+                 |                                     |
                 ^                    |                             |                                     |
                 |                    v                             |                                     |
      +-------------------------------------------+                |                                     |
      |              <<QEMU.SLiRP>>               |                |                                     |
      |  DHCP Server + Eth<=>TCP/UDP Translation  |                |                                     |
      |  TCP/UDP Socket-based Host Interaction    |                |                                     |
      +-------------------------------------------+                |                                     |
                            ^                                   |                                     |
                            |                                   v                                     v     |
   +--------------------------------------------------------------------------------------------------------+ |
   |                                                 <<HOST>>                                                    | |
   |                                      TCP/UDP Network Socket Stack                                         | |
   +--------------------------------------------------------------------------------------------------------+ |

  5. Set up SystemC 2.3.3

    master

    The co-simulation requires SystemC. Follow these steps to download, configure, and install it into your ~/cosim directory:

    1. Create the base directory: mkdir ~/cosim.
    2. Download and extract SystemC 2.3.3.
    3. Configure with CXXFLAGS=-std=c++11 and set the prefix to ${HOME}/cosim/.
    4. Build and install.
    # Build and install the binaries
cd ~/cosim
wget https://www.accellera.org/images/downloads/standards/systemc/systemc-2.3.3.tar.gz
tar xf systemc-2.3.3.tar.gz
cd systemc-2.3.3/
CXXFLAGS=-std=c++11 ./configure --prefix=${HOME}/cosim/
make
make install

  6. Build the DCMAC + AXIDMA SystemC Model

    master

    To run the CoSim demo, you must build a SystemC subsystem model for DCMAC + AXIDMA. This model communicates with a virtual VPK120 board via the AMD/Xilinx Remote-Port protocol.

    Follow these steps:

    1. Clone the repository containing this documentation into your desired directory (/PATH/TO/COSIM_DEMO).
    2. Refer to the top-level README.md in the repository to install SystemC, configure your build environment, and understand the specific build process.
    3. Execute the build for the Versal DCMAC Demo.
    4. Locate the resulting binary at: /PATH/TO/COSIM_DEMO/versal-dcmac-demo/versal_dcmac_demo.
    # Note: The actual build command is located in the top-level README.md
# The resulting binary will be at:
/PATH/TO/COSIM_DEMO/versal-dcmac-demo/versal_dcmac_demo

  7. Build Xilinx QEMU with Remote-Port support

    master

    To perform co-simulation, you must use a version of Xilinx QEMU that includes Remote-Port modules. Clone the Xilinx QEMU repository and configure it for the required architectures (aarch64, microblazeel, and riscv64).

    git clone https://github.com/Xilinx/qemu.git
mkdir build-qemu
cd build-qemu
../qemu/configure --target-list=aarch64-softmmu,microblazeel-softmmu,riscv64-softmmu
make

  8. Generate pl.dtsi using Vivado and xsct sdtgen

    master

    To generate the pl.dtsi device-tree file from the DCMAC example design, follow these steps:

    1. Export Hardware from Vivado:

      • Open the design in Vivado (Note: This design requires Vivado 2023.2; newer versions may lack the transceiver block design).
      • Navigate to File | Export | Export Hardware....
      • Select Include device image.
      • Complete the wizard to generate an .xsa file.

    2. Generate Device Tree via xsct:

      • Open a terminal and run xsct.
      • At the xsct% prompt, run the following commands (replacing paths as necessary):
        sdtgen set_dt_param -dir /PATH/TO/DT_OUTDIR -xsa /PATH/TO/EXPORTED/*.xsa
sdtgen generate_sdt
exit
    3. **Retrieve File**:
   - Locate and copy `pl.dtsi` from the specified `/PATH/TO/DT_OUTDIR/` directory.

    sdtgen set_dt_param -dir /PATH/TO/DT_OUTDIR -xsa /PATH/TO/EXPORTED/*.xsa
sdtgen generate_sdt

  9. Run the LMAC Co-Simulation

    master

    Co-simulation requires two concurrent processes: QEMU (the hardware/software emulator) and the SystemC simulator (the hardware model).

    Terminal 1: QEMU Run QEMU using the virt-cosim machine and point it to the modified virt.dtb. You must also configure a remote-port-net device to connect to the co-sim engine.

    Terminal 2: SystemC Run the compiled LMAC demo binary, pointing it to the socket created by QEMU in /tmp/machine-riscv64/.

    Verification: When Linux boots, check the kernel logs. You should see a message indicating the discovery of the LMAC device: lmac2 28030000.lmac2: Lewiz at 0x28030000 mapped to 0xFFFFFFFF44020000, tx-irq=9 rx-irq=11

    # Terminal 1: QEMU
mkdir /tmp/machine-riscv64/
qemu-system-riscv64 -M virt-cosim -smp 4 -serial stdio -display none -m 2G \
	-dtb virt.dtb -bios opensbi-riscv64-virt-fw_jump.bin -kernel Image \
	-append "root=/dev/vda ro console=ttyS0" \
	-drive file=rootfs.ext2,format=raw,id=hd0 \
	-device virtio-blk-device,drive=hd0 \
	-object rng-random,filename=/dev/urandom,id=rng0 \
	-device virtio-rng-device,rng=rng0 \
	-device virtio-net-device,netdev=usernet -netdev user,id=usernet \
	-netdev user,id=net4 \
	-device remote-port-net,rp-adaptor0=/machine/cosim,rp-chan0=256,rp-chan1=266,netdev=net4 \
	-machine-path /tmp/machine-riscv64

# Terminal 2: SystemC
riscv_virt_lmac2_demo unix:/tmp/machine-riscv64/qemu-rport-_machine_cosim 10000

  10. Launch the cpm5-qdma-demo or cpm4-qdma-demo

    master

    After the cloud image is running, launch the SystemC demo in a second terminal. You must provide the path to the SystemC library via LD_LIBRARY_PATH and specify the Unix socket path used by QEMU for the remote port connection.

    Replace /path/to/ with your SystemC installation path.

    # For cpm5-qdma-demo
$ cd systemctlm-cosim-demo
$ LD_LIBRARY_PATH=/path/to/systemc-2.3.3/lib-linux64/ ./pcie/versal/cpm5-qdma-demo unix:/tmp/machine-x86/qemu-rport-_machine_peripheral_rp0_rp 10000

# For cpm4-qdma-demo
$ cd systemctlm-cosim-demo
$ LD_LIBRARY_PATH=/path/to/systemc-2.3.3/lib-linux64/ ./pcie/versal/cpm4-qdma-demo unix:/tmp/machine-x86/qemu-rport-_machine_peripheral_rp0_rp 10000

  11. Set up QEMU for Co-Simulation

    master

    To emulate the Xilinx Zynq-7000 SoC, build a specific version of QEMU. This process involves cloning the repository, checking out a specific commit (74d70f8008), and configuring it with a target list that includes various architectures while disabling KVM and Xen.

    cd ~/cosim
git clone https://github.com/Xilinx/qemu.git
cd qemu
git checkout 74d70f8008

# Configure and build
./configure --target-list="arm-softmmu,aarch64-softmmu,microblazeel-softmmu,riscv32-softmmu,riscv64-softmmu,x86_64-softmmu" --enable-fdt --disable-kvm --disable-xen
make -j$((`nproc`+1))
make install DESTDIR=${HOME}/cosim/

  12. Test the Virtual VPK120 Board Readiness

    master

    Before modifying the workspace, verify that the environment is correctly prepared by performing a test run of the virtual VPK120 board using runqemu.

    1. Activate the AMD-EDF SDK.
    2. Navigate to your workspace directory.
    3. Run runqemu with the .qemuboot.conf file.
    4. Log in using the credentials amd-edf.

    To exit the simulation:

    1. Inside the Linux shell, run: sudo systemctl poweroff.
    2. Wait for the reboot: Power down message.
    3. Use the QEMU escape sequence: Ctrl+A then x.
    source /PATH/TO/EDF_SDK/environment-setup-cortexa72-cortexa53-amd-linux

cd /PATH/TO/VPK120_DCMAC
runqemu BOOT-versal-vpk120-sdt-seg.qemuboot.conf nographic slirp snapshot

# Once at the prompt:
# Login: amd-edf
# To shutdown:
# sudo systemctl poweroff