Hypervisor in 1,000 Lines

repository·main·Indexed 43 days ago

https://github.com/nuta/hypervisor-in-1000-lines

An educational reference implementation of a RISC-V hypervisor written in Rust. The project demonstrates hypervisor construction through a concise implementation, featuring a custom BumpAllocator, guest page table management (Sv48x4), a Linux kernel loader, and VCpu state management for a no_std environment.

Tokens
10.7K
Snippets
41
Records
49
Agent score
55%

What's inside Hypervisor in 1,000 Lines

  1. SBI Call Conventions for Hypercalls

    main

    When implementing hypercall handlers, follow the SBI specification conventions:

    • Control Transfer: Use ECALL to transfer control from the supervisor to the SEE (Supervisor Execution Environment/Hypervisor).
    • Extension ID (EID): Encoded in register a7.
    • Function ID (FID): Encoded in register a6 (for SBI v0.2+).
    • Register Preservation: The callee must preserve all registers except a0 and a1.
    • Return Values: Functions must return a pair of values in a0 and a1, where a0 contains the error code.
  2. Understand RISC-V Guest Modes (VU-mode and VS-mode)

    main

    The RISC-V hypervisor extension introduces virtualization modes that allow the guest operating system to run with its own user and kernel modes, separate from the host's modes.

    • VU-mode (Virtual User mode): The guest's user mode.
    • VS-mode (Virtual Supervisor mode): The guest's kernel mode.
    • U-mode (User mode): The host's user mode.
    • HS-mode (Hypervisor-extended Supervisor mode): The host's kernel mode.

    In a hardware-assisted hypervisor, the CPU handles transitions between VU-mode and VS-mode (like guest system calls) automatically. The hypervisor acts as an event handler, primarily intervening during VM exits (e.g., when the guest performs memory-mapped I/O) by jumping to a trap handler.

  3. How hypercalls work in the hypervisor

    main

    Hypercalls are the mechanism used by a guest OS to request services from the hypervisor, similar to how system calls work for an OS. The flow is as follows:

    1. The guest program executes the ecall instruction with parameters passed in registers.
    2. The CPU switches to HS-mode (Hypervisor Supervisor mode) and jumps to the trap handler defined in the stvec CSR.
    3. The trap handler saves the guest state and determines the cause of the trap by reading the scause CSR (e.g., an ecall from the guest).
    4. After the hypervisor handles the request, the trap handler restores the guest state and returns control to the guest.
  4. Understand Guest-physical vs. Host-physical addresses

    main

    In hardware-assisted virtualization (like RISC-V), the CPU uses four address spaces to provide isolation between VMs. When the CPU is in guest mode, address translation occurs in two stages:

    1. Stage 1: Guest-virtual $\rightarrow$ guest-physical (controlled by the page table in satp).
    2. Stage 2: Guest-physical $\rightarrow$ host-physical (controlled by the guest page table in hgatp).

    This two-stage translation allows the hypervisor to map each guest's perceived physical memory to distinct regions of the actual host physical memory, enabling multiple VMs to run simultaneously without direct access to host memory.

  5. Implement a trap handler to debug CPU events

    main

    A trap handler is triggered when the CPU encounters events like invalid memory access or illegal instructions. To implement one:

    1. Read CSRs: Use a macro to read Control and Status Registers (CSRs) like scause (cause), sepc (program counter), and stval (trap-specific value).
    2. Map Causes: Match the scause value against known RISC-V trap causes (e.g., 2 for illegal instruction).
    3. Register the Handler: Write the handler's address to the stvec CSR.
    4. Linker Configuration: Ensure the handler is placed in a specific section (e.g., .text.stvec) using the #[unsafe(link_section = ".text.stvec")] attribute so the linker can align it correctly.

    Once implemented, you can use llvm-addr2line with the address from sepc to find the exact line of code that caused the trap.

    #[unsafe(link_section = ".text.stvec")]
    pub fn trap_handler() -> ! {
        let scause = read_csr!("scause");
        let sepc = read_csr!("sepc");
        let stval = read_csr!("stval");
        // ... match scause to string ...
        panic!("trap handler: {} at {:#x} (stval={:#x})", scause_str, sepc, stval);
    }
  6. Install QEMU for emulation

    main

    QEMU is required to emulate the environment where your hypervisor will run. Installation commands vary by operating system:

    • macOS: Use Homebrew.
    • Ubuntu: Install the RISC-V 64-bit system emulator.
    • Windows: Install Windows Subsystem for Linux (WSL2) and follow the Ubuntu instructions.
    ### macOS
    ```sh
    brew install qemu

    Ubuntu

    sudo apt install qemu-system-riscv64
  7. Build and run the hypervisor with QEMU

    main

    To build the hypervisor, use cargo build with specific RUSTFLAGS to pass the linker script and specify the rust-lld linker. The resulting ELF file is then executed via qemu-system-riscv64.

    Build Command Requirements:

    • RUSTFLAGS="-C link-arg=-Thypervisor.ld -C linker=rust-lld"
    • Target: riscv64gc-unknown-none-elf

    QEMU Execution Flags:

    • -machine virt and -cpu rv64
    • -bios default
    • -m 128M (Memory size)
    • -d cpu_reset,unimp,guest_errors,int -D qemu.log (Logging/Debugging)
    • -kernel hypervisor.elf
    #!/bin/sh
    set -ev
    
    RUSTFLAGS="-C link-arg=-Thypervisor.ld -C linker=rust-lld" \
      cargo build --bin hypervisor --target riscv64gc-unknown-none-elf
    
    cp target/riscv64gc-unknown-none-elf/debug/hypervisor hypervisor.elf
    
    qemu-system-riscv64 \
        -machine virt \
        -cpu rv64 \
        -bios default \
        -smp 1 \
        -m 128M \
        -nographic \
        -d cpu_reset,unimp,guest_errors,int -D qemu.log \
        -serial mon:stdio \
        --no-reboot \
        -kernel hypervisor.elf
  8. Use the `alloc` crate for dynamic collections

    main

    Once the global allocator is initialized, you can use dynamic memory features by adding extern crate alloc; to your entry point. Note that you do not need to add alloc to your Cargo.toml as it is bundled with the standard library.

    You must explicitly use objects from the alloc crate (e.g., alloc::vec::Vec).

    #![no_std]
    #![no_main]
    
    extern crate alloc;
    
    fn main() -> ! {
        // Ensure allocator is initialized first
        allocator::GLOBAL_ALLOCATOR.init(&raw mut __heap, &raw mut __heap_end);
    
        let mut v = alloc::vec::Vec::new();
        v.push('a');
        v.push('b');
        v.push('c');
        
        loop {}
    }
  9. Enter Guest Mode (HS-mode) using RISC-V CSRs

    main

    To transition the CPU from the host kernel into the guest kernel (HS-mode), you must configure the Hypervisor Status register (hstatus) and the Supervisor Exception Program Counter (sepc), then execute the sret instruction.

    Key components:

    • hstatus: A register containing flags that describe the CPU state for the transition.
    • sepc: The program counter address where the CPU will begin execution in guest mode.
    • sret: The trap-return instruction that performs the transition based on the values in hstatus and sepc.
    fn main() -> ! {
        /* ... */
    
        let mut hstatus: u64 = 0;
        hstatus |= 2 << 32; // VSXL: XLEN for VS-mode (64-bit)
        hstatus |= 1 << 7; // SPV: Supervisor Previous Virtualization mode
    
        let sepc: u64 = 0x1234abcd;
        unsafe {
            asm!(
                "csrw hstatus, {hstatus}",
                "csrw sepc, {sepc}",
                "sret",
                hstatus = in(reg) hstatus,
                sepc = in(reg) sepc,
            );
        }
    
        unreachable!();
    }