CKB VM

repository·develop·Indexed 19 days ago

https://github.com/nervosnetwork/ckb-vm

A pure software implementation of the RISC-V instruction set used as the scripting virtual machine for the Nervos CKB blockchain. It supports full IMCB instructions for 32-bit and 64-bit register sizes and features two execution modes: a Rust interpreter for development and an assembly-based interpreter (ASM mode) for production.

Tokens
11.2K
Snippets
37
Records
55
Agent score
64%

What's inside ckb-vm

  1. Understand CKB VM execution modes

    develop

    CKB VM supports two different execution modes. For production-grade consistency and to avoid potential bugs caused by implementation differences, you should only use ASM mode.

    • Rust interpreter mode: Primarily used for development assistance. It is not used in production and may exhibit inconsistent behavior compared to ASM mode.
    • Assembly based interpreter mode (ASM mode): The standard mode used for consistent behavior and production environments.
  2. Compiling custom RISC-V binaries for CKB VM

    develop

    While CKB VM includes binaries for testing, you may need a RISC-V compiler to create your own contracts. CKB VM uses standard RISC-V instructions and the ELF binary format. Any RISC-V compatible compiler should theoretically work.

    Recommended tools:

    • riscv-tools
    • A custom-compiled GCC with RISC-V support.
  3. Build CKB VM from source

    develop

    To build CKB VM, ensure you are using a stable Rust version on a 64-bit Linux, macOS, or Windows environment. The repository includes RISC-V binaries used for testing, so a RISC-V compiler is not strictly required for the initial build.

    To build the project using Cargo:

    1. Clone the repository.
    2. Navigate to the directory.
    3. Run cargo build.
    # download CKB VM
    $ git clone https://github.com/nervosnetwork/ckb-vm
    $ cd ckb-vm
    $ cargo build
  4. Understand CKB VM versions and compatibility

    develop

    The VM uses version constants to maintain backward compatibility and handle bug fixes in instruction execution or memory access:

    • VERSION0: Initial version used in CKB Lina mainnet.
    • VERSION1: Fixes memory access bugs (e.g., reading the last byte in memory).
    • VERSION2: Adds checks for stack overflow when handling large argv arrays.
    • VERSION3: Current/latest version.

    When implementing SupportMachine or using load_elf, the version parameter determines how memory is initialized and how the stack is laid out.

  5. Understand `ProgramMetadata` and `LoadingAction` structures

    develop

    When parsing an ELF file, the VM uses ProgramMetadata to determine how to map the file into memory.

    ProgramMetadata

    Represents the complete loading plan for an ELF file.

    • actions: A Vec<LoadingAction> describing each segment to be loaded.
    • entry: The entry point address (u64).

    LoadingAction

    Describes a specific memory segment's loading requirements.

    • addr: The aligned starting address in memory.
    • size: The total size of the segment (including padding).
    • flags: The RISC-V memory flags (e.g., FLAG_EXECUTABLE, FLAG_FREEZED).
    • source: The Range<u64> within the original ELF file bytes that constitutes this segment.
    • offset_from_addr: The offset used to adjust the file data to the aligned memory address.
  6. Use Snapshot2Context to manage VM snapshots

    develop

    The Snapshot2Context manages the relationship between a VM's memory and an external DataSource. It tracks which memory pages are backed by the DataSource versus which pages are "dirty" (modified and requiring full storage).

    Key Workflows:

    1. Loading a Program efficiently

    To reduce snapshot size, you should track memory pages belonging to the program using mark_program. This requires parsing the ELF first to get ProgramMetadata.

    // 1. Parse ELF to get metadata
    let metadata = elf::parse_elf(elf_data)?;
    // 2. Load program into machine
    machine.load_program_with_metadata(metadata.clone(), ...)?;
    // 3. Mark pages in context to track them as coming from the DataSource
    context.mark_program(&mut machine, &metadata, &program_id, program_offset)?;

    2. Creating a Snapshot

    Use make_snapshot to capture the current state of a SupportMachine. The resulting Snapshot2 contains:

    • pages_from_source: Pages that can be reconstructed from the DataSource.
    • dirty_pages: Pages that have been modified and must be stored as raw bytes.
    • Machine state: registers, pc, cycles, and max_cycles.

    3. Resuming from a Snapshot

    Use resume to restore a machine to a previously captured state. This clears the current context's page tracking and restores registers, PC, and memory (both from the source and dirty pages).

    // Example: Creating a snapshot
    let snapshot = context.make_snapshot(&mut machine)?;
    
    // Example: Resuming a machine
    context.resume(&mut machine, &snapshot)?;
  7. How Macro-Operation Fusion (MOP) works

    develop

    Macro-Operation Fusion (MOP) is an optimization where the decoder identifies a sequence of adjacent instructions that can be treated as a single, more complex instruction. This reduces the overhead of the execution loop.

    When calling decode_mop, the decoder:

    1. Decodes the instruction at the current pc using decode_raw.
    2. Checks the opcode of the decoded instruction against known fusion rules (e.g., OP_ADD, OP_SUB, OP_LUI, OP_AUIPC).
    3. If a pattern matches (e.g., an ADD followed by an SLTU and another ADD that forms a specific arithmetic chain), it returns a single fused Instruction with a combined length.
    4. If no pattern matches, it falls back to returning the single instruction decoded by decode_raw.

    Common fusion patterns include:

    • Arithmetic chains: Combining ADD, SLTU, and ADD into OP_ADD3A, OP_ADD3B, or OP_ADD3C.
    • Jump optimizations: Combining LUI + JALR into OP_FAR_JUMP_ABS or AUIPC + JALR into OP_FAR_JUMP_REL.
    • Immediate loading: Combining LUI + ADDIW into OP_CUSTOM_LOAD_IMM.
  8. Use SparseMemory for efficient memory allocation

    develop

    The SparseMemory<R> struct implements a sparse flat memory model where pages are only allocated when they are actually accessed. This is useful for simulating large address spaces without consuming the full physical memory upfront.

    Key characteristics:

    • Lazy Allocation: Pages are added to the internal storage only when requested via a load or store operation.
    • No Permission Checking: Unlike some memory models, this implementation focuses on storage efficiency and does not perform permission checks.
    • Memory Size: The total addressable memory size is defined at initialization and must be a multiple of RISCV_PAGESIZE.
    • Generic Register Type: It is generic over a type R that implements the Register trait.
    // Example initialization (conceptual)
    let memory = SparseMemory::<MyRegister>::new(128 * 1024 * 1024); // 128MB
  9. Use FlatMemory for simple RISC-V memory access

    develop

    The FlatMemory<R> struct provides a contiguous chunk of memory for a RISC-V machine. It is designed for simplicity and lacks permission checking logic. It implements the Memory trait, allowing for standard load/store operations.

    Key characteristics:

    • Memory Layout: A single Vec<u8> representing the entire address space.
    • Endianness: Follows the RISC-V ISA standard of little-endian memory.
    • Page Management: Uses RISCV_PAGESIZE to manage flags and dirty status for pages.
    • Constraints: The memory_size must be a multiple of RISCV_PAGESIZE.

    Note that FlatMemory implements Deref and DerefMut to Vec<u8>, allowing you to access the underlying raw byte buffer directly.

    // Example conceptual usage
    let mut memory = FlatMemory::<MyRegisterType>::new(RISCV_PAGESIZE * 1024);
    
    // Storing data
    memory.store_bytes(0x1000, &[0xDE, 0xAD, 0xBE, 0xEF]).unwrap();
    
    // Loading data
    let val = memory.load32(&MyRegisterType::from_u64(0x1000)).unwrap();
  10. How trace decoding and direct threading works

    develop

    CKB VM uses traces to accelerate execution. Traces consist of instructions paired with 'thread' addresses (absolute addresses used for direct threading).

    • Fixed Traces: Pre-decoded sequences of instructions stored in a fixed-size array.
    • Dynamic Traces: For longer sequential blocks, the VM generates DynamicTrace objects. These have a similar memory layout to FixedTrace (address, length, cycles, followed by threads) but allow for a variable number of instructions.
    • Direct Threading: The label_from_fastpath_opcode function converts an opcode into an absolute memory address of the corresponding execution logic, allowing the VM to jump directly to the next instruction's handler.
  11. Understand instruction fusion in the decoder

    develop

    The CKB VM decoder implements instruction fusion, where specific sequences of instructions are combined into a single, more efficient internal instruction (a "fuze" instruction). This is an optimization performed during the decoding stage.

    Common fusion patterns include:

    • Immediate Loading: Combining multiple instructions to load a large immediate value into a register using OP_CUSTOM_LOAD_IMM.
    • Wide Arithmetic: Combining multiplication and division/remainder operations into single wide operations. For example:
      • OP_MULH + OP_MUL $\rightarrow$ OP_WIDE_MUL (R4type)
      • OP_MULHU + OP_MUL $\rightarrow$ OP_WIDE_MULU (R4type)
      • OP_MULHSU + OP_MUL $\rightarrow$ OP_WIDE_MULSU (R4type)
      • OP_DIV + OP_REM $\rightarrow$ OP_WIDE_DIV (R4type)
      • OP_DIVU + OP_REMU $\rightarrow$ OP_WIDE_DIVU (R4type)

    Fusion occurs only when specific register constraints are met (e.g., ensuring the destination register does not overlap with source registers in a way that violates the fusion logic) and the subsequent instruction is the expected operand for the fusion.

  12. Configure and build a DefaultMachine using AbstractDefaultMachineBuilder

    develop

    To create a functional VM instance, use the AbstractDefaultMachineBuilder (or RustDefaultMachineBuilder for Rust-based VMs). The builder allows you to inject custom syscalls, a debugger, a pause mechanism, and a custom instruction cycle function to control how many cycles each instruction consumes.

    Key builder methods:

    • instruction_cycle_func(Box<dyn Fn(Instruction) -> u64 + Send + Sync>): Sets a function to determine the cycle cost of an instruction.
    • syscall(Box<dyn Syscalls<Inner>>): Adds a custom syscall implementation.
    • debugger(Box<dyn Debugger<Inner>>): Attaches a debugger.
    • pause(Pause): Sets the interruption mechanism.
    • build(): Finalizes the construction of the DefaultMachine.
    // Example pattern for building a machine
    let machine = AbstractDefaultMachineBuilder::new(inner_machine)
        .instruction_cycle_func(Box::new(|_| 1))
        .syscall(Box::new(my_custom_syscall))
        .debugger(Box::new(my_debugger))
        .build();