demoparser2 Documentation

repository·main·Indexed 18 days ago

https://github.com/laihoe/demoparser

A high-performance tool for analyzing Counter-Strike 2 replay files (demos) featuring a Rust core with Python, JavaScript, and WASM interfaces. It utilizes a query-based approach to extract specific game events, player properties per tick, grenade data, and demo header metadata. Available as a Python package via pip and as NodeJS packages including architecture-specific binaries for macOS, Linux, and FreeBSD.

Tokens
14.4K
Snippets
46
Records
63
Agent score
72%

What's inside demoparser2

  1. Understand the DEM_Packet structure

    main

    The DEM_Packet is the most common command type in a demo file. It is decoded by reading a bitstream where each message consists of a message type and a size.

    Main message types within a DEM_Packet include:

    • svc_PacketEntities: Contains the majority of entity data (coordinates, health, etc.). Note that these only send changes; if a value hasn't changed, it won't be updated in the current tick.
    • GE_Source1LegacyGameEventList: Provides the decoding schema for game events. This must be processed before any game events can be decoded.
    • GE_Source1LegacyGameEvent: Key-value pair events triggered by game actions (e.g., weapon fire).
    • svc_CreateStringTable / svc_UpdateStringTable: Contains baseline/default values for entities (e.g., default player health).
  2. Understand the two-pass parsing architecture

    main

    The demoparser uses a two-pass system to process Counter-Strike 2 demo files efficiently:

    1. 1st Pass (parser/src/parser.rs): A fast pass that identifies packet boundaries (start and end points) to prepare for the second pass. It also handles one-off data such as descriptors for entity data and game events.
    2. 2nd Pass (parser/src/parser_thread.rs): A multi-threaded pass that performs the heavy lifting by parsing the majority of the demo data.

    Understanding this separation is key to understanding how the parser achieves high performance through parallelization.

  3. How demoparser2 works: Query-based approach

    main
    Unlike traditional parsers that use a streaming event-hook model, demoparser2 uses a query-based approach. Instead of listening to a stream of events as they happen, you request specific data (like certain events or specific field values at certain ticks) from the demo, similar to how you would interact with a database. This allows for more efficient and targeted data extraction.
  4. Install the @laihoe/demoparser2-darwin-x64 binary

    main

    Use the @laihoe/demoparser2-darwin-x64 package if you are running on an Intel-based macOS system (x86_64-apple-darwin). This package provides the specific binary required for the @laihoe/demoparser2 library to function on this architecture.

    npm install @laihoe/demoparser2-darwin-x64
  5. Explore the core entity decoding logic

    main

    If you are exploring the Rust source code to understand how data is extracted, the most critical logic resides in:

    • parser/src/entities.rs: Defines entity structures.
    • parser/src/collect_data.rs: Handles data collection.

    Specifically, all newly decoded values are processed through the decode_entity_update function. This is the primary entry point for understanding how raw demo data is transformed into usable entity updates.

  6. Install @laihoe/demoparser2 for Linux ARM64 (musl)

    main

    This package provides the aarch64-unknown-linux-musl binary for the @laihoe/demoparser2 library. Use this specific package if your target environment is a Linux system running on ARM64 architecture using the musl C library (common in lightweight distributions like Alpine Linux).

    npm install @laihoe/demoparser2-linux-arm64-musl
  7. Reproduce the parsing performance baseline

    main

    To benchmark the parser's performance and reproduce the baseline metrics, use the parse_bench binary. The benchmark performs a representative full parse (including player properties, all events, and ticks) using a release build with Link Time Optimization (LTO) enabled.

    Run the benchmark using the following command structure:

    # Build the benchmark binary in release mode
    cargo build --release --bin parse_bench
    
    # Run the benchmark
    # Usage: ./target/release/parse_bench <demo_path> <iterations> <mode>
    ./target/release/parse_bench "<path_to_demo>" 5 both

    Arguments:

    • <demo>: Path to the .dem file.
    • 5: Number of timed iterations.
    • both: Run both single-threaded (st) and multi-threaded (mt) modes.
    cargo build --release --bin parse_bench
    ./target/release/parse_bench "<nuke.dem>" 5 both
  8. Install demoparser2 for Python, NodeJS, or WASM

    main

    You can install the demoparser2 library depending on your target environment:

    • Python (requires Python >= 3.8): Use pip.
    • NodeJS: Use npm to install the standard package.
    • WASM: Use npm to install the WASM-specific package.
    # Python
    pip install demoparser2
    
    # NodeJS
    npm i @laihoe/demoparser2
    
    # WASM
    npm i demoparser2
  9. Install demoparser2 via maturin

    main

    To install demoparser2 into your current Python environment, use maturin. It is recommended to perform this installation within a virtual environment (venv) or a conda environment to avoid conflicts with system packages.

    Run the following command from the root of the repository:

    maturin develop --release