DOCSEARCHbyAlgolia
Submit Docs

AlpaSim Documentation

repository·main·Indexed 22 days ago

https://github.com/nvlabs/alpasim

A modular, lightweight, and data-driven research simulator for autonomous driving designed for testing end-to-end AV policies in closed-loop environments. AlpaSim features a microservices architecture with a pluggable renderer supporting NuRec and OmniDreams, and integrates driving policies such as Alpamayo-R1, Alpamayo 1.5, VaVAM, and Transfuser. It includes tools for controller benchmarking, scene metadata management via CSV, and support for local Docker or SLURM cluster deployments.

Tokens
113.2K
Snippets
318
Records
442
Agent score
78%

What's inside AlpaSim

  1. Overview of AlpaSim

    main

    AlpaSim is an open-source, modular, and data-driven autonomous vehicle (AV) simulation platform designed for research and development. It provides a closed-loop testing environment for end-to-end AV policies by simulating realistic sensor data, vehicle dynamics, and traffic scenarios.

    Key Capabilities

    • Algorithm Validation: Test new autonomous driving algorithms in realistic environments.
    • Safety Analysis: Evaluate vehicle behavior in edge cases and challenging scenarios.
    • Performance Benchmarking: Compare different models and configurations for regression testing.
    • Debugging: Analyze and understand complex autonomous driving behaviors.

    Core Design Principles

    • Sensor Fidelity: Features a pluggable renderer service. It supports NuRec by default and can use OmniDreams video-model rendering via FlashDreams. It provides high-fidelity camera feeds with configurable FOV, resolution, and frame rates, along with realistic sensor noise.
    • Research Hackability: Built with a Python-based implementation and a modular gRPC interface, allowing researchers to swap components with custom implementations.
    • Horizontal Scalability: Uses a microservices architecture to enable distributed computing and multi-node deployments.

  2. Overview of the Evaluation module

    main

    The Evaluation module is a refactored version of the KPI service designed to process autonomous driving simulation results. It performs the following tasks:

    • Reads ASL logs: Parses simulation logs into structured data.
    • Computes metrics: Uses various "Scorers" to calculate performance indicators.
    • Saves results: Outputs computed metrics locally as Parquet files.
    • Generates videos: Produces visual representations of the simulation, including per-timestamp metric results, to help verify if results look reasonable.

    Note: This codebase is new and may contain corner cases in metric calculations.

  3. Understand AlpaSim performance telemetry

    main

    AlpaSim includes built-in Prometheus telemetry enabled by default. When a simulation starts, the AlpaSim wizard automatically:

    • Allocates necessary ports.
    • Starts a Prometheus support service.
    • Starts /metrics endpoints on runtime workers.
    • Writes scrape configuration into the run directory.

    At the end of a simulation, the runtime automatically queries the Prometheus server to generate a metrics_plot.png file. Prometheus data is persisted in the prometheus/data directory of the run.

  4. Use Replay Services to replay recorded simulation data

    main

    Replay Services in Alpasim allow for replaying recorded simulation data, primarily used for verifying runtime integration. A common use case is running tests like test_runtime_integration_replay.py to ensure the runtime sends the same requests as recorded in the logs, which helps prevent regressions during refactoring.

    Core Components

    • ASLReplay: Replays ASL (Alpasim Scene Language) files that contain actor trajectories and scene data.
    • ReplayServiceBase: The base class used for implementing custom replay services.

  5. Runtime Lifecycle and Synchronization

    main

    The Alpasim runtime follows a specific sequence for starting simulation components:

    1. Egomotion: Starts from the rig egomotion start.
    2. Rendering: Anchors rendering at the first configured USDZ camera frame.
    3. Policy: Starts the policy after the duration specified by force_gt_duration_us.

  6. Understand the AlpaSim Architecture

    main

    AlpaSim is a modular system composed of networked microservices that communicate to close the loop for autonomous driving simulation.

    Core Microservices:

    • Renderer: Provides observed video frames (defaults to NuRec; can be backed by OmniDreams via FlashDreams).
    • Physics Simulation: Constrains actors to the road surface.
    • Controller: Provides egomotion history.
    • Driver: Receives information and provides driving decisions.
    • Runtime: Orchestrates the loop by requesting frames from the renderer and egomotion from the controller, and communicating with physics and the driver.
    • Traffic Simulation: Manages traffic actors.

    This architecture allows for high research hackability and horizontal scalability.

  7. Understand Alpasim scene and test suite metadata files

    main

    Alpasim uses CSV files to manage scene artifact metadata and mappings between test suites and specific artifacts.

    Key files include:

    • sim_scenes.csv: Contains public scene artifact metadata for current releases (e.g., 26.01 and 26.04).
    • sim_suites.csv: Maps public test suites to specific scene artifacts.
    • sim_scenes_2505.csv / sim_suites_2505.csv: Legacy metadata for the 25.07 release (not loaded by default).

    Selection Logic:

    • Suite Mapping: Each row in a suite CSV provides a readable scene ID and the specific UUID of the artifact.
    • Scene Selection: Using the configuration scenes.scene_ids=[...] selects the most recently modified artifact for each provided scene ID.

  8. Understand the Alpasim Controller architecture and execution

    main

    The controller models vehicle dynamics and control using a Model Predictive Control (MPC) approach.

    Workflow

    1. A SystemManager forwards trajectory reference commands and vehicle state information to a System (comprising vehicle dynamics + controller).
    2. The System uses an MPC to compute commanded steering and acceleration.
    3. The vehicle model propagates the dynamics to the requested time and returns the new state.

    Step Execution Lifecycle

    For every time step, the system performs the following:

    1. Overrides the current vehicle state (local to rig transformation and optionally velocities).
    2. Drops a new reference frame coincident/aligned with the vehicle.
    3. Resets the MPC initial state based on the current vehicle state.
    4. Transforms the reference trajectory to the rig frame.
    5. Runs the MPC controller to compute commanded steering and acceleration.
    6. Propagates the vehicle model to the requested time.
    7. Applies the relative motion to the local to rig transformation.

  9. Supported Driving Policies in AlpaSim

    main

    AlpaSim supports several pre-integrated driving policies for testing:

    • Alpamayo-R1: NVIDIA Alpamayo, a VLA (Vision-Language-Action) driving policy featuring chain-of-causation reasoning.
    • Alpamayo 1.5: Support for public Alpamayo 1.5 checkpoints.
    • VaVAM: An autoregressive video-action driving policy.
    • Transfuser: Latent TransFuser v6 (LTFv6) policy, developed for NAVSIM (provisional support).

  10. Choose between Linear and Nonlinear MPC implementations

    main

    Alpasim provides two MPC implementations. Both use the same cost function and constraints but differ in formulation and solver, creating a trade-off between speed and accuracy.

    ImplementationSolverCharacteristics
    Nonlinear MPCIPOPT (via do_mpc)More accurate for aggressive maneuvers or tight turns. Uses the full nonlinear dynamic model.
    Linear MPCOSQPFaster. Casts the problem as a reduced form quadratic program (QP) by linearizing dynamics about the 'free' trajectory.

    Selection Methods

    • Wizard Controller Config Group: Use controller=nonlinear to select the implementation.
    • Standalone Server: Pass a YAML configuration via the --config flag.

  11. Understand the Alpasim microservice architecture

    main

    Alpasim is designed as a collection of microservices that communicate via gRPC. This architecture prioritizes horizontal scalability and research hackability over real-time physics precision.

    Core Components

    • Runtime: The central orchestrator. It maintains the world state, drives the simulation loop by issuing calls to other services, and produces logs. It can run in one-shot execution (orchestrating a batch of rollouts) or daemon mode (exposing a gRPC server for on-demand requests).
    • Renderer: Produces camera frames for the ego vehicle. It can be backed by the Neural Rendering Engine (NRE) or the OmniDreams video-model renderer.
    • Driver: The ego-vehicle policy network (the main target of simulation) that receives sensor readings and produces actuation requests (planned paths).
    • Controller/Vehicle Model: Models vehicle dynamics and controller behavior, providing uncorrected egomotion based on actuation requests.
    • Physics: Applies ground constraints to ensure both ego and non-ego actors stay on the ground.
    • Trafficsim: Actuates non-ego actors (pedestrians, vehicles, etc.) based on world state bounding boxes.
    • Eval: An external module that consumes simulation logs to compute autonomous driving metrics.

    Scalability Model

    Services can be replicated based on computational demand. A typical priority for replication is: ego policy > renderer > controller sim > traffic sim > physics sim

  12. Understand Alpasim Log (asl) file formats

    main

    Simulation outputs in Alpasim are stored as .asl files (AlpaSim Log). These files are size-delimited protobuf streams using a custom schema. Each rollout generates a unique .asl file containing three primary types of messages:

    • RolloutMetadata: A header containing metadata for reproducibility and bookkeeping.
    • ActorPoses: Messages describing the location of all actors (including the 'EGO' vehicle) in global coordinate space.
    • *_request and *_return: Microservice requests and responses. These allow for replaying the behavior of specific services without running the full simulator.

    Note:

    • RolloutCameraImage requests can be used to assemble .mp4 videos from an .asl log.
    • The simulation header does not specify the usdz file UUID.