Project AirSim

repository·main·Indexed 20 days ago

https://github.com/iamaisim/projectairsim

A high-fidelity simulation platform for autonomous systems (drones, robots) built on Unreal Engine 5. It includes a C++ client library for interacting with the simulation server, featuring APIs for drone control (arm, takeoff, land), scene management via the World object, and asynchronous command handling. The platform consists of simulation libraries, an Unreal Engine plugin, and a client library, supporting Windows 10/Server 2019 and Ubuntu 20.04.

Tokens
96.3K
Snippets
250
Records
402
Agent score
72%

What's inside projectairsim

  1. Supported data aggregation formats

    main

    Once data is generated in Project AirSim, it can be aggregated into several specific formats. These formats are designed for ingestion into data curation tools or for fine-tuning pre-trained models. The supported formats are:

    • COCO JSON: Specifically supports segmentation data (only available when segmentation data collection is enabled).
    • JSON: A custom structured format containing metadata, image details, and bounding box/polygon data.
    • JSONL: JSON Lines format, typically used for streaming or line-delimited data processing.
    • CSV: A tabular format containing image names, pose data, categories, and coordinate points.
  2. Use the action controller to translate MoveIt trajectories to cmd_vel

    main

    The action_controller is a simple controller designed to bridge MoveIt and the Hector Quadrotor controller. It translates trajectories produced by the MoveIt plugin into cmd_vel commands compatible with the Hector Quadrotor controller.

    It interfaces with the MoveIt plugin using a custom action type: MultiDofFollowJointTrajectory.action.

  3. What is the LVMon debugging facility?

    main

    The Live Value Monitoring (LVMon) facility is a low-overhead, asynchronous publish/subscribe messaging system designed for inspecting rapidly changing values during Project AirSim simulation.

    Unlike standard breakpoints (which can disrupt simulation timing) or print statements (which can clutter output), LVMon allows you to monitor values with minimal impact on performance.

    Key Characteristics:

    • Unidirectional: The LVMon server (running inside Project AirSim) publishes values; clients (subscribers) only receive them.
    • Lightweight: Publishing requires only a single function call with no complex setup.
    • Data Types: Supports only Signed 64-bit integer, Unsigned 64-bit integer, 64-bit floating-point, and UTF-1 string.
    • Platform Support: The LVMon server runs on Windows and Linux. The LVMonitor viewing application runs on Windows only.
  4. Understand the Project AirSim architecture layers

    main

    Project AirSim development is organized into three distinct layers. Depending on your needs, you may only need to interact with the Client Library, or you may need to build the entire framework from source to extend features.

    1. Project AirSim Sim Libs: The core C++ simulation libraries located at the projectairsim/ root. This is a CMake-based project.
    2. Project AirSim Plugin: The Unreal Engine integration. This is a script-generated project located in projectairsim/unreal/Blocks that requires Unreal Engine to build.
    3. Project AirSim Client Library: The interface used to drive simulations, located in projectairsim/client/python (Python) or projectairsim/client/cpp (C++).

    Folder Structure Overview

    • client/python/projectairsim: Python client library.
    • client/python/sim-config: Configuration JSONC files.
    • multirotor_api: Multirotor controller components.
    • physics: Physics engine components.
    • core_sim: Core simulation logic.
    • unreal/Blocks/Plugins/ProjectAirSim: The Unreal Engine plugin.
    • unreal/Blocks/Plugins/Drone: Drone-specific content plugin.
  5. Understanding script mirroring conventions

    main

    Project AirSim uses a 'mirror' relationship for its Python scripts. A script in this directory is considered a mirror if it performs the same function as a corresponding script in the AirSim OSS PythonClient folder.

    To ensure maintainability and ease of use, the following rules are enforced for mirror scripts:

    1. Relative Location: The script must reside in the same relative directory structure as its mirror in the PythonClient folder.
    2. Naming: The filename of the Project AirSim script must be identical to its mirror script.
  6. Compare SITL and HITL configurations for PX4

    main

    When using the PX4 Autopilot with Project AirSim, you can choose between Software-In-The-Loop (SITL) or Hardware-In-The-Loop (HITL).

    FeatureSITL (Recommended)HITL
    PX4 DeviceNoRequired
    Manual FlightVia game controller or RC transmitter over USBOptional via RC transmitter & receiver
    AirframesAll Project AirSim airframes"HIL Quadcopter X" only
    Setup & RunHarderEasier
    Modifying/Debugging PX4 SoftwareEasierHarder
  7. Understand the ProjectAirSim Plugin and Blocks Project structure

    main

    The simulation environment is built on the Blocks Project (the main UE project) and the ProjectAirSim Plugin.

    Blocks Project (unreal/Blocks/)

    • Blocks.uproject: Project manifest.
    • BlocksMap.umap: Main simulation level.
    • GISMap.umap: Geographic information system level.

    ProjectAirSim Plugin (unreal/Blocks/Plugins/ProjectAirSim/)

    • ProjectAirSim.uplugin: Plugin manifest.
    • Source/: C++ API headers (Public) and Implementation (Private).
    • SimLibs/: Compiled C++ DLLs loaded at runtime.
    • Content/: Unreal Engine assets.
    • Binaries/: Platform-specific binaries.
  8. Core Components of Project AirSim

    main

    The system is divided into three primary layers:

    1. Unreal Engine 5 Foundation

    Located in unreal/Blocks/, this is the primary UE5 project. It is configured for UE versions 5.2 and 5.7. It manages content assets, Blueprints, and simulation levels.

    2. ProjectAirSim Plugin (Bridge Layer)

    Located in unreal/Blocks/Plugins/ProjectAirSim/, this plugin integrates Sim Libs with Unreal's runtime. Key Classes:

    • AProjectAirSimGameMode: Manages the simulation lifecycle.
    • AUnrealSimLoader: Loads and manages Sim Libs.
    • AUnrealRobot: A UE Actor representing a robot.
    • AUnrealSensor: The base class for all sensors.

    3. Sim Libs (Core Simulation Engine)

    Located in the root projectairsim/ directory as a CMake project. It contains the following modules:

    • core_sim/: Simulation loop and scene management.
    • vehicle_apis/: Robot controllers (SimpleFlight, ArduPilot, PX4).
    • physics/: Physics engines (FastPhysics, Matlab).
    • mavlinkcom/: MAVLink protocol implementation.
    • sensors/: Sensor simulation.
    • simserver/: Network server and API management.
  9. Configure simulation clock types

    main

    The clock object in the scene configuration determines how simulation time advances. There are four supported types:

    1. Steppable Clock (Fixed-step)

    Best for deterministic, repeatable simulations. It advances by step-ns (nanoseconds) every real-time-update-rate (nanoseconds).

    • Deterministic scaling: To run at 2x real-time, set step-ns to twice the real-time-update-rate.
    • pause-on-start: If true, the simulation starts paused, allowing manual control via APIs.

    2. Real-time Clock (Variable-step)

    A variable-step clock where the simulation time step equals the real-time interval between execution periods.

    • Warning: This is not deterministic or repeatable and cannot be paused.

    3. Engine-driven Clock (Fixed-step)

    A fixed-step clock driven by a host loop outside the core simulation executor.

    • Use case: When Project AirSim is embedded in another runtime (e.g., another game engine).
    • In UnrealNative scenes, this is known as the unreal-driven-clock.
    • Note: Does not use real-time-update-rate and does not support pause/resume via SimClock APIs.

    4. External-clock

    Currently behaves identically to engine-driven. It is intended as a future entry point for driving simulations from external applications while maintaining the same runtime behavior.

    // Example: Steppable clock
    "clock": {
      "type": "steppable",
      "step-ns": 3000000,
      "real-time-update-rate": 3000000,
      "pause-on-start": false
    }
    
    // Example: Real-time clock
    "clock": {
      "type": "real-time",
      "real-time-update-rate": 3000000
    }
    
    // Example: Engine-driven clock
    "clock": {
      "type": "engine-driven",
      "step-ns": 3000000
    }
  10. Constraints for multiple robots in Project AirSim

    main

    When configuring multiple robots, keep the following constraints in mind:

    • Interaction: Robots interact with the common environment and the simulation clock, but they do not interact with each other except for basic collision detection.
    • Physics Types: Each robot can have its own physics type, but a single robot cannot have multiple physics types assigned to it.
    • Identification: The client interacts with drones using the unique name identifier provided in the actor configuration.
  11. Understand the Robot and Sensor Actor Hierarchy

    main

    Project AirSim uses a structured Unreal Engine Actor hierarchy to represent physical entities.

    Robot Actor System

    Robots are composed of three main levels:

    • AUnrealRobot: The base class containing the Skeletal Mesh, Physics components, and sensor attachments.
    • AUnrealRobotLink: Represents the physical links, containing collision shapes, visual meshes, and joint attachments.
    • AUnrealRobotJoint: Manages the connections between links using constraint components, motor controllers, and state publishers.

    Sensor Actor System

    Sensors are integrated as specialized actors:

    • AUnrealSensor: The base class handling tick-based updates, data publishing, and configuration.
    • Camera Sensors: Includes AUnrealCamera and AUnrealViewportCamera using a render request system.
    • Distance Sensors: Includes AUnrealLidar (utilizing Raycasting and GPU Compute), AUnrealRadar, and AUnrealDistanceSensor.
    • Environmental Sensors: Includes IMU simulation, GPS/Geodetic conversion, and Barometer/Altimeter.