DOCSEARCHbyAlgolia
Submit Docs

openpilot Driver Assistance System

repository·carrot-wip·Indexed 41 days ago

https://github.com/ajouatom/openpilot

An open source driver assistance system featuring opendbc, a Python API for vehicle CAN bus interfacing, and cereal, a messaging system using Cap'n Proto and MSGQ for high-performance inter-process communication. The system includes tools for reverse engineering CAN messages via Cabana and a framework for porting new car models through fingerprinting and DBC file generation.

Tokens
141.5K
Snippets
390
Records
670
Agent score
45%

What's inside openpilot

  1. Overview of openpilot tools directory structure

    carrot-wip

    The tools/ directory contains various utilities for interacting with, visualizing, and simulating openpilot:

    • cabana/: View and plot CAN messages from drives or in realtime.
    • camerastream/: Network streaming for cameras.
    • joystick/: Control the car using a joystick.
    • lib/: Libraries supporting tools and reading openpilot logs.
    • plotjuggler/: Tool for plotting openpilot logs.
    • replay/: Replay drives and mock openpilot services.
    • scripts/: Miscellaneous scripts.
    • serial/: Tools for using the comma serial.
    • sim/: Run openpilot in a simulator.
    • webcam/: Run openpilot on a PC using webcams.

  2. Overview of Xiaoge Vision: V-ASM and Lane Detection

    carrot-wip

    Xiaoge Vision is a local vision extension for CarrotPilot. It runs two ONNX models using comma3's VisionIPC camera frames. The system writes versioned xiaogeVision JSON messages to customReservedRawData0. The card.py component then merges lane type and visual blind spot (V-ASM) results into carState for use by existing UI and lane change logic.

    Key Components:

    • Lane Detection: Uses VISION_STREAM_ROAD and assets/lane.onnx. It classifies lanes as 1 (solid), 0 (dashed), or -1 (unknown).
    • V-ASM (Blind Spot Detection): Uses VISION_STREAM_WIDE_ROAD and assets/v_asm_model.onnx. It detects objects in a user-defined ROI on the target side during lane changes to save CPU.
    • Managed Entrypoint: openpilot.selfdrive.carrot.xiaoge_data. This process starts only when the ShareData parameter is enabled. It provides both local vision recognition and the original TCP 7711 data service.

    Important Note: Do not manually start v_asm_server.py, as it will conflict with the managed entrypoint over port 8082 and create duplicate inference processes.

  3. Overview of Cabana

    carrot-wip
    Cabana is a tool designed for viewing raw CAN data. It is primarily used for creating and editing CAN Dictionaries (DBC files). It features direct integration with commaai/opendbc, allowing users to load DBC files directly from the source or save changes to their own forks. Additionally, Cabana can load drive routes from comma connect.

  4. What is openpilot?

    carrot-wip

    openpilot is an open-source driver assistance system. It provides the following functions for supported vehicles:

    • Adaptive Cruise Control (ACC): Manages vehicle speed and following distance.
    • Automated Lane Centering (ALC): Assists with steering to keep the vehicle in its lane.
    • Forward Collision Warning (FCW): Alerts the driver to potential frontal collisions.
    • Lane Departure Warning (LDW): Alerts the driver if the vehicle drifts from its lane.
    • Driver Monitoring (DM): A camera-based feature that alerts distracted or asleep drivers while the system is engaged.

    openpilot works by utilizing the car's existing APIs for its built-in Advanced Driver-Assistance System (ADAS) to provide improved acceleration, braking, and steering inputs compared to stock systems.

  5. What is MSGQ?

    carrot-wip

    MSGQ is a high-performance Inter-Process Communication (IPC) pub-sub system designed for a single publisher and multiple subscribers. It utilizes a ring buffer in shared memory for efficient data transfer.

    Key characteristics:

    • Single Producer, Multiple Consumer: Optimized for one writer and many readers.
    • Efficient Memory Usage: Writing can be done without a copy if the data size is known in advance. Reading requires a copy.
    • Deterministic Testing: Includes a spoofed implementation for testing purposes.
    • Specialized Variants: Includes visionipc, an IPC system specifically optimized for large contiguous buffers like images or video.

  7. Understand the opendbc project structure

    carrot-wip

    The opendbc repository is organized into several functional areas:

    • opendbc/dbc/: A repository of DBC (CAN Database) files.
    • opendbc/can/: A library for parsing and building CAN messages from DBC files.
    • opendbc/car/: A high-level Python library for interfacing with specific car models.
    • opendbc/safety/: Functional safety logic for all supported cars.

  8. Hardware Overview: tinybox variants

    carrot-wip

    The tinybox is a purpose-built AI infrastructure system available in different configurations:

    • Red Box: Includes six 7900XTX GPUs.
    • Green Box: Includes six 4090 GPUs.
    • Green v2 Box: Includes four 5090 GPUs.

    All boxes ship with a basic Ubuntu 22.04 installation. The default image includes tinygrad and PyTorch.

  10. Carrot Cluster HUD Display Features

    carrot-wip

    The Carrot Cluster HUD provides several visual indicators for vehicle status and navigation:

    Drive Status (Top HUD)

    Displays the following elements in a transparent rounded-square outline:

    • Gear: P/R/N/D or 1-8 (derived from carState.gearShifter and gearStep).
    • Front Gap Bars: Shows distance to the vehicle ahead. All four bars are visible; active bars use #bb3d91 and inactive bars are gray.
    • Cruise Set Speed: Shows the set speed. If cruise is paused, the speed is shown in gray. If inactive, it shows --- km/h.
    • LFA Active Icon: Uses selfdrive/assets/icons_mici/carrot_wheel_org.png, rotates by -carState.steeringAngleDeg, and turns green when active.
    • Lane Overlay: When controlsState.activeLaneLine is true, a carrot_wheel_lane.png overlay is drawn.

    Navigation and Speed Limits

    • Road Speed Limit: Uses carState.speedLimit (km/h).
    • Navigation Speed Limits: Accepts km/h or m/s. The cluster holds the last valid limit briefly during update gaps to prevent flicker.
    • Navigation Sources:
      • External (cam, section, bump, police, waze, road, atc, atc2, route): Displayed in orange.
      • Vehicle CAN (hda, hda_bump, school): Displayed in lavender.
      • Other Labels: turn (TBT), vturn (lane/curvature), model (prediction), and gas (gas override, in orange).
    • Connectivity Status:
      • NAVI (Orange): Appears if a remote sender or valid Carrot Navi v2 service is connected.
      • vNAVI (Lavender): Appears if Hyundai CAN-FD 0x4BE has been observed.

    Vehicle Status

    • EV Mode: A green EV indicator appears between vehicle speed and cruise-set speed when carState.evModeValid and carState.evModeActive are true. On Hyundai/Kia CAN-FD hybrids, this is enabled when 0xFA and 0x230 are present on ECAN with DLC32.
    • Driving Mode: Displays longitudinalPlan.myDrivingMode beside the traffic-state icon:
      • 1: eco (green)
      • 2: safe (orange)
      • 3: normal (white)
      • 4: high speed (red)
    • TPMS: A toy-car diagram shows tire pressure. Values below 31 psi are shown in red; missing values show --.

  12. Overview of Xiaoge Vision extension

    carrot-wip

    Xiaoge Vision is a local CarrotPilot vision extension that runs two ONNX models on comma3 VisionIPC frames. It publishes a versioned xiaogeVision JSON payload through customReservedRawData0. The card.py component merges lane-marking types and visual blindspot state into carState for consumption by existing UI and lane-change logic.

    Key technical details:

    • Managed Entry Point: openpilot.selfdrive.carrot.xiaoge_data.
    • Dependencies: Requires camerad to be running.
    • Conflict Warning: Do not manually launch v_asm_server.py, as it will conflict on port 8082 and create a duplicate vision process.
    • Data Service: The manager starts only when the ShareData parameter is enabled, providing both local vision and the existing TCP 7711 data service.