openpilot

repository·master·Indexed 13 days ago

https://github.com/commaai/openpilot

An open source operating system for robotics that serves as an advanced driver assistance system for over 300 supported car models. It includes cereal for pub/sub messaging using Cap'n proto, a comprehensive coordinate system for robotics transformations, and tools for CAN data visualization via Cabana.

Tokens
49.2K
Snippets
124
Records
190
Agent score
100%

What's inside openpilot

  1. Overview of openpilot capabilities

    master

    openpilot is an open source driver assistance system designed to run on the comma four hardware. It provides several Advanced Driver Assistance System (ADAS) functions, including:

    • Adaptive Cruise Control (ACC): Manages vehicle speed and following distance.
    • Automated Lane Centering (ALC): Assists with steering to keep the vehicle in its lane.
    • Forward Collision Warning (FCW): Alerts the driver to potential frontal collisions.
    • Lane Departure Warning (LDW): Alerts the driver if the vehicle drifts from its lane.
    • Driver Monitoring (DM): A camera-based feature that alerts distracted or asleep drivers while the system is engaged.

    openpilot works by utilizing the car's existing APIs for its built-in ADAS systems, providing improved acceleration, braking, and steering inputs compared to stock systems.

  2. What is Cabana?

    master
    Cabana is a tool designed for viewing raw CAN data. It is primarily used for creating and editing CAN Dictionaries (DBC files). It integrates directly with commaai/opendbc, allowing users to load DBC files from the source and save changes to their own forks. Additionally, it supports loading routes from comma connect.
  3. Overview of openpilot tools directory structure

    master

    The tools/ directory contains various utilities for development, analysis, and simulation. Key subdirectories include:

    • cabana/: View and plot CAN messages from drives or in realtime.
    • camerastream/: Stream camera data over the network.
    • joystick/: Control your car using a joystick.
    • lib/: Libraries supporting tools and reading openpilot logs.
    • plotjuggler/: Tool for plotting openpilot logs.
    • replay/: Replay recorded drives and mock openpilot services.
    • sim/: Run openpilot within a simulator.

    Other relevant directories in the root include:

    • car_porting/: Tools for porting new cars.
    • release/: Scripts for building openpilot releases.
    • scripts/: Miscellaneous scripts.
  4. Debug controls using joystick_control

    master

    The joystick_control.py tool allows you to connect a laptop to a comma device over a network to debug vehicle controls using a joystick or keyboard.

    Prerequisites:

    • The car must be off.
    • openpilot must be offroad.
    • Hardware: A device running openpilot, a laptop, and an optional joystick.

    joystick_control uses the inputs Python library, which supports many common gamepads and joysticks.

  5. Identify supported vehicles for openpilot

    master
    A supported vehicle is one that provides a seamless experience when a comma device is installed, offering improved performance over the stock driver assistance system. Unless otherwise specified, all supported vehicles refer to the US market.
  6. Safety requirements for using openpilot

    master

    openpilot is a Level 2 Driver Assistance System (Adaptive Cruise Control and Automated Lane Centering) and is a failsafe passive system.

    Critical Safety Requirements:

    • Driver Alertness: The driver must remain alert and pay attention at all times. Driver monitoring is included to detect distraction, but driver alertness is necessary but not sufficient for safe use.
    • Manual Control: The driver must always be able to immediately retake manual control by stepping on the brake pedal or pressing the cancel button.
    • Actuator Constraints: The system is designed so the vehicle does not alter its trajectory too quickly for a driver to react. Actuators are constrained to operate within reasonable limits (observing ISO11270 and ISO15622, such as limiting lateral actuation to 0.9 seconds for a 1m lateral deviation).

    openpilot is provided with no warranty of fitness for any purpose.

  7. Understand the limitations of openpilot ACC and FCW

    master

    openpilot ACC (Adaptive Cruise Control) and FCW (Forward Collision Warning) are not intended for inattentive driving. Drivers must remain ready to take control of the gas and brake at all times.

    Key Operational Constraints:

    • Acceleration/Deceleration: openpilot is designed with limits on the amount of acceleration and deceleration it can produce, which may affect performance during abrupt braking maneuvers.
    • Speed Limits: openpilot does not detect posted speed limits. If the posted limit is below your selected set speed, the system may not function as expected.
    • Detection Gaps: openpilot does not currently detect traffic signs or stop lights. It may struggle with stationary vehicles in the same lane or close cut-ins from neighboring lanes.
    • Environmental Interference: Approaching toll booths, bridges, or large metal plates, and interference from other radar-generating equipment.

    Factors that may interfere with performance:

    • Visibility/Weather: Poor visibility (rain, snow, fog) or bright light.
    • Hardware Obstructions: Obstructed road-facing camera or radar (mud, ice, snow) or incorrect device mounting.
    • Road/Traffic Conditions: Pedestrians, cyclists, hills, narrow/winding roads, or extreme temperatures.
    • Vehicle Modifications: Excessive paint, wraps, stickers, or rubber coatings on the vehicle.
  8. Data collection and privacy

    master

    By default, openpilot uploads driving data to comma servers to train models and improve the software.

    Data logged includes:

    • Road-facing cameras
    • CAN bus data
    • GPS, IMU, magnetometer, and thermal sensors
    • Crashes and operating system logs

    Privacy Notes:

    • Driver-facing camera and microphone: These are only logged if you explicitly opt-in via settings.
    • Opt-out: Users can disable data collection if they wish.
    • Data Access: You can access your data through comma connect.
  9. Best practices for cereal messaging

    master

    When defining or modifying cereal messages, follow these rules to ensure data quality and usability:

    • Use SI units: All fields must describe quantities in SI units unless the field name explicitly specifies otherwise.
    • Unambiguous naming: Field names must be completely unambiguous within their context.
    • Human-readability: Values should be easy to plot and human-readable with minimal parsing.
  10. Maintain backwards-compatibility in cereal

    master

    To ensure old logs can be parsed by new versions of cereal, follow these rules for modifying the messaging spec:

    • Safe changes: Adding new structs or adding new members to existing structs is generally safe.
    • Unsafe changes: Most other modifications (like changing existing field types or IDs) will break compatibility.
    • Reference: For detailed Cap'n proto compatibility rules, see Cap'n proto documentation.
  11. Understand the reference frames used in openpilot

    master

    openpilot uses several coordinate systems (reference frames) for positioning and orientation. Transformations between these frames are typically performed by generating a rotation matrix and multiplying.

    Key Reference Frames

    Name[x, y, z]UnitsNotes
    Geodetic[Latitude, Longitude, Altitude]geodeticAvoid using this frame if possible; it is sometimes used as [lon, lat, alt].
    ECEF[x, y, z]metersEarth-Centered, Earth-Fixed. Uses ITRF14 (IGS14). This is the global Mesh3D frame.
    NED[North, East, Down]metersNorth-East-Down. Relative to earth's surface; useful for visualization.
    Device[Forward, Right, Down]metersThe Mesh3D local frame. Relative to the camera, not the IMU.
    Calibrated[Forward, Right, Down]metersThe frame used for model outputs. Aligned with Car frame in pitch/yaw, and Device frame in roll. Origin is the same as Device frame.
    Car[Forward, Right, Down]metersAligned with the vehicle's direction of travel and the road plane (on flat roads/no turns). Origin is on the road plane directly below the Device frame.
    View[Right, Down, Forward]metersSimilar to Device frame, but follows camera conventions.
    Camera[u, v, focal]pixels2D representation on the camera image.
    Normalized Camera[u / focal, v / focal, 1]/
    Model[u, v, focal]pixelsThe sampled rectangle of the full camera frame used by the model.
    Normalized Model[u / focal, v / focal, 1]/