DOCSEARCHbyAlgolia
Submit Docs

Limelight Vision System Documentation

website·Indexed 33 days ago

https://docs.limelightvision.io/docs/docs-limelight/getting-started/summary

Documentation for Limelight vision sensors, including hardware guides for Limelight 1, 2, 2+, and 3. Covers Limelight OS, AprilTag tracking with MegaTag2, neural network classifier training, barcode decoding, and Python SnapScript pipelines. Includes technical specifications for 3D coordinate systems, JSON status APIs, and tutorials for robot localization, visual servoing, and distance estimation in FRC.

Tokens
47.5K
Snippets
148
Records
352
Agent score
63%

What's inside Limelight

  1. Overview of Limelight Robotic Perception capabilities

    The Limelight Smart Camera provides a platform for robotic perception that supports both zero-code configuration and custom scripting. Users can configure computer vision pipelines for color blobs, AprilTags, and neural networks via a built-in web interface. For advanced users, custom Python SnapScript pipelines can be written using TensorFlow and OpenCV through the web interface or Visual Studio Code.

  2. Overview of MegaTag2 Robot Localization

    MegaTag2 is a precise, ambiguity-free AprilTag-based localizer introduced in 2024. It provides robust pose estimates from a single tag regardless of perspective and reduces the need for heavy filtering. Unlike MegaTag1, MegaTag2 requires the robot's heading (yaw) to be provided. For FRC 2024 and beyond, the coordinate system origin is always the 'blue' origin; teams should use botpose_orb_wpiblue for pose-related functionality.

  3. Overview of ChArUco Camera Calibration

    Camera calibration determines the camera matrix (K) and distortion coefficients to provide more accurate 2D and 3D measurements. While every Limelight unit comes with a default calibration suitable for most use cases, custom calibration is recommended for tasks requiring higher precision beyond standard 'servoing' (aligning a target to a crosshair).

    Key Calibration Components:

    • Camera Matrix (K): A 3x3 matrix containing focal lengths (fx, fy) and the principal point (cx, cy), which is the pixel aligned with the optical axis.
    • Distortion Coefficients: Five coefficients ([k1, k2, p1, p2, k3]) that account for radial distortion (curved lines) and tangential distortion (lens not parallel to the image plane).

  5. Overview of Python SnapScript Pipelines

    Python SnapScript allows developers to build custom vision pipelines using OpenCV and numpy. The Python interpreter is integrated into Limelight's C++ backend, providing a sandboxed environment where crashes are handled gracefully and script changes are applied instantly. Limelight manages hardware, camera interfacing, networking, and streaming, requiring the user to only implement the runPipeline() function.

  7. Configure Crosshair Calibration modes

    Limelight provides two crosshair calibration modes:

    • Single: The standard mode where robots are aligned manually and calibrated to re-zero targeting values about the crosshair.
    • Dual: An advanced mode for robots requiring a dynamic crosshair that automatically adjusts based on the target's area or distance. This is useful for robots with curved shot paths or misaligned/uncentered Limelight mounts.

    Calibration supports separate X and Y axis adjustments.

  9. Limelight 4 Software Capabilities

    Limelight OS provides a browser-based UI for configuring vision pipelines. Supported interfaces include REST/HTTP, Websocket, NetworkTables, ModbusTCP, and ROS.

    Pipeline Capabilities:

    • AprilTag Tracking: 2D/3D tracking, Robot Localization (MegaTag1/MegaTag2), and IMU Fusion (Internal/External).
    • Neural Networks: MobileNetSSD (CPU, 20FPS) and YoloV8 (Requires Hailo-8, 80FPS).
    • Other Pipelines: Color Blobs (120-240FPS), Barcode Tracking, Lens Focusing, and low-latency MJPEG streaming.
    • Calibration: Built-in Charuco Intrinsics Calibration interface.

    Known Issues:

    • Hailo Accelerated Apriltag framerates are too low when downscaling is enabled.
    • NetworkTables temperature readings are inaccurate while throttling is enabled.

  10. Use MegaTag for stable robot pose estimation

    MegaTag is a botpose implementation that combines all visible AprilTags into a single 3D tag with multiple keypoints. This approach provides greater accuracy, reduced noise, and resilience to tag flipping (ambiguity) compared to averaging individual tag poses. It is most effective when multiple tags are in view and are not close or coplanar; stability increases as the number of keypoints and the physical distance between tags increase. It supports tags in any 3D orientation, including floor and ceiling tags.

  11. Retrieve robot location using MegaTag localization

    When the Limelight's robot-space pose and a field map are configured in the web UI, the robot's location in field space is calculated using the MegaTag algorithm. MegaTag is resilient to individual tag ambiguities and noise, especially when multiple tags are in view. The robot's location (x, y, z in meters; roll, pitch, yaw in degrees) is available via the getBotpose() method or the botpose NetworkTables array.

  12. Utilize Hailo 8L AI acceleration and classification

    Limelight OS 2026.1 provides several Hailo 8L enhancements:

    • Default Detector: Ships with a default 8L detector and labels file for immediate testing.
    • Accelerated AprilTags: Provides up to 50% performance improvement at 1x downscaling.
    • Zero-Shot Classification: Uses a transformer-based backend for language-based classification. Users can edit the label list at runtime without training. Results are output via the Classifier JSON array and NetworkTables keys (tcclass, classifier class name/ID).
    • UI Indicator: The top-right corner of the UI indicates if a Hailo module is detected.