DOCSEARCHbyAlgolia
Submit Docs

Frigate NVR

repository·dev·Indexed 12 days ago

https://github.com/blakeblackshear/frigate

A local, AI-powered Network Video Recorder (NVR) designed for real-time object detection from IP cameras. It features deep integration with Home Assistant and utilizes efficient motion detection to trigger AI processing.

Tokens
139.4K
Snippets
344
Records
640
Agent score
98%

What's inside Frigate

  1. What is Birdseye and how to use it

    dev

    Birdseye is a portable heads-up view of your cameras that allows you to see activity across your property without monitoring every camera simultaneously. It can intelligently show or hide cameras based on activity.

    To view Birdseye in the Web UI:

    1. Go to the Live page.
    2. Press the pencil icon in the sidebar to add a Camera Group.
    3. Choose Birdseye as one of the cameras in the group.

    Birdseye can also be used in Home Assistant dashboards or cast to media devices.

    Note on Image Quality: Birdseye uses frames from the stream assigned the detect role. If the Birdseye view looks low quality, you must increase the detect stream resolution (width/height) for that camera.

  2. Overview of Third Party Frigate Extensions

    dev

    Frigate's open-source nature allows for various third-party extensions and integrations. Most of these services are designed to interface with Frigate's unauthenticated REST API, typically accessible over port 5000.

    Note: This list is not exhaustive and the project does not recommend or rate these external tools. Users should vet them independently before installation.

  3. Overview of Frigate NVR™

    dev

    Frigate NVR™ is a complete local Network Video Recorder (NVR) designed for Home Assistant integration. It performs real-time AI object detection on IP camera streams locally using OpenCV and TensorFlow.

    Key features include:

    • AI Object Detection: Uses TensorFlow running in separate processes to maximize FPS.
    • Resource Efficiency: Employs low-overhead motion detection to trigger object detection only when necessary.
    • Home Assistant Integration: Deeply integrates via a custom component.
    • Communication: Uses MQTT for easy integration with other systems.
    • Recording Logic: Supports 24/7 recording and allows setting retention times based on detected objects.
    • Streaming: Reduces camera connection load via RTSP re-streaming and supports low-latency viewing through WebRTC and MSE.
    • Hardware Acceleration: Strongly recommends using a GPU or AI accelerators (e.g., Google Coral or Hailo) for significantly higher efficiency and lower power consumption compared to CPUs.

  4. Supported hardware for object detection

    dev

    Frigate uses dedicated AI accelerators or GPUs to perform computationally intensive object detection. A detector is the specific hardware-and-model backend used for inference.

    Supported Hardware Categories:

    • Most Hardware: Google Coral EdgeTPU (USB, Mini PCIe, m.2), Hailo-8/8L, and MemryX MX3.
    • AMD: ROCm (on AMD Discrete GPUs) or ONNX.
    • Apple Silicon: M1 and newer devices.
    • Intel: OpenVINO (Arc GPUs, integrated GPUs, or CPUs) or ONNX.
    • Nvidia GPU: ONNX (in -tensorrt image) or TensorRT (on Jetson devices).
    • Rockchip: RKNN (on Rockchip NPUs).
    • Synaptics: synap models (on Synaptics NPUs).
    • AXERA: AXEngine (on AXERA AI acceleration).
    • Testing: CPU detector (not recommended for production; OpenVINO in CPU mode is preferred).

    Important Constraint: You cannot mix different detector types for object detection (e.g., you cannot use OpenVINO and Coral EdgeTPU simultaneously for detection tasks).

  5. Supported hardware detectors overview

    dev

    Frigate supports a wide range of hardware accelerators for object detection across different architectures:

    Most Hardware

    • Hailo-8 / Hailo-8L: AI Acceleration modules (M.2 or RPi HAT). Best with tiny or small models.
    • Google Coral EdgeTPU: USB or M.2 format. Supports ssdlite and mobilenet architectures.
    • MemryX MX3: M.2 accelerator. Best with tiny, small, or medium models.

    AMD

    • ROCm: For discrete AMD GPUs. Supports limited model architectures.

    Apple Silicon

    • Apple Silicon (M1+): Uses the onboard NPU. Primarily supports ssdlite and mobilenet. Note: Runs via a ZMQ proxy as it cannot run inside a container.

    Intel

    • OpenVINO: Supports Intel Arc GPUs, integrated GPUs (iGPU), and Intel NPUs. Supports most model architectures.

    Nvidia

    • Nvidia GPU: Uses the onnx detector type. Supports most architectures via ONNX. Works with Jetson devices via TensorRT or ONNX (Jetpack 6).

    Other Platforms

    • Rockchip (RKNN): Uses Rockchip NPUs. Best with tiny or small models.
    • Synaptics: Uses synap models on Synaptics devices.
    • AXERA: Uses axera models via AXEngine.

  6. Configure hardware acceleration for video decoding

    dev

    Hardware acceleration is highly recommended to offload video decoding from the CPU to an integrated or discrete GPU.

    Verification

    • Logs: Check if Frigate reports that hardware acceleration was automatically detected or if there is a warning that it was not.
    • Errors: If you have manually specified hardware acceleration in your config, ensure the logs are free from errors. Note that there is no CPU fallback if hardware acceleration fails.

    Supported Hardware Presets

    Frigate provides presets for several hardware types:

    • AMD: Uses VAAPI for integrated and discrete GPUs.
    • Intel: Uses VAAPI or Intel QuickSync (QSV) for integrated and Arc GPUs.
    • Nvidia GPU: For modern Nvidia GPUs.
    • Raspberry Pi 3/4: Uses the Raspberry Pi media engine.
    • Nvidia Jetson: Uses the Jetson media engine.
    • Rockchip: Uses the RockChip SOC media engine.

  8. What is Semantic Search in Frigate?

    dev

    Semantic Search allows you to find tracked objects in your review items using images, user-defined text descriptions, or automatically generated descriptions. It works by creating embeddings (numerical vector representations) for both images and text. By comparing these embeddings, Frigate can perform similarity searches.

    Key capabilities:

    • Text -> Image: Search for objects using text descriptions (e.g., typing "red car" in the search box).
    • Image -> Image: Find similar objects by clicking "Find Similar" on a tracked object's detail pane.

    Frigate uses models from Jina AI to generate these embeddings locally. Note that a one-time internet connection is required to download the models from HuggingFace; once cached, the feature works fully offline.

  9. What is a detector in Frigate?

    dev
    A detector is a hardware device optimized for running object detection inferences efficiently. Using a dedicated detector instead of the CPU reduces latency and dramatically lowers CPU load by offloading TensorFlow operations. Frigate is designed with the expectation that a hardware detector will be used to achieve very low inference speeds.

  10. Optimize detector performance and capacity

    dev

    The detector runs the AI model for object detection. Its performance is measured by inference speed, which determines the total capacity of detections per second your system can handle.

    Calculating Capacity: Capacity = 1000ms / inference speed (ms). Example: An inference speed of 10ms allows for 100 detections per second.

    Model Size Trade-offs:

    • 320x320 (Small): Faster inference; Frigate is specifically optimized for this size.
    • 640x640 (Large): Slower inference; may provide higher accuracy for objects occupying most of the frame.

    If your cameras require more detections per second than your detector's capacity, you will experience delays or missed detections.

  11. Restrict review items to specific zones

    dev

    By default, review items are created if the configured labels are detected anywhere in the camera frame. To reduce noise, you can restrict review items so they are only created when an object enters a specific zone.

    Note: Because zones are spatial, they do not apply to audio labels; audio labels will always be marked as a detection by default.

  12. Verify ONVIF PTZ compatibility for autotracking

    dev

    To use Frigate's autotracking feature, your PTZ camera must support specific ONVIF features. You can check compatibility using the ONVIF Conformant Products Database.

    Required ONVIF features for autotracking:

    • PTZRelative
    • PTZRelativePanTilt
    • PTZRelativeZoom

    If these features are missing, autotracking will not function, although basic PTZ controls may still work via the WebUI. Note that even if a camera claims support, it may still fail to respond; refer to the community-maintained compatibility list for known working models.