DOCSEARCHbyAlgolia
Submit Docs

FiftyOne Documentation

repository·develop·Indexed 27 days ago

https://github.com/voxel51/fiftyone

An open-source tool for visual AI development, enabling dataset visualization, labeling, and computer vision model evaluation. Includes detailed technical documentation on the Annotation Engine architecture, featuring support for retained-mode and declarative surfaces, atomic transactions, undo/redo management, and high-frequency transient data handling via SignalPipe.

Tokens
287K
Snippets
678
Records
1.3K
Agent score
91%

What's inside FiftyOne

  1. Overview of @fiftyone/video-annotation

    develop

    The @fiftyone/video-annotation package provides a specialized UI surface for annotating video content one frame at a time. It allows users to draw bounding boxes, mark keyframes, and use propagation tools (like SAM2) to fill in gaps between keyframes.

    Key architectural concepts:

    • Surface vs. Engine: This package is a video-specific UI surface. It manages gestures, media tiles, streams, and the timeline. The actual label state (identity, transactions, undo, selection, and persistence) is managed by the underlying @fiftyone/annotation engine.
    • Renderer Abstraction: The renderer (using @fiftyone/lighter) does not interact with the video directly. Instead, it uses an ExternalCanonicalMedia overlay to determine frame position and size, while the actual video pixels are rendered by a standard <video> element or a <canvas> sitting behind the overlay layer.
    • Temporal Truth: Tracks are reconstructed entities. On disk, data exists as individual detections on specific frames tagged with an instanceId. A 'track' is a reconstructed view of these detections grouped by ID across a clip.

  5. Overview of Data Lens

    develop

    Data Lens is a feature within the FiftyOne Enterprise App that enables users to explore and import samples from external data sources (such as PostgreSQL, Databricks, or BigQuery) directly into FiftyOne.

    The workflow consists of three main steps:

    1. Define your search experience: Customize how you interact with your data source using a flexible framework.
    2. Connect your data source: Provide configuration to connect FiftyOne to your external data.
    3. Interact with your data: Search, visualize, and import samples into a FiftyOne dataset for analysis.

  6. Overview of Looker media viewer

    develop
    Looker is a client-side JavaScript media viewer designed to render label overlays on top of media. It supports both Image and Video formats and includes features for zooming, panning, cropping, and fullscreen viewing. It is capable of rendering various label types including Classification, Detection, Keypoint, Polyline, and Segmentation.

  7. Overview of Albumentations Integration

    develop

    The Albumentations integration allows you to apply image augmentation pipelines directly to FiftyOne datasets and visualize the results in the App.

    Supported Label Types:

    • Detections
    • Keypoints
    • Segmentation
    • Heatmap
    • Images (pixel-level transformations)

    Key Features:

    • Apply transformations to a whole dataset, a current view, or selected samples.
    • Visualize effects in real-time.
    • View samples from the most recent transformation run.
    • Save augmented samples and transformation pipelines to the dataset for reproducibility.

  8. Overview of the FiftyOne Model and Dataset Zoo

    develop

    The FiftyOne Zoo provides a unified interface for managing computer vision resources. It is divided into two main components:

    Dataset Zoo

    Provides access to built-in datasets across various CV tasks:

    • Classification: CIFAR-10, CIFAR-100, ImageNet, Caltech-101, Caltech-256
    • Detection: COCO, VOC, Open Images, KITTI, BDD100K
    • Video: ActivityNet, Kinetics, UCF101, HMDB51
    • 3D: Quickstart-3D, KITTI Multiview
    • Specialized: Fashion-MNIST, Places, LFW, FIW

    It also supports Remotely-Sourced Datasets hosted on GitHub repositories, cloud storage URLs, or custom dataset definitions.

    Model Zoo

    Provides ready-to-use pre-trained models for:

    • Classification Models: Image classification architectures.
    • Detection Models: Object detection and localization.
    • Segmentation Models: Instance and semantic segmentation.
    • Video Models: Action recognition and video understanding.

    Users can also integrate Custom Models from GitHub, custom model wrappers, or community/team-specific contributions.

  9. Overview of Dataset Versioning in FiftyOne Enterprise

    develop

    FiftyOne Enterprise supports dataset versioning through a linear sequence of read-only Snapshots.

    • HEAD: The current working version of a dataset. It can be edited (additions, updates, deletions) as long as you have sufficient permissions.
    • Snapshots: Immutable records of the dataset's state at a specific point in time. They can be created from the HEAD but cannot be edited directly.

    What is tracked

    Snapshots capture dataset-level metadata (schema, tags, saved views, runs/results) and sample-level metadata (tags, labels, detections, segmentations, custom fields, and video frame metadata).

    What is NOT tracked

    Snapshots do not track external information that is assumed to be immutable, such as:

    • Media files (images, videos, point clouds) stored in cloud buckets or on disk.
    • Segmentations or heatmaps stored externally.
    • Run results or embeddings stored in external systems (e.g., a vector database like Qdrant).

  11. Overview of FiftyOne Enterprise

    develop
    FiftyOne Enterprise is a collaborative platform built on top of open-source FiftyOne workflows. It enables multiple users to securely collaborate on datasets and models, either on-premises or in the cloud. It is fully backwards-compatible with the open-source project, meaning all open-source FiftyOne documentation and workflows apply to Enterprise deployments as well.