DOCSEARCHbyAlgolia
Submit Docs

CVAT (Computer Vision Annotation Tool)

repository·develop·Indexed 12 days ago

https://github.com/cvat-ai/cvat

An open-source, web-based platform for annotating images, videos, and 3D point clouds to build high-quality visual datasets for computer vision and AI. Includes a CLI for managing projects and tasks, as well as specialized modules like CVAT-CANVAS and CVAT-CANVAS-3D for interactive annotation rendering.

Tokens
166.7K
Snippets
335
Records
756
Agent score
97%

What's inside CVAT

  1. Explore CVAT Documentation Categories

    develop

    The CVAT documentation is organized into several key functional areas to help users navigate from basic usage to advanced development:

    • Getting Started: Basic overview, vocabulary, and keyboard shortcuts.
    • Workspace: Managing Projects, Tasks, Jobs, Cloud Storages, and Models.
    • Account Management: Handling access, roles, organizations, SSO, and subscriptions.
    • Dataset Management: Working with data formats, import/export, and annotation specifications.
    • Annotation: Using the editor, annotation tools, and automated labeling features.
    • QA & Analytics: Quality control processes, consensus mechanisms, and analytics.
    • Integrations: Connecting with external tools like FiftyOne and Human Protocol.
    • System Administration: Installation guides, AWS setup, superuser management, and advanced configuration.
    • Developers: Technical resources including the API, SDK, CLI, and token management.

  2. Available Auto-annotation libraries

    develop

    Beyond torchvision, CVAT supports auto-annotation functions built on several other major computer vision libraries:

    • Hugging Face Transformers: Supports all models compatible with image classification, object detection, and image segmentation pipelines.
    • Segment Anything Model 2 (SAM2).
    • Ultralytics: Supports all numbered YOLO models (e.g., YOLOv3, YOLOv5, etc.).

    For specific usage instructions for these models, refer to the ai-models directory in the CVAT source repository.

  3. Compare CVAT Online paid plans

    develop

    CVAT Online offers several paid plans depending on your use case:

    • Solo: Designed for personal use only. It removes limitations of the Free plan but is not intended for organizational collaboration. While it allows creating an organization and up to 2 members for trial purposes, these members will still be subject to Free plan limitations.
    • Team: Designed for collaboration within an organization. It removes all Free plan limitations for the entire organization.
      • Pricing Note: The monthly cost depends on the number of team members. The organization owner counts as a seat. For example, if you have 2 annotators, you must pay for 3 seats (2 annotators + 1 owner).
    • Annual Plans: Available for both Solo and Team users. Subscribing annually can save up to 30% compared to monthly billing.

  4. Choose the right CVAT edition

    develop

    CVAT is available in three primary editions depending on your deployment needs and organizational requirements:

    • CVAT Community: A free, open-source edition that you can deploy on-premises or in your own cloud. It includes the full annotation toolset, import/export formats, and core workflows. Best for technical teams managing their own infrastructure.
    • CVAT Online: A hosted cloud edition with managed infrastructure, automatic updates, and maintenance. It offers multiple subscription tiers (Free, Solo, Team) and is designed for fast onboarding and easy collaboration.
    • CVAT Enterprise: Designed for large organizations and regulated environments. It includes advanced features like SSO/LDAP, audit logs, dedicated support, and custom SLAs. Can be deployed as a managed service, on-premises, or in a private cloud.

  6. Use the Low-level API for direct REST interactions

    develop

    The low-level API (located in cvat_sdk.api_client) allows you to work directly with the CVAT REST API while benefiting from data validation and IDE syntax assistance. It is autogenerated and provides a structured way to interact with server endpoints.

    Key features:

    • Data Validation: Uses model objects to ensure request data conforms to the expected schema.
    • Flexible Input: You can pass parameters as plain Python dictionaries (with JSON-serializable data) or as formal model objects from cvat_sdk.api_client.models.
    • Error Handling: Catch exceptions.ApiException to handle server-side errors.
    from cvat_sdk.api_client import Configuration, ApiClient, models, apis, exceptions

configuration = Configuration(host="http://localhost", username='USER', password='PASSWORD')

with ApiClient(configuration) as api_client:
    try:
        # Using a dict for a request
        task_spec = {'name': 'example task'}
        (task, response) = api_client.tasks_api.create(task_spec)
    except exceptions.ApiException as e:
        print(f"Exception: {e}")

  7. Integrate CVAT with external services

    develop

    CVAT supports several integrations to enhance annotation and dataset management:

    • FiftyOne: An open-source tool for dataset management and model analysis that is closely integrated with CVAT for label refinement and annotation enhancement.
    • Hugging Face & Roboflow: In CVAT Online, models from these platforms can be integrated to enhance computer vision tasks.
    • Human Protocol: Integrates CVAT to augment annotation services within the Human Protocol framework.

  8. Access CVAT via Server API, Python SDK, or CLI

    develop

    CVAT provides multiple interfaces for interacting with the server, depending on your use case:

    • Server API: A HTTP REST API used by client applications, browsers, or custom scripts to interact with CVAT via standard HTTP requests and responses.
    • Python SDK: A Python library that provides high-level access to server interactions, including built-in data validation and serialization capabilities.
    • Command Line Tool (CLI): A tool for managing CVAT tasks directly from the terminal. It is currently used for basic task management and is evolving into a more advanced administration tool.

  9. Integration layers for CVAT

    develop

    CVAT can be integrated into workflows through three primary layers:

    1. Server REST API: Accessible via standard HTTP requests; includes a Swagger schema for exploration.
    2. Python SDK: A client library providing both a low-level REST API client and high-level wrappers for easier development.
    3. Command-line tool (CLI): A tool for interacting with CVAT via the terminal.

  10. Overview of CVAT SDK API layers

    develop

    The cvat_sdk package is organized into several functional layers:

    LayerModuleDescription
    Server API Wrapperscvat_sdk.api_clientLow-level wrappers for the CVAT Server API.
    High-level Toolscvat_sdk.coreCore high-level abstractions for interacting with CVAT.
    PyTorch Adaptercvat_sdk.pytorchIntegration layer for PyTorch models.
    Auto-annotationcvat_sdk.auto_annotationSupport for automated annotation workflows.
    Mask Utilitiescvat_sdk.masksUtilities for handling masks (requires [masks] extra).
    Attribute Utilitiescvat_sdk.attributesUtilities for managing attributes.

  11. What is a Dataset Manifest and when to use it

    develop

    A Dataset Manifest is a JSONL text file used to provide CVAT with extra information about input data. This is primarily used to:

    • Reduce network traffic and speed up task creation when working with cloud storages.
    • Enable caching mode when using a video file or a set of images as a data source.
    • Control file ordering when the predefined sorting method is selected in task configuration.

    Important for Image Archives: If you are using image archives (like .zip) and want to use the predefined sorting method, you must provide a .jsonl manifest file alongside the archive in the input list. The manifest must be outside the archive, not inside it.

  12. Use Models for Request and Response data

    develop

    The low-level API uses distinct models for requests and responses, found in cvat_sdk.api_client.models:

    • Request Models: Typically suffixed with Request (e.g., TaskWriteRequest).
    • Response Models: Have no suffix (e.g., Task).

    Key Capabilities:

    • Recursive Dict Support: You can pass a plain Python dictionary to a method or a model constructor, and the SDK will automatically parse the fields into the correct types.
    • Interface Classes: Most models provide an interface class named I<ModelName> (e.g., ITaskWriteRequest) which provides type annotations for implementation.
    • Serialization: Use model.to_dict() or cvat_sdk.api_client.model_utils.to_json() to convert model instances back to dictionaries or JSON.
    from cvat_sdk.api_client import models

# Using a model object instead of a dict
task_spec = models.TaskWriteRequest(
    name='example task',
    labels=[
        models.PatchedLabelRequest(
            name="car",
            color="#ff00ff"
        )
    ]
)
api_client.tasks_api.create(task_spec)