DOCSEARCHbyAlgolia
Submit Docs

Geti Vision AI Framework

repository·develop·Indexed 22 days ago

https://github.com/open-edge-platform/geti

An end-to-end Vision AI framework for taking users from raw images to deployed computer vision models. Geti provides a no-code web application for managing the model lifecycle—including data upload, annotation, training, and evaluation—as well as the getitune Python library for programmatic model optimization and deployment on Intel® XPU and NVIDIA® hardware.

Tokens
79.9K
Snippets
127
Records
299
Agent score
77%

What's inside Geti

  1. Overview of Geti Library (getitune) Skills

    develop

    The getitune library (located in library/) provides a suite of capabilities for machine learning workflows, including model management, dataset preparation, training, export, optimization, and inference. These capabilities are accessible via both the Python API and the getitune CLI.

    Core Capabilities

    SkillDescription
    getitune-discovering-modelsFinding and listing available models using list_models(...) or getitune find with task/name filtering.
    getitune-preparing-datasetsManaging datasets in formats like COCO, YOLO, or VOC. Supports Datumaro auto-detection and arguments like data= or --data_root.
    getitune-training-a-modelCreating engines with create_engine(...) and running engine.train() or engine.test(), or using getitune train/test CLI commands.
    getitune-exporting-a-modelExporting models via engine.export(...) or getitune export to formats like OpenVINO IR or ONNX with specific ExportFormat and Precision (e.g., FP32, FP16).
    getitune-optimizing-a-modelQuantizing models using OVEngine.optimize() for INT8 NNCF quantization, requiring a calibration set.
    getitune-running-inferencePerforming inference using engine.predict() or engine.test(), or getitune predict/test CLI commands across PyTorch, OpenVINO, or ONNX backends.

  2. Overview of Geti dataset import and export

    develop

    Geti provides functionality to transfer datasets into and out of the platform. This enables several workflows:

    • Migration to Geti: Import existing datasets prepared via manual methods or other tools (e.g., CVAT, Label Studio) to use Geti's training and inference features.
    • Integration with other tools: Export annotated datasets from Geti for use in external machine learning frameworks.
    • Backup and sharing: Export datasets for local backups or to share with users who do not have access to the specific Geti instance.
    • Dataset versioning: Export datasets at specific stages of annotation or preprocessing to maintain versions for experimentation.
    • Project forking: Export a dataset from one project and import it into another project within Geti, which allows for potential changes to task types and labels to facilitate data reuse.

  3. Overview of the Geti Library (getitune)

    develop

    The Geti™ library (getitune) is a low-code transfer learning framework designed for Computer Vision tasks. It allows developers to train, evaluate, optimize, and deploy models using PyTorch and the OpenVINO™ toolkit without requiring deep deep-learning expertise.

    Key capabilities include:

    • Multi-task support: Classification, object detection, instance segmentation, semantic segmentation, and keypoint detection.
    • Tiling: Support for processing large images in detection and segmentation tasks.
    • Multiple Backends: Training via PyTorch Lightning or Ultralytics YOLO; exporting/inference via ONNX and OpenVINO™.
    • Hardware Acceleration: Support for Intel GPU (XPU) and NVIDIA CUDA.
    • Data Handling: Uses a Datumaro frontend with automatic format detection (COCO, YOLO, VOC, etc.).
    • Advanced Training: Distributed training, mixed-precision training, and class-incremental learning.
    • Deployment: Export to OpenVINO™ IR and ONNX formats.

  4. Overview of Application Agent Skills

    develop

    The application stack consists of a FastAPI backend (application/backend/), a React + TypeScript frontend (application/ui/), and an OpenAPI contract connecting them. Development is organized into specific 'skills' that cover different parts of the stack.

    Important Note: Run commands from the relevant component root (application/backend/ or application/ui/) as they use different runtimes and toolchains.

  5. Overview of the Geti Application

    develop

    Geti is a full-stack web application designed for building and deploying computer vision AI models. It provides a graphical interface to manage the entire model lifecycle without requiring code for most tasks.

    Key Capabilities:

    • No-code model lifecycle: Handles data upload, annotation, training, evaluation, and deployment within a single UI.
    • Data and annotation workflows: Includes tools for managing datasets, labels, and revisions using both manual and AI-assisted annotation.
    • Pipeline-based deployment: Allows connecting data sources (cameras or files) to trained models and routing predictions to sinks like storage, MQTT, or webhooks.
    • Edge-oriented optimization: Supports exporting OpenVINO-optimized models for efficient inference on Intel hardware and other accelerators.

  6. Supported tasks in the Ultralytics Backend

    develop

    The Ultralytics backend supports the following computer vision tasks:

    • Detection: Object detection with bounding boxes.
    • Instance Segmentation: Instance segmentation with masks and bounding boxes.
    • Multi-class Classification: Single-label image classification.
    • Multi-label Classification: Multi-label image classification.
    • Semantic Segmentation: Dense pixel-wise classification.

  7. Geti™ Ecosystem

    develop

    Geti™ integrates with several specialized tools in the Open Edge Platform ecosystem:

    • Anomalib: Anomaly detection suite with experiment management and edge inference.
    • Instant Learn: Framework for zero-shot visual prompting on the edge.
    • Datumaro: Python library and CLI for dataset management and analysis.
    • OpenVINO™ & OpenVINO™ Model Server: For optimizing and serving models at scale.
    • Model API: Wrapper classes that simplify preprocessing, postprocessing, and routine procedures.
    • Physical AI Studio: Framework for imitation learning in robotics.

  8. Understand the quantization job lifecycle

    develop

    A quantization job follows a structured execution sequence:

    1. Validate Model: Verifies source model existence, successful training, availability of OpenVINO IR files, and ensures it isn't already quantized.
    2. Prepare Calibration Dataset: Loads the training dataset revision and extracts a validation subset limited by max_calibration_subset_size.
    3. Initialize OV Engine: Creates an OVEngine instance with the source model's OpenVINO IR and configures the data module.
    4. Run Quantization: Executes OVEngine.optimize().
      • If max_drop is provided: Uses nncf.quantize_with_accuracy_control() to maintain accuracy within the threshold.
      • If max_drop is omitted: Uses nncf.quantize() for standard Post-Training Quantization (PTQ).
    5. Evaluate Quantized Model: Evaluates the new model on the testing subset and stores metrics for comparison.
    6. Store Artifacts: Moves files to the model directory and updates the database with metadata.

  9. Understand the Benchmark Manifest schema

    develop

    The benchmark manifest (benchmark_manifest.yaml) defines the scope of automated model benchmarking. It is decoupled from the dataset catalog and the runner. It consists of three main sections:

    1. defaults: Global settings like num_seeds, eval_upto (the last phase to execute, e.g., optimize), and deterministic (enabling true ensures reproducibility but may slow training by 10–30%).
    2. experiments: Organized by task (e.g., detection, classification/multi_class_cls). Each task defines:
      • models: A list of models with name, priority (core, extended, or exploratory), and a recipe path.
      • datasets: References to entries in the benchmark_catalog.yaml.
      • scenarios: Optional parameter overrides (e.g., tiling, lr_high).
      • criteria: Accuracy metrics (e.g., mAP, accuracy) and thresholds for regression testing.
    3. scenarios: Named sets of overrides applied to models and datasets.
    version: 1

defaults:
  num_seeds: 3
  eval_upto: optimize
  deterministic: true

experiments:
  detection:
    models:
      - name: yolox_s
        priority: core
        recipe: detection/yolox_s.yaml
    datasets:
      - pothole_tiny
    scenarios:
      - name: lr_high
        overrides:
          model.init_args.optimizer.init_args.lr: 0.002
    criteria:
      accuracy_metric: mAP
      thresholds:
        "training:val/{metric}": { compare: ">=", margin: 0.10 }

  10. Understand Geti training tasks and types

    develop

    Models in getitune are organized by task types defined in getitune.types.TaskType. When selecting a model, ensure the task matches your dataset annotations.

    Supported Task Types:

    • Classification: MULTI_CLASS_CLS, MULTI_LABEL_CLS, H_LABEL_CLS
    • Detection: DETECTION, ROTATED_DETECTION, KEYPOINT_DETECTION
    • Segmentation: INSTANCE_SEGMENTATION, SEMANTIC_SEGMENTATION

    Special Recipe Features:

    • Tiling: Recipes ending in _tile enable a tiling pipeline designed for processing large images.
    • OpenVINO Inference: Each task directory contains an openvino_model.yaml recipe used for running pre-exported OpenVINO IR models.

  11. Model and Dataset Management

    develop

    Geti™ provides tools to manage the lifecycle of your computer vision assets:

    • Version Tracking: Track evolution of datasets and models, link models to specific dataset revisions, and view training hyperparameters.
    • Fine-tuning: Ability to fine-tune from any previous version.
    • Data Formats: Import and export in COCO, Pascal VOC, YOLO, and a Geti-optimized native format.
    • Label Filtering: Supports selective inclusion or exclusion of labels during import and export processes.

  12. Manage benchmark datasets via preparation scripts

    develop

    Benchmark datasets are managed through in-repo preparation scripts located in scripts/benchmark_datasets/. These scripts handle downloading, transforming, and placing data locally, which avoids dependency on a single hosting service.

    For specific use cases, the system supports:

    • local_path: For datasets that are fully externally managed (no script execution).
    • raw_dir: To skip the network fetch portion of a script and use an existing directory.