DOCSEARCHbyAlgolia
Submit Docs

Raster Vision

repository·master·Indexed 24 days ago

https://github.com/azavea/raster-vision

An open source Python library and framework for building computer vision models on large geospatial imagery sets, such as satellite, aerial, and oblique drone imagery. It supports chip classification, object detection, and semantic segmentation using PyTorch backends. The framework features a modular, plugin-based architecture allowing it to be used as a low-code pipeline for non-developers or as a library for developers to integrate into custom code. It includes support for local execution and scaling via AWS Batch.

Tokens
20.5K
Snippets
43
Records
106
Agent score
79%

What's inside rastervision

  1. What is the Raster Vision Pipeline?

    master

    The Raster Vision Pipeline is a framework for configuring repeatable machine learning workflows for geospatial imagery. It automates the core components of a machine learning lifecycle, including data analysis, chip creation, model training, prediction, evaluation, and deployment bundling.

    Input: A set of images and training data, optionally including Areas of Interest (AOIs) that define labeled regions. Output: A model bundle designed for deployment in various scenarios (batch processes, live servers, etc.).

  2. Overview of Raster Vision usage modes

    master

    Raster Vision can be used in two primary ways depending on your expertise and goals:

    As a Low-Code Framework

    Ideal for non-developers. You configure parameters to execute a machine learning pipeline that includes:

    • Analyzing training data
    • Creating training chips
    • Training models
    • Creating predictions
    • Evaluating models
    • Bundling model files and configuration for deployment.

    As a Python Library

    Ideal for developers who want to integrate Raster Vision into custom code. It provides utilities for:

    • Reading geo-referenced data
    • Training models
    • Making predictions
    • Writing predictions in geo-referenced formats.

  3. Manage predictions with Labels and LabelStore

    master

    Raster Vision uses two primary abstractions for handling model outputs:

    • Labels: An in-memory representation of ground truth or model predictions. Common types include ChipClassificationLabels, ObjectDetectionLabels, and SemanticSegmentationLabels.
    • LabelStore: An abstraction for writing Labels to disk or reading them back for evaluation. Examples include ChipClassificationGeoJSONStore, ObjectDetectionGeoJSONStore, and SemanticSegmentationLabelStore.

  4. How File Systems work in Raster Vision

    master

    Raster Vision uses a .FileSystem architecture that selects a file system implementation based on the URI provided. This allows the pipeline to interact with various storage backends seamlessly.

    Supported file systems include:

    • Local and HTTP: Built into the rastervision.pipeline package.
    • AWS S3: Provided via the rastervision.aws_s3 plugin.
    • GDAL VSI: Any file that can be opened using GDAL VSI is supported via the rastervision.gdal_vsi plugin.

    Some systems are read-only (like HTTP), while others support read/write operations. You can extend this functionality by adding new .FileSystem subclasses via a plugin.

  5. Understand the Raster Vision package architecture

    master

    Raster Vision is built using a modular, plugin-based architecture. The core functionality is contained in the required rastervision.pipeline package, while domain-specific logic (like geospatial deep learning or AWS integration) is provided via optional plugin packages.

    All plugin packages must reside under the rastervision native namespace package. This modularity allows users to install only the subsets of functionality they need.

    Core Packages:

    • rastervision.pipeline: The foundation for defining, configuring, and running computational pipelines in various environments (local, parallel, GPU) and file systems.
    • rastervision.aws_s3: Provides S3 read/write capabilities.
    • rastervision.aws_batch: Enables running pipelines on AWS Batch.
    • rastervision.core: Contains geospatial deep learning abstractions (chip classification, object detection, semantic segmentation) and backend/data format abstractions.
    • rastervision.pytorch_learner: Handles model building and training using torch and torchvision (can be used independently of core).
    • rastervision.pytorch_backend: Provides backends for rastervision.core pipelines using pytorch_learner for heavy lifting.

  6. Understand Raster Vision output structure

    master

    When a pipeline runs, it creates a directory structure in your output folder organized by command name. Key outputs include:

    • pipeline-config.json: A serialized JSON version of your configuration.
    • bundle/: Contains the model-bundle.zip.
    • eval/: Contains evaluation metrics (e.g., eval.json).
    • predict/: Contains prediction results (e.g., labels.tif).
    • train/: Contains training artifacts like dataloaders, model weights (last-model.pth), logs, and TensorBoard logs (tb-logs/).

  7. Use GeoDataset for PyTorch training

    master

    A GeoDataset (provided by the rastervision.pytorch_learner plugin) is a PyTorch-compatible dataset. It can be wrapped in a DataLoader and used with standard PyTorch training code, Learner, or 3rd party libraries like PyTorch Lightning.

    Common subclasses include:

    • SlidingWindowGeoDataset: ClassificationSlidingWindowGeoDataset, SemanticSegmentationSlidingWindowGeoDataset, ObjectDetectionSlidingWindowGeoDataset, RegressionSlidingWindowGeoDataset.
    • RandomWindowGeoDataset: ClassificationRandomWindowGeoDataset, SemanticSegmentationRandomWindowGeoDataset, ObjectDetectionRandomWindowGeoDataset, RegressionRandomWindowGeoDataset.

  9. Make predictions with Model Bundles

    master

    A model bundle is a zip file containing model weights and the configuration required to run predictions (including model architecture, RasterTransformer settings, and band subsets). This abstraction allows you to deploy prediction software without needing to know the specifics of the model architecture.

    Using the CLI

    Use the bundle command to generate a bundle, then use the predict CLI command to run predictions on a single scene.

    Using the API for high-throughput

    If you need to make predictions on a large number of scenes, do not use the CLI for every scene, as loading the model for every call is extremely slow. Instead, use the Predictor class programmatically. This allows you to load the model into memory once and reuse it for multiple prediction calls.

    Overriding Configuration

    By default, predictions follow the configuration of the pipeline that created the bundle. You can override specific settings (such as data.raster_source.raster_source_config.RasterSourceConfig.channel_order) using options provided to the predict CLI command.

  10. How Pipelines and PipelineConfigs work together

    master

    In Raster Vision, a Pipeline represents a sequence of commands that share a common configuration defined by a PipelineConfig.

    To implement a custom pipeline, you typically need:

    1. A Pipeline class defining the command sequence.
    2. A PipelineConfig class (often using a @register_config decorator) to define the schema.
    3. A get_config() function in a configuration file that returns an instantiated PipelineConfig object.

    When a pipeline runs, it generates a pipeline-config.json file. This file is a serialized, language-independent record of the fully-instantiated configuration. It includes type_hint fields which allow the JSON to be deserialized back into the original Python classes.

  11. Understand the core RVPipeline task types

    master

    Raster Vision provides three concrete pipeline types for deep learning on remote sensing imagery, all of which derive from RVPipeline in the rastervision.core package:

    1. Chip Classification (ChipClassification): Divides a scene into a grid of cells and classifies each cell. Best for coarse spatial predictions (e.g., identifying areas of grass) with low labeling effort.
    2. Object Detection (ObjectDetection): Predicts a bounding box and a class for each object. Provides localization and individuation but requires higher labeling effort and more training time.
    3. Semantic Segmentation (SemanticSegmentation): Predicts the class of every pixel in a scene. Provides the highest spatial precision but requires the most labeling effort and training time.

  12. Parallelize splittable commands

    master

    Certain commands (like chip and predict) are designated as split_commands in their corresponding Pipeline class. You can run these in parallel using the --split option.

    When you use --splits <number>, the workload is divided. For example, if you have 50 scenes and use --splits 5, Raster Vision will run 5 parallel commands, each processing 15 scenes.

    Runner Behavior for Splits:

    • Local Runner: Runs all splits simultaneously. Ensure the number of splits is appropriate for your available CPUs.
    • AWS Batch Runner: Uses AWS Batch array jobs to run commands in parallel. The number of simultaneous jobs is determined by your Batch Compute Environment resources.

    Downstream Dependencies: If a command (e.g., TRAIN) depends on a split command (e.g., CHIP), the downstream command will automatically wait until all individual splits are completed before starting.