Open Data Cube Core

ODC provides services to serve and explore indexed data:

• datacube-ows: Provides WMS, WMTS, and WCS web service endpoints. It can serve raw data or rendered visualisations to web map clients (e.g., TerriaJS, Leaflet) or desktop GIS applications (e.g., QGIS). (Depends on odc-geo and datacube-core)
• datacube-explorer: A web-based front end and STAC-API endpoint for browsing and searching ODC database contents. It includes visualisation for data extents and provenance. (Depends on odc-geo and datacube-core)

The Open Data Cube ecosystem relies on several core packages for geometry handling, data loading, and discovery:

• odc-geo: Provides low-level geometry, CRS (Coordinate Reference System) handling, reprojecting utilities, and .odc xarray extensions.
• odc-loader: Tools for loading data from local or cloud storage into xarray objects. (Depends on odc-geo)
• odc-stac: Supports integrated data discovery from STAC API endpoints and loading data into xarray objects. (Depends on odc-geo and odc-loader)
• datacube (aka datacube-core): Supports integrated data discovery from a locally-maintained ODC database and loading data into xarray objects. It also handles administrative collection-management activities. (Depends on odc-geo and odc-loader)

Utilities for managing and preparing data for the ODC ecosystem:

• odc-apps-dc-tools: A collection of command line tools for indexing large data collections from local file systems or S3 buckets into an ODC database. These tools require metadata to be available in either eo3 or STAC format. (Depends on odc-geo and datacube-core)
• eodatasets3: Tools for generating or converting eo3 format metadata and repackaging data in Cloud Optimized GeoTIFF (COG) format. (Depends on odc-geo and datacube-core)

Tools designed for large-scale analysis and derivative product generation:

• odc-algo: A Python library providing parallelisable EO (Earth Observation) processing and analysis tools and methods. (Depends on odc-geo and datacube-core)
• odc-stats (aka Statistician): Supports cloud-scalable generation of statistical summary products. (Depends on odc-geo, datacube-core, and odc-algo)
• datacube-alchemist: Supports cloud-scalable generation of derivative products. It can be used with AWS Simple Queue Service (SQS) to process large numbers of datasets and produce fully packaged datasets. (Depends on odc-geo, datacube-core, odc-algo, eodatasets3, and odc-apps-dc-tools)

Open Data Cube (ODC) is a software collection designed to manage large-scale Earth observation data stored on file systems or cloud platforms. It provides a Python-based API for high-performance querying and data access, enabling scientists to perform exploratory data analysis and continental-scale processing. A key feature is the tracking of data provenance to support quality control and updates.

The system is built around the datacube-core library, with various additional tools that can be installed to provide web services (like WMS/WCS) or metadata exploration capabilities.

The datacube.utils.dask module provides a set of utility functions to facilitate parallel processing and distributed computing using Dask. These utilities are designed to help manage Dask clusters, map operations across partitions, and handle data persistence (to local files or S3) within a Dask workflow.

Available utilities include:

• start_local_dask: Initializes a local Dask client.
• partition_map: Maps a function across Dask partitions.
• pmap: A parallel map implementation.
• compute_tasks: Orchestrates the computation of a set of tasks.
• save_blob_to_file: Persists a data blob to a local filesystem.
• save_blob_to_s3: Persists a data blob to an Amazon S3 bucket.

The datacube.utils.geometry module provides several classes for handling spatial data, coordinate systems, and bounding boxes. These classes are used to define spatial extents, manage projections, and perform geometric operations within the Open Data Cube ecosystem.

Key classes include:

• Coordinate: Represents a single point in space.
• BoundingBox: Defines a rectangular area using min/max coordinates.
• CRS: Handles Coordinate Reference Systems.
• Geometry: A general abstraction for spatial shapes.
• GeoBox: A specialized geometric container.
• GeoboxTiles: Used for managing tiling operations with GeoBox.
• GridSpec: Defines the structure of a grid for spatial sampling or analysis.

A product definition in Open Data Cube specifies the required structure of a dataset, provides metadata for automatic dataset matching, and includes load hints to assist Datacube during data ingestion.

Key components include:

• metadata: Used by Datacube to automatically match a product to a dataset.
• measurements: Defines the individual data layers (equivalent to assets in STAC), including their name, dtype, nodata values, and units.
• storage (optional): Provides load hints such as crs, tile_size, and resolution so that default parameters are known when loading data.