earthaccess Documentation

earthaccess

earthaccess is a Python library designed to simplify the process of interacting with NASA Earth science data. It provides a high-level interface for three primary tasks:

1. Search data: Explore NASA's extensive collections using the CMR (Common Metadata Repository) API.
2. Access data: Download files to a local machine or stream them directly into Python scripts using [fsspec].
3. Authenticate: Manage NASA Earthdata Login (EDL) and Earthdata Cloud credentials automatically.

This library is open-source under the MIT license and is community-driven.

The earthaccess library is a Python tool designed to simplify the discovery and access of NASA Earth science data. It provides a high-level abstraction over NASA's Common Metadata Repository (CMR) Search API, allowing users to search for data using simple notation instead of complex low-level HTTP queries.

Key capabilities include:

• Authentication: Automatically handles authentication with NASA's OAuth2 API (Earthdata Login - EDL).
• Data Access: Provides HTTP and AWS S3 sessions compatible with xarray and other PyData libraries, enabling direct access to NASA EOSDIS datasets in the cloud.

When working with NASA Earthdata, the location of your compute resources relative to the data storage affects cost and performance.

• In-region: Your compute resources (e.g., EC2 instances, SageMaker notebooks) are running in the same AWS region as the data. NASA Earthdata is primarily hosted in the AWS us-west-2 region. Accessing data from the us-west-2 S3 region is free of charge.
• Out-of-region: Your compute resources are running locally or in a different AWS region. Note that data from NASA Earthdata's S3 buckets currently cannot be accessed from a different AWS region. Generally, out-of-region access incurs egress charges and results in slower data transfer speeds.

The object returned by earthaccess.search_data() can be used in several ways:

1. Iterate for links: Loop through the results to retrieve HTTP or S3 links for the granules.
2. Download files: Use the results to download granules directly to a local directory.
3. Stream content: Open the files and stream their content directly into data analysis libraries like xarray without saving to disk first.

The project distinguishes between Discussions and Issues to keep the workflow organized:

When to use Discussions

• Brainstorming and idea generation.
• Project feedback.
• General questions and topics.

When to use Issues

• Reporting bugs and errors.
• Tracking progress on specific tasks or projects.
• Requesting changes or improvements.

Migration Guidelines

• Migrate a Discussion to an Issue if a specific task is identified, a bug is reported, or a change/improvement is requested.
• Migrate an Issue to a Discussion if the issue is a nebulous idea needing workshopping, a general question, or is not specific/actionable.

Note: Use your judgment. If a discussion contains many topics, it may be better to open a new issue for a specific task rather than migrating the entire thread.

When using earthaccess, it is helpful to understand the following core NASA Earth Observing System Data and Information System (EOSDIS) terms:

• Collection: A set of data products or files belonging to the same release of a product, or data generated during a specific experiment or campaign.
• concept-id: A unique identifier for a collection, granule, service, or tool. You can use a concept-id to search for datasets or granules within earthaccess.
• Granule: The smallest aggregation of data, such as an individual scene or swath acquired at a specific timestep. Granules can be thought of as individual files.

NASA Earthdata is organized into a hierarchy that you should understand when searching:

1. Collections (Datasets): A collection is a logical grouping of related data (e.g., MODIS/TERRA Global Daily Snow Cover). You search for these using search_datasets.
2. Granules (Files): A collection contains many granules. Each granule is an individual data file (e.g., a single day's worth of data for a specific area). You search for these using search_data.

To find a specific dataset, use search_datasets. Once you have identified a dataset, you can find its specific files (granules) using search_data or by using the dataset's concept_id.

The project follows Semantic Versioning (SemVer) using the MAJOR.MINOR.PATCH format. Use this to anticipate the impact of upgrading:

• Major version changes (X.0.0): Breaking changes are possible. Always check the documentation before upgrading.
• Minor version changes (1.X.0): New features are added, but existing functionality should not break.
• Patch version changes (1.2.X): Contains only bugfixes.

Once data is located, earthaccess provides two primary ways to consume it:

1. Download: Save data granules directly to your local machine.
2. Stream: Open data directly in memory for processing without a full local download.