netCDF4-python

To open a file for parallel access in a program running in an MPI environment using mpi4py, set parallel=True when creating the Dataset instance.

Requirements:

• netcdf-c and hdf5 must be built with MPI support.
• mpi4py must be installed.

# Example usage (requires version 1.3.1+)
# ds = nc.Dataset(filename, mode, parallel=True)

For interactive learning, the repository provides two Jupyter/IPython notebooks from the Unidata python workshop:

• reading_netcdf.ipynb: Focuses on techniques for reading data from netCDF files.
• writing_netcdf.ipynb: Focuses on techniques for creating and writing data to netCDF files.

You can install the netCDF4 package using either pip or the conda package manager.

Using pip:

pip install netCDF4

Using Conda:

conda install -c conda-forge netCDF4

To install the package from source for development purposes, follow these steps:

1. Clone the repository:

   git clone https://github.com/Unidata/netcdf4-python.git

2. Prerequisites:
   • Python 3.8 or newer.
   • numpy and Cython installed.
   • HDF5 and netcdf-4 installed.
   • The nc-config utility must be available in your Unix PATH.
3. Build and Install:

   python setup.py build
   pip install -e .

4. Run Tests: To execute the test suite, navigate to the test directory and run the test runner:

   cd test && python run_all.py

netCDF4 supports hierarchical data organization using Groups, which act like directories in a filesystem. Groups can contain variables, dimensions, attributes, and other groups.

• Create a group using ncfile.createGroup(name).
• Dimensions defined in a parent group can be accessed by variables created in a child group (recursive upward search).

ncfile = netCDF4.Dataset('data/new2.nc', 'w', format='NETCDF4')

# Create groups
grp1 = ncfile.createGroup('model_run1')
grp2 = ncfile.createGroup('model_run2')

# Create dimensions in the root group
lat_dim = ncfile.createDimension('lat', 73)

# Create variable in a group using root-level dimensions
temp1 = grp1.createVariable('temp', np.float64, ('lat',))
# Enable compression for the variable
temp1 = grp1.createVariable('temp', np.float64, ('lat',), zlib=True)

Missing Values

When a variable contains values defined as the missing_value attribute, netCDF4-python returns a masked array. This allows you to perform computations while ignoring invalid data points.

Packed Integer Data

Variables with scale_factor and add_offset attributes (often used to store high-precision data as short integers to save space) are automatically unpacked. They are returned as floating-point data with the scale and offset applied.

# Example: checking if a variable is a masked array
print(f'type={type(soilmvar[0,0,:,:])}, missing_value={soilmvar.missing_value}')

Use netCDF4.Dataset to create a new file or open an existing one.

Modes:

• mode='r': Read-only (default).
• mode='a': Append to an existing file (does not clobber data).
• mode='w': Write mode. Warning: This will clobber (overwrite) any existing data in the file unless clobber=False is used (which raises an exception if the file exists).

Formats:

• NETCDF4: Default. Supports groups, compression, and unlimited dimensions.
• NETCDF4_CLASSIC: Uses HDF5 storage but enforces the classic netCDF 3 data model for backward compatibility.
• NETCDF3_CLASSIC / NETCDF3_64BIT: Older formats.

import netCDF4
import numpy as np

# Create a new file in NETCDF4_CLASSIC format
ncfile = netCDF4.Dataset('data/new.nc', mode='w', format='NETCDF4_CLASSIC')

The netCDF4.Dataset API supports seamless remote data access via the OpenDAP protocol. You can pass a URL (e.g., from a THREDDS server) directly to the Dataset constructor. This allows you to work with remote files (including GRIB2 data served via DAP) as if they were local, generating metadata on-the-fly without downloading the entire dataset.

# Accessing a remote GRIB2 file via a THREDDS DAP URL
URL = 'https://thredds.ucar.edu/thredds/dodsC/grib/NCEP/GFS/Global_0p5deg/...' 
remote_ds = netCDF4.Dataset(URL)

# Access variables just like a local file
sfctmp = remote_ds.variables['Temperature_surface']
print(sfctmp)

The nc3tonc4 command-line tool converts netCDF 3 files into the netCDF 4 format. It supports optional features such as unpacking short integer variables to floats, applying zlib compression, using the HDF5 shuffle filter, and adding fletcher32 checksums. You can also quantize data to specific decimal precisions to improve compression ratios.

Basic Usage:

nc3tonc4 netcdf3filename netcdf4filename

The ncinfo command is a utility used to print summary information about a netCDF file. It allows you to inspect the structure of the file, including specific groups, variables, and dimensions.

ncinfo filename

The nc4tonc3 command-line utility converts netCDF 4 files (specifically those in NETCDF4_CLASSIC format) to the netCDF 3 format.

Usage Syntax:

nc4tonc3 [options] netcdf4filename netcdf3filename

nc4tonc3 input_file.nc output_file.nc

WARNING: Thread Safety

netcdf-c is not thread-safe. While netcdf4-python implements internal locking, users should expect segmentation faults if using netcdf4-python on multiple threads with free-threaded Python (e.g., Python 3.13+ with the GIL disabled).

Recommendation: Users must exercise care to only call netcdf from a single thread to avoid instability.