DOCSEARCHbyAlgolia
Submit Docs

New Mexico Water Data Initiative (NMWDI) Developer Documentation

website·Indexed 33 days ago

https://developer.newmexicowaterdata.org/

Developer resources for the New Mexico Water Data Initiative, featuring documentation for the AMP API, ST2 SensorThings API, and Weaver. Includes guides on data integration, uploading data to CKAN, QGIS integration, and specifications for water level data file structures and formats.

Tokens
8.1K
Snippets
19
Records
74
Agent score
55%

What's inside New Mexico Water Data Initiative

  1. Overview of the New Mexico Water Data Catalog

    The New Mexico Water Data Catalog (catalog.newmexicowaterdata.org) is built on CKAN (Comprehensive Knowledge Archive Network), an open-source data platform used to manage and publish collections of data. While searching and downloading data is open to the public, adding new data requires membership in an organization and valid login credentials.

  2. Use Data Integration Engine (DIE) for data transformation and harmonization

    The Data Integration Engine (DIE) is an open-source Python-based tool used for data integration, transformation, and harmonization from multiple sources. It can be deployed as a standalone package via PIP or used as a library within a larger application.

  3. Understand CKAN Datasets and Resources

    In CKAN, data is organized into 'datasets' (sometimes referred to as 'packages' in older versions or API calls).

    A dataset consists of:

    1. Metadata: Information about the data, such as title, publisher, date, formats, and licenses.
    2. Resources: The actual data files. A single dataset can contain multiple resources. Resources can be stored internally in CKAN or linked to an external URL. Supported formats include CSV, Excel, XML, PDF, images, and RDF.

  4. Use Weaver for component-based data visualization

    Weaver is a modern web application built with React, Vite, and PrimeReact. It is designed to access data from multiple sources and display them within a single, maintainable, and easily deployable interface.

  8. Apply Reusability (FAIR principles) to data

    Reusability is the ability of data to be used by others, particularly those without prior experience with the dataset. High-quality data description (metadata) is the primary factor that enables effective re-use. To facilitate this, data should be made available under an open license.

  9. Integrate data from multiple sources

    Data integration is the process of combining data from different sources. For successful integration, datasets must be compatible, meaning they use the same names for the same objects and the same units or coordinates. Linked data aims to make this process automatic.

  10. Perform data cleaning for better consumption

    Data cleaning involves processing a dataset to make it easier to consume. This includes: fixing inconsistencies and errors, removing non-machine-readable formatting, using standard labels for headings, ensuring appropriate representation of numbers and dates, and converting to suitable file formats.

  11. Prepare water level data files for WDI submission

    Water level measurements must be submitted as two separate .csv files: one containing location and well construction data, and one containing water level measurements. Both files must include a site_id field to allow the datasets to be joined. To maintain efficiency, all measurement data for multiple wells should be consolidated into a single measurement file.

  12. Request access to publish data in the New Mexico Water Data Catalog

    To add new data to the catalog, you must be a member of an organization with login credentials. The Water Data Team at the New Mexico Bureau of Geology and Mineral Resources manages organization pages and credentials. Contact the following for access: