DOCSEARCHbyAlgolia
Submit Docs

eurostat R Package

website·Indexed 29 days ago

https://ropengov.github.io/giscoR/index.html

An R package for downloading and processing data from the Eurostat Dissemination API. It provides tools to get Eurostat data, dictionaries, and geospatial data from GISCO, as well as utilities for time column conversions, NUTS region code harmonization, and transforming TSV files into row-column-value format.

Tokens
66.8K
Snippets
305
Records
504
Agent score
73%

What's inside eurostat

  1. Overview of giscoR API wrappers

    The giscoR package provides wrappers for two primary GISCO API families. These are intended for targeted lookups (starting from a coordinate, address, or identifier) rather than downloading complete datasets. Because these functions use live services, results may change over time, and unavailable services will return NULL.

  2. Overview of the eurostat R package

    The eurostat package provides tools to download data from the Eurostat database, including search and manipulation utilities. It supports data retrieval from the Eurostat SDMX 2.1 Dissemination API and the Eurostat API Statistics (JSON API).

  4. Overview of giscoR

    giscoR is an R package that provides a simple interface to the Eurostat GISCO geodata distribution. It allows users to access geospatial data for the European Union, member states, and subnational regions. The package returns objects compatible with the sf package. Data is available in multiple resolutions (high-resolution: 01M, 03M; lighter: 10M, 20M, 60M) and coordinate reference systems including EPSG:4326, EPSG:3035, and EPSG:3857.

  6. Overview of giscoR for Eurostat GISCO spatial data

    giscoR is an R package used to download spatial data from the Eurostat GISCO (Geographic Information System of the Commission) data distribution. It provides helper functions to retrieve various datasets, including:

    • Country boundaries
    • NUTS (Nomenclature of Territorial Units for Statistics) regions
    • Administrative units
    • Statistical units
    • Transport networks
    • Basic service locations

    Note: This package is not officially related to or endorsed by Eurostat.

  7. Replicate the old dimlst.dic dictionary using ALL_CONCEPTSCHEMES.xml

    To recreate a global list of labels similar to the old dimlst.dic using the new SDMX 2.1 metadata, you can download the ALL_CONCEPTSCHEMES.xml file and parse the <s:Concept> elements. The id attribute of the concept serves as the code, and the <c:Name> element with xml:lang='en' provides the English label.
    library(xml2)
library(dplyr)

# Download and unpack ALL_CONCEPTSCHEMES.xml from:
# https://ec.europa.eu/eurostat/api/dissemination/sdmx/2.1/conceptscheme/ESTAT/?compressed=true

xml_object <- xml2::read_xml("ALL_CONCEPTSCHEMES.xml")
concepts <- xml2::xml_find_all(xml_object, ".//s:Concept")
number <- length(concepts)

dic_df <- data.frame(
  code_name = xml2::xml_attr(concepts, "id"),
  full_name = xml2::xml_text(xml2::xml_find_all(concepts, "./c:Name[@xml:lang='en']"))
)

# Clean and format to match old .dic style
new_df_sort <- dic_df %>%
  distinct() %>%
  mutate(code_name = toupper(code_name)) %>%
  arrange(code_name)

rownames(new_df_sort) <- NULL

  8. List cached Eurostat items as a data frame

    The list_eurostat_cache_items() function parses the cache_list.json file and returns a data frame containing information about cached datasets. The resulting data frame includes three columns: dataset code, download date, and the query md5 hash.
    list_eurostat_cache_items(cache_dir = NULL)

  9. Understand the search_eurostat return value

    The search_eurostat function returns a tibble with nine columns containing metadata about the matching items. The code column is particularly important as it is used in get_eurostat() and get_eurostat_raw() to retrieve the actual data.

    Columns include:

    • title: Dataset title in the specified language.
    • code: Unique identifier for the item in the Eurostat API.
    • type: Item type (dataset, folder, or table).
    • last.update.of.data: Last update date (DD.MM.YYYY).
    • last.table.structure.change: Last structure modification date (DD.MM.YYYY).
    • data.start: Oldest value date (e.g., YYYY).
    • data.end: Most recent value date (e.g., YYYY).
    • values: Number of actual values in the dataset.
    • hierarchy: Data navigation tree hierarchy level.

  10. Understand Urban Audit dataset categories and levels

    The Urban Audit dataset defines cities at several conceptual levels:

    • Core city ("CITIES"): Uses an administrative definition.
    • Functional Urban Area/Large Urban Zone ("FUA"): Approximates the functional urban region, covering the EU, Iceland, Norway, and Switzerland.

    In the resulting data, the URAU_CATG field identifies the category:

    • "C": City
    • "F": Functional urban area service type

  11. Understand the get_eurostat_toc return value

    The function returns a tibble with nine columns providing metadata about the Eurostat catalogue:

    • title: Dataset title.
    • code: Unique identifier used in the API to retrieve datasets.
    • type: Categorized as 'dataset', 'folder', or 'table'.
    • last.update.of.data: Date of last data update (DD.MM.YYYY).
    • last.table.structure.change: Date of last structural modification (DD.MM.YYYY).
    • data.start: Oldest value date (various formats like YYYY, YYYY-MM, etc.).
    • data.end: Most recent value date (various formats).
    • values: Total number of actual values in the dataset.
    • hierarchy: The navigation tree level (represented by indentation in the source).

  12. Handle Eurostat metadata via the eurostat R package

    Starting with version 4.0.0, the eurostat R package handles the complexities of XML parsing for concept schemes internally. This allows users to maintain existing workflows and expected outputs without needing to manually manage the differences between dimlst.dic and ALL_CONCEPTSCHEMES.xml. Users relying on functions like label_eurostat_vars() can continue to use them as the package abstracts the underlying metadata source.