DOCSEARCHbyAlgolia
Submit Docs

naniar

repository·main·Indexed 20 days ago

https://github.com/njtierney/naniar

An R package for principled, tidy ways to summarise, visualise, and manipulate missing data. It integrates with tidyverse and ggplot2, offering tools like geom_miss_point(), gg_miss_upset(), and gg_miss_var(). The package provides numerical summaries via miss_var_summary() and miss_case_summary(), shorthand helpers like n_miss() and pct_miss(), and statistical testing for Missing Completely At Random (MCAR) data using mcar_test(). It also introduces shadow matrices and nabular format for managing missingness indicators.

Tokens
2.6K
Snippets
16
Records
16
Agent score
22%

What's inside naniar

  1. Use shadow matrices and nabular format

    main

    A shadow matrix is a tidy data structure for missing data that has the same dimensions as your original data. It consists of binary indicators where NA represents a missing value and !NA represents a present value. Variable names in the shadow matrix are suffixed with _NA (e.g., Ozone_NA).

    Nabular format is a portmanteau of NA and tabular. It is created by binding the shadow matrix to your original data using bind_shadow() or nabular(). This format is useful for visualisations where you want to split or color data by missingness.

    # Create a shadow matrix
as_shadow(airquality)

# Create a nabular data frame (original data + shadow matrix)
bind_shadow(airquality)
# OR
nabular(airquality)

# Example: Using nabular format to color a density plot by missingness
airquality %>%
  bind_shadow() %>%
  ggplot(aes(x = Temp, fill = Ozone_NA)) +
  geom_density(alpha = 0.5)

  2. Install naniar from CRAN or GitHub

    main

    You can install the stable version of naniar from CRAN using install.packages(). To install the development version from GitHub, use the remotes package.

    # Install from CRAN
install.packages("naniar")

# Install development version from GitHub
# install.packages("remotes")
remotes::install_github("njtierney/naniar")

  3. Use shadow matrices and nabular data structures

    main

    A shadow matrix is a tidy data structure with the same dimensions as your original data, consisting of binary indicators of missingness. Variable names in the shadow matrix are suffixed with _NA (e.g., Ozone_NA).

    Nabular format is a portmanteau of NA and tabular. You can convert a dataset into this format using bind_shadow() or nabular(). This format is useful for managing missingness and performing visualizations where you split data by missingness.

    # Create a shadow matrix
as_shadow(airquality)

# Convert to nabular format
bind_shadow(airquality)

# Use nabular data for ggplot2
airquality %>%
  bind_shadow() %>%
  ggplot(aes(x = Temp, fill = Ozone_NA)) +
  geom_density(alpha = 0.5)
    bind_shadow(airquality)

  4. Install naniar

    main

    You can install the stable version of naniar from CRAN or the development version from GitHub.

    From CRAN:

    install.packages("naniar")

    From GitHub (development version):

    # install.packages("remotes")
remotes::install_github("njtierney/naniar")
    install.packages("naniar")

  6. Visualize missing data points with geom_miss_point()

    main

    Standard ggplot2 geoms like geom_point() simply remove rows containing NA values and issue a warning. naniar provides geom_miss_point(), which shifts missing values to be 10% lower than the minimum value of the variable and uses a distinct color. This makes missingness pre-attentive in your plots. Because it is a ggplot2 geom, it supports standard features like facet_wrap() and theme().

    library(naniar)
library(ggplot2)

# Basic usage with geom_miss_point()
ggplot(data = airquality, aes(x = Ozone, y = Solar.R)) +
  geom_miss_point()

# Usage with faceting
p1 <- ggplot(data = airquality, aes(x = Ozone, y = Solar.R)) +
  geom_miss_point() +
  facet_wrap(~Month, ncol = 2) +
  theme(legend.position = "bottom")

p1

  7. Visualize missingness over time with gg_miss_span()

    main

    Use gg_miss_span() to visualize the number of missing values in a specific variable over a repeating span (e.g., time intervals).

    # Visualise missingness in a variable at a repeating span
gg_miss_span(pedestrian, var = hourly_counts, span_every = 1500)

  8. Visualize missingness patterns with gg_miss_upset()

    main

    Use gg_miss_upset() to create an UpSet plot, which visualizes the combinations of missingness across different cases (rows) in your dataset.

    # Plot combinations of missingness across cases
gg_miss_upset(airquality)

  9. Perform statistical tests for missingness (MCAR)

    main

    Use mcar_test() to perform Little's (1988) statistical test for Missing Completely At Random (MCAR) data.

    • Null Hypothesis: The data is MCAR.
    • Test Statistic: A chi-squared value.
    • Interpretation: A high statistic value and a low p-value suggest that the data is not missing completely at random.

    The function returns a tibble containing the statistic, df (degrees of freedom), p.value, and the number of missing.patterns identified.

    # Test if data is Missing Completely At Random
mcar_test(airquality)

  10. Get numerical summaries of missing data

    main

    naniar provides functions to summarize missingness in your datasets. These functions follow a consistent naming convention:

    • miss_var_summary(): Summarizes missingness by variable. It returns a dataframe showing the number and percentage of missing values for each column.
    • miss_case_summary(): Summarizes missingness by case (the original row order). It returns a dataframe showing the number and percentage of missing values for each row.

    Both functions return dataframes and are compatible with dplyr::group_by(), allowing you to calculate missingness summaries within specific groups (e.g., missingness per variable within different levels of a categorical column).

    # Summary by variable
miss_var_summary(airquality)

# Summary by case (row)
miss_case_summary(airquality)

# Summary by variable within groups
library(dplyr)
airquality %>%
  group_by(Month) %>%
  miss_var_summary()

  11. Calculate missingness summaries with n_miss() and pct_miss()

    main

    The package provides shorthand functions to calculate the count, proportion, and percentage of missing and complete observations:

    • n_miss(): Number of missing values.
    • n_complete(): Number of complete observations.
    • prop_miss(): Proportion of missing values.
    • prop_complete(): Proportion of complete observations.
    • pct_miss(): Percentage of missing values.
    • pct_complete(): Percentage of complete observations.
    # Summary examples
n_miss(airquality)      # Returns count
n_complete(airquality)  # Returns count
prop_miss(airquality)   # Returns proportion
prop_complete(airquality) # Returns proportion
pct_miss(airquality)    # Returns percentage
pct_complete(airquality) # Returns percentage

  12. Visualize missing data with geom_miss_point()

    main

    The geom_miss_point() ggplot2 geom allows you to visualize missing values in a scatterplot. Unlike standard geom_point(), which ignores NA values and issues a warning, geom_miss_point() shifts missing values to be 10% below the minimum value of the variable and colors them differently. This makes missingness pre-attentive. Because it is a standard ggplot2 geom, it supports faceting and other ggplot features.

    library(naniar)
library(ggplot2)

ggplot(data = airquality, aes(x = Ozone, y = Solar.R)) +
  geom_miss_point()
    library(naniar)

ggplot(data = airquality,
       aes(x = Ozone,
           y = Solar.R)) +
  geom_miss_point()