tidyfast R Package

tidyfast provides fast and efficient alternatives to certain tidyr and dplyr functions by using data.table under the hood.

Key characteristics:

• Prefixing: All functions use the dt_ prefix (e.g., dt_nest()) to enable easy autocomplete in IDEs like RStudio.
• Automatic Coercion: Any function that takes data as its first argument will automatically coerce the input to a data.table using as.data.table() if it isn't one already.
• Return Type: All functions return a data.table.
• Relationship to dtplyr: While tidyfast complements dtplyr, it does not use the lazy_dt() framework. Note that some dtplyr functionality may overlap with tidyfast.
• Dependencies: The package imports data.table and cpp11.

You can install the stable version of tidyfast from CRAN or the development version from GitHub.

From CRAN:

install.packages("tidyfast")

From GitHub (Development version):

# install.packages("remotes")
remotes::install_github("TysonStanley/tidyfast")

The goal of tidyfast is to provide fast and efficient alternatives to tidyr and dplyr functions using data.table under the hood.

Key Characteristics:

• Prefixing: All functions use the dt_ prefix (e.g., dt_nest()) to allow for easy autocomplete in IDEs like RStudio.
• Automatic Coercion: Any function that takes data as its first argument will automatically coerce it to a data.table using as.data.table() if it isn't one already.
• Return Type: All functions return a data.table.
• Grammar: It attempts to maintain tidyr syntax and grammar while translating it into high-performance data.table calls.

Note: Some functionality in tidyfast may be redundant with dtplyr. Check dtplyr for a list of functions handled within that framework.

The dt_case_when() function provides a syntax similar to dplyr::case_when() but is built on the highly efficient data.table::fifelse(). It is significantly faster and more memory-efficient than dplyr::case_when() for large datasets.

# Syntax: dt_case_when(condition1 ~ label1, condition2 ~ label2, ...)

x_cat <- dt_case_when(x < medianx ~ "low",
               x >= medianx ~ "high",
               is.na(x) ~ "unknown")

dt_separate() is a fast alternative to tidyr::separate(). It splits a single column into multiple columns based on a separator (e.g., splitting "A.B" into "A" and "B" using the period). It is built on data.table::tstrsplit().

Note: This function is currently less feature-rich than tidyr::separate() and is noted as being less tested.

The dt_separate() function is a high-performance alternative to tidyr::separate(). It is significantly faster and more memory-efficient, especially for large datasets.

Important Usage Note: Currently, the into argument must be supplied with the maximum number of possible columns that could be created by the separation. If the number of columns provided in into is less than the actual number of items produced by the split, an error will be thrown.

# Basic usage
dt_to_split <- data.table(x = paste(letters, LETTERS, sep = "."))
dt_separate(dt_to_split, x, into = c("lower", "upper"))

# Note: If separation produces 3 items but 'into' only has 2, it will error:
# dt_separate(data.table(col = "A.B.C"), col, into = c("A", "B"))

The dt_count() and dt_uncount() functions provide tidyverse-style counting operations optimized for data.table.

• dt_count(dt, ...): Wraps data.table logic to provide concise grouping and counting, similar to dplyr::count().
• dt_uncount(dt, n): Expands rows based on a count column, similar to tidyr::uncount().

# Count occurrences of a group
counted <- dt_count(dt, grp)

# Uncount (expand) based on a count column
uncounted <- dt_uncount(counted, N)

The dt_fill() function replaces NA values with values from surrounding rows (up, down, or both). It utilizes efficient C++ code from tidyr (fillUp() and fillDown()) and is faster and more memory-efficient than tidyr::fill().

Arguments:

• ...: The columns to fill.
• id: A list of variables to group by (e.g., id = list(grp)).
• .direction: The direction of filling. Options include "down", "up", or "downup" (which tries both).
• immutable: Boolean indicating whether to modify the object in place.

# Fill all columns by default
dt_fill(dt_with_nas, y, z, a, immutable = FALSE)

# Fill by group (id) using 'downup' direction
dt_fill(dt_with_nas, 
        y, z, a, 
        id = list(id), 
        .direction = "downup")