HonestDiD R Package

To use HonestDiD with the fixest package's Sun and Abraham method, you must first extract the aggregated event-study coefficients and their variance-covariance matrix using a helper function (e.g., sunab_beta_vcv).

1. Run a fixest::feols regression using the sunab option.
2. Use the extraction function to get beta and sigma.
3. Pass these to HonestDiD::constructOriginalCS and HonestDiD::createSensitivityResults_relativeMagnitudes to perform the sensitivity analysis.
4. Visualize the results using HonestDiD::createSensitivityPlot_relativeMagnitudes.

# 1. Run fixest with sunab
df$year_treated <- ifelse(is.na(df$yexp2), Inf, df$yexp2)
formula_sunab <- dins ~ sunab(year_treated, year) | stfips + year
res_sunab <- fixest::feols(formula_sunab, cluster="stfips", data=df)

# 2. Extract the beta and vcv (using the provided sunab_beta_vcv function)
beta_vcv <- sunab_beta_vcv(res_sunab)

# 3. Run sensitivity analysis for relative magnitudes
kwargs <- list(betahat        = beta_vcv$beta,
               sigma          = beta_vcv$sigma,
               numPrePeriods  = sum(beta_vcv$cohorts < 0),
               numPostPeriods = sum(beta_vcv$cohorts > -1))
extra <- list(Mbarvec=seq(from = 0.5, to = 2, by = 0.5), gridPoints=100)

original_results <- do.call(HonestDiD::constructOriginalCS, kwargs)

sensitivity_results <- do.call(HonestDiD::createSensitivityResults_relativeMagnitudes,
          c(kwargs, extra))

# 4. Plot
HonestDiD::createSensitivityPlot_relativeMagnitudes(sensitivity_results, original_results)

When using regression results that include auxiliary variables (e.g., controls) alongside event-study coefficients, you must subset betahat and sigma to include only the coefficients and variance-covariance matrix corresponding to the event-study.

If you pass the full coefficient vector including controls to HonestDiD functions, the results will be incorrect because the dimensions and indices will not align with the expected event-study structure.

# Example: Subsetting coefficients from a fixest::feols object
# Assume the control is at position 8

betahat <- summary(twfe_results)$coefficients
sigma <- summary(twfe_results)$cov.scaled

# Subset to exclude the control
betahat <- betahat[-8]
sigma <- sigma[-8, -8]

# Now use the subsetted values in HonestDiD
HonestDiD::createSensitivityResults(
  betahat = betahat,
  sigma = sigma,
  numPrePeriods = 5,
  numPostPeriods = 2,
  Mvec = seq(from = 0, to = 0.05, by = 0.01)
)

By default, HonestDiD focuses on the effect for the first post-treatment period. To perform inference on other scalar parameters, use the l_vec argument.

l_vec defines a weight vector for the dynamic treatment effects $\tau_{post} = (\tau_1, \dots, \tau_{\bar{T}})'$. The parameter of interest is calculated as $\theta = l_{vec}' \tau_{post}$.

Common use cases:

• Average effect over multiple periods: Set l_vec to a vector of equal weights, e.g., c(0.5, 0.5) for two periods.
• Effect for a specific period: Use basisVector(period_index, numPostPeriods) to target a specific period (e.g., the second period after treatment).

Use createSensitivityResults_relativeMagnitudes and constructOriginalCS when working with relative magnitudes, then visualize with createSensitivityPlot_relativeMagnitudes.

delta_rm_results_avg <- HonestDiD::createSensitivityResults_relativeMagnitudes(
  betahat = betahat,
  sigma = sigma,
  numPrePeriods = 5,
  numPostPeriods = 2, 
  Mbarvec = seq(0, 2, by = 0.5),
  l_vec = c(0.5, 0.5)
)

originalResults_avg <- HonestDiD::constructOriginalCS(
  betahat = betahat,
  sigma = sigma,
  numPrePeriods = 5,
  numPostPeriods = 2,
  l_vec = c(0.5, 0.5)
)

HonestDiD::createSensitivityPlot_relativeMagnitudes(delta_rm_results_avg, originalResults_avg)

To use HonestDiD with the did package's Callaway and Sant’Anna method, use the honest_did wrapper function designed for AGGTEobj objects.

Requirements:

• The did::att_gt call must use base_period = "universal".
• The did::aggte call must be of type = "dynamic" (an event study).
• The time vector must be consecutive, with -1 acting as the reference period.

Workflow:

1. Compute ATT(g,t) using did::att_gt with base_period = "universal".
2. Aggregate results into an event study using did::aggte(..., type = "dynamic").
3. Call honest_did(es, ...) to compute the sensitivity analysis.
4. Plot using HonestDiD::createSensitivityPlot_relativeMagnitudes.

# 1. Run the CS event-study with 'universal' base-period option
cs_results <- did::att_gt(yname = "dins",
                          tname = "year",
                          idname = "stfips",
                          gname = "yexp2",
                          data = df %>% mutate(yexp2 = ifelse(is.na(yexp2), 3000, yexp2)),
                          control_group = "notyettreated",
                          base_period = "universal")

# 2. Aggregate to dynamic event study
es <- did::aggte(cs_results, type = "dynamic", min_e = -5, max_e = 5)

# 3. Run sensitivity analysis
sensitivity_results <- honest_did(es,
                                   e=0,
                                   type="relative_magnitude",
                                   Mbarvec=seq(from = 0.5, to = 2, by = 0.5))

# 4. Plot
HonestDiD::createSensitivityPlot_relativeMagnitudes(sensitivity_results$robust_ci, 
                                                     sensitivity_results$orig_ci)

HonestDiD implements tools for robust inference in difference-in-differences and event study designs based on Rambachan and Roth (2022). It addresses the concern that pre-trends might be informative about violations of parallel trends in the post-treatment period.

Key formalization methods include:

• Bounds on relative magnitudes ($\bar{M}$): Imposes that post-treatment violations of parallel trends cannot be more than a constant $\bar{M}$ larger than the maximum pre-treatment violation. For example, $\bar{M}=1$ means post-treatment violations are no larger than the worst pre-treatment violation.
• Smoothness restrictions ($M$): Imposes that the slope of the pre-trend cannot change by more than $M$ across consecutive periods. $M=0$ implies a perfectly linear counterfactual trend.
• Other restrictions: The framework supports combining these methods, as well as imposing monotonicity or sign restrictions.

These methods allow researchers to calculate robust confidence intervals and perform sensitivity analysis by reporting how much a violation of parallel trends (the "breakdown value") would be required to invalidate a conclusion.

To use HonestDiD with the sunab method in fixest, you must first extract the coefficients and the full-variance covariance matrix using the helper function sunab_beta_vcv.

Workflow:

1. Run fixest::feols using the sunab function.
2. Extract results: beta_vcv <- sunab_beta_vcv(res_sunab).
3. Map beta_vcv components to HonestDiD arguments:
   • betahat = beta_vcv$beta
   • sigma = beta_vcv$sigma
   • numPrePeriods = sum(beta_vcv$cohorts < 0)
   • numPostPeriods = sum(beta_vcv$cohorts > -1)
4. Use HonestDiD::constructOriginalCS and HonestDiD::createSensitivityResults_relativeMagnitudes as usual.

# Extract beta and vcv
beta_vcv <- sunab_beta_vcv(res_sunab)

# Prepare arguments for HonestDiD
kwargs <- list(
  betahat        = beta_vcv$beta,
  sigma          = beta_vcv$sigma,
  numPrePeriods  = sum(beta_vcv$cohorts < 0),
  numPostPeriods = sum(beta_vcv$cohorts > -1)
)

# Run analysis
original_results <- do.call(HonestDiD::constructOriginalCS, kwargs)
sensitivity_results <- do.call(HonestDiD::createSensitivityResults_relativeMagnitudes, 
                                c(kwargs, list(Mbarvec=seq(0.5, 2, by = 0.5))))

HonestDiD::createSensitivityPlot_relativeMagnitudes(sensitivity_results, original_results)

To use HonestDiD with the did package, use the honest_did helper function (provided in the repository's R/honest_did.R file) to format the output from did::aggte.

Workflow:

1. Run did::att_gt with base_period = "universal".
2. Aggregate results using did::aggte(..., type = "dynamic").
3. Pass the aggregated object to honest_did() specifying the period of interest e and the type (e.g., "relative_magnitude").
4. Use the returned robust_ci and orig_ci with createSensitivityPlot_relativeMagnitudes.

# 1. Run CS event-study
cs_results <- did::att_gt(yname = "dins", tname = "year", idname = "stfips", 
                          gname = "yexp2", data = df, 
                          control_group = "notyettreated", base_period = "universal")

# 2. Aggregate to dynamic effects
es <- did::aggte(cs_results, type = "dynamic", min_e = -5, max_e = 5)

# 3. Format for HonestDiD
sensitivity_results <- honest_did(
es, e=0, type="relative_magnitude", Mbarvec=seq(0.5, 2, by = 0.5))

# 4. Plot
HonestDiD::createSensitivityPlot_relativeMagnitudes(sensitivity_results$robust_ci, 
                                                     sensitivity_results$orig_ci)

The honest_did.AGGTEobj method provides a high-level interface for performing sensitivity analysis on results from the did package.

Parameters:

• es: An object of class AGGTEobj resulting from did::aggte (must be a dynamic event study).
• e: The event time to compute the sensitivity analysis for. Defaults to 0 (the "on impact" effect).
• type: The type of sensitivity analysis:
  • "smoothness": Allows for violations of linear trends in pre-treatment periods.
  • "relative_magnitude": Based on the relative magnitudes of deviations from parallel trends in pre-treatment periods.
• gridPoints: Number of grid points used for the underlying test inversion. Default is 100.
• ...: Additional parameters passed to createSensitivityResults or createSensitivityResults_relativeMagnitudes (e.g., Mbarvec).

Returns: A list containing:

• robust_ci: The results of the sensitivity analysis.
• orig_ci: The original confidence interval.
• type: The type of analysis performed.

To assess how sensitive your estimates are to violations of parallel trends using relative magnitudes, use createSensitivityResults_relativeMagnitudes. This method formalizes the idea that post-treatment violations cannot be much larger than the maximum pre-treatment violation (controlled by $\bar{M}$).

Parameters:

• betahat: The vector of estimated coefficients.
• sigma: The covariance matrix of the estimates.
• numPrePeriods: Number of pre-treatment coefficients.
• numPostPeriods: Number of post-treatment coefficients.
• Mbarvec: A sequence of values for $\bar{M}$ to test.

To visualize these results, first calculate the original confidence intervals using constructOriginalCS, then pass both the sensitivity results and the original results to createSensitivityPlot_relativeMagnitudes.

# 1. Calculate sensitivity results
delta_rm_results <- HonestDiD::createSensitivityResults_relativeMagnitudes(
                                    betahat = betahat, # coefficients
                                    sigma = sigma, # covariance matrix
                                    numPrePeriods = 5, # num. of pre-treatment coefs
                                    numPostPeriods = 2, # num. of post-treatment coefs
                                    Mbarvec = seq(0.5, 2, by = 0.5)
                                    )

# 2. Prepare original results for plotting
originalResults <- HonestDiD::constructOriginalCS(
                                                  betahat = betahat,
                                                  sigma = sigma,
                                                  numPrePeriods = 5,
                                                  numPostPeriods = 2)

# 3. Plot the results
HonestDiD::createSensitivityPlot_relativeMagnitudes(delta_rm_results, originalResults)