tensorQTL

tensorQTL requires three main inputs:

1. Phenotypes: Provided in BED format (.bed, .bed.gz) or .parquet. The file must have a single header line starting with #. The first four columns must be chr, start, end, and phenotype_id. Subsequent columns must be sample identifiers matching the genotype input.
   • If start == end-1, the position represents the center of the cis-window (e.g., TSS).
   • If start != end, the cis-window is defined as [start-window, end+window].
2. Covariates: A tab-delimited text file (format: covariates x samples) or a dataframe (format: samples x covariates) with row and column headers.
3. Genotypes:
   • Preferred: PLINK2 format (.pgen, .pvar, .psam). These can be generated from a VCF using plink2.
   • Alternatives: BED/BIM/FAM format or a parquet dataframe (genotypes x samples).

You can install tensorQTL via pip or by cloning the repository. For a complete environment including dependencies like Pgenlib (required for PLINK2 files), it is recommended to use the provided mamba/conda environment file.

Pip installation:

pip3 install tensorqtl

Installation from source (using mamba/conda):

git clone git@github.com:broadinstitute/tensorqtl.git
cd tensorqtl
mamba env create -f install/tensorqtl_env.yml
conda activate tensorqtl

Install latest version from GitHub:

pip install pip@git+https://github.com/broadinstitute/tensorqtl.git

To use PLINK 2 binary files (.pgen, .pvar, .psam), you must install Pgenlib. This is included in the tensorqtl_env.yml environment, but can be installed manually:

Via pip:

pip install Pgenlib

From source:

git clone git@github.com:chrchang/plink-ng.git
cd plink-ng/2.0/Python/
python3 setup.py build_ext
python3 setup.py install

When running in trans mode, you can choose between sparse and dense output formats:

1. Sparse Output (Default):

   • Returns only significant pairs above --pval_threshold.
   • Output file: {prefix}.trans_qtl_pairs.parquet (or .txt.gz if --output_text is used).
   • If --return_r2 is specified, $R^2$ values are included.

2. Dense Output:

   • Returns all phenotype-variant pairs.
   • Triggered by the --return_dense flag.
   • Produces multiple parquet files:
     • {prefix}.trans_qtl_pval.parquet (p-values)
     • {prefix}.trans_qtl_beta.parquet (beta coefficients)
     • {prefix}.trans_qtl_beta_se.parquet (beta standard errors)
     • {prefix}.trans_qtl_af.parquet (allele frequencies)

TensorQTL uses a background decorator and a BackgroundGenerator class to implement prefetching. This allows the next batch of data to be loaded into a queue in a separate thread while the current batch is being processed by the GPU/CPU, reducing I/O bottlenecks.

• The @background(max_prefetch=N) decorator wraps a generator.
• It uses a queue.Queue of size max_prefetch to buffer items.
• The BackgroundGenerator is a threading.Thread that runs the original generator and puts items into the queue.

The map_independent function implements a two-pass regression strategy to find conditionally independent variants:

1. Forward Pass: Starting from the most significant variant identified in cis_df, the algorithm iteratively adds variants to the covariate set. A variant is added if its empirical p-value (calculated via calculate_beta_approx_pval) remains below the significance threshold (signif_threshold) while accounting for previously added variants.
2. Backward Pass: The algorithm then attempts to find additional significant variants by removing one variant at a time from the current set and re-testing the others. This ensures that the set of variants is robust and captures independent signals.

If group_s is provided, the mapping is performed on groups of phenotypes simultaneously, which is more efficient for highly correlated phenotypes.

The trc function can automatically filter covariates before performing the main QTL regression.

If select_covariates=True is passed:

1. A linear regression is performed between the covariates and the log-transformed counts (for non-zero counts).
2. Covariates with an absolute t-statistic $|t| > 2$ are retained.
3. The intercept (the first column of covariates_t) is always kept.
4. The remaining selected covariates are used to calculate residuals, which are then used as the response variable in the subsequent QTL mapping step.

The --mode flag determines the type of QTL mapping performed. Each mode has specific requirements and behaviors:

• cis: Standard cis-QTL mapping. Requires phenotype BED files. Uses a sliding window (default 1Mb) around the phenotype position.
• cis_nominal: Performs nominal cis-QTL mapping (e.g., using EigenMT). Often used as a precursor to independent mapping.
• cis_independent: Performs independent cis-QTL mapping. Requires --cis_output to be provided (the output from a previous cis run).
• cis_susie: Performs fine-mapping of cis-QTLs using SuSiE. Requires --cis_output (output from cis or cis_nominal).
• trans: Performs trans-QTL mapping. Can return sparse or dense output. Supports interaction terms.
• trans_susie: Performs fine-mapping at specific loci. Requires --susie_loci (a table of phenotype_id, chr, pos).

When performing mixed-model QTL mapping, trc provides two ways to handle zero counts during log transformation to avoid mathematical errors:

1. 'offset': Uses log1p() (calculates $\log(1 + x)$).
2. 'half_min': Replaces zero counts with half of the minimum non-zero count value in the dataset, then applies a standard log() transformation.

This is controlled via the imputation parameter.

To perform full cis-eQTL mapping including permutations for FDR control, use tensorqtl.cis.map_cis. This function requires genotype data, variant metadata, phenotype data, phenotype positions, and an optional covariates DataFrame. After mapping, use tensorqtl.post.calculate_qvalues to compute q-values from the resulting association DataFrame.

# Run permutations and map cis-associations
cis_df = cis.map_cis(
    genotype_df, 
    pgr.variant_df, 
    phenotype_df, 
    phenotype_pos_df, 
    covariates_df=covariates_df, 
    seed=3042235018, 
    warn_monomorphic=False
)

# Compute q-values for FDR control
post.calculate_qvalues(cis_df, fdr=0.05, qvalue_lambda=0.85)

To run tensorQTL effectively, especially for large-scale QTL mapping, ensure your environment meets the following requirements:

• GPU: A compatible NVIDIA GPU is required for accelerated computation.
• Memory: Approximately 50GB of system memory is recommended.