pyranges0 Documentation

PyRanges is a Python library designed for efficient and intuitive manipulation of genomics data, specifically genomic intervals (e.g., genes, genomic features, or reads). It is optimized for fast querying and manipulation of genomic annotations.

Key features include:

• High speed and memory efficiency.
• Pythonic/Pandastic interface: It uses Pandas DataFrames, allowing it to integrate seamlessly with the broader Python data science stack.
• Terse syntax that supports method chaining.

PyRanges relies on a continuous integration model with specific standards:

Documentation Standards

PyRanges uses the NumPy/SciPy-style for Python docstrings. This allows Sphinx to automatically generate API documentation. All new functions must have appropriate docstrings, and existing ones must be updated if the function logic changes.

Testing Layers

• Unit tests: Fast, mandatory tests for core functionality.
• Doctest: Mandatory tests that verify code snippets within the documentation (tutorials/how-tos) produce expected results.
• Property-based tests: Time-consuming tests that generate random data to validate PyRanges results against reference bioinformatics tools. These are run by the core team during backbone edits.

You can perform 'group by then apply' operations (common in Pandas) by using apply on a PyRanges object. Since PyRanges processes each Chromosome/Strand combination independently, you can use standard Pandas groupby within the apply method to operate on specific groups (like an 'ID' column) within those chromosomes.

To get the first (5'-most) exon of each CDS group, sort the intervals in 5' -> 3' order using sort('5'), then use apply with a Pandas groupby().first() chain.

( ann.subset(lambda x:x.Feature=='CDS')
    .drop(['Parent', 'Feature'])
    .sort('5')
    .apply(lambda x:x.groupby('ID', as_index=False).first())
    )

When using subset, assign, or apply, you provide a function that is applied to each DataFrame in the PyRanges collection (where each DataFrame represents a unique Chromosome/Strand combination).

• subset(func): The function must return a boolean Series with the same number of rows as the input PyRanges. It is used as a row selector.
• assign(name, func): The function must return a Series with the same number of rows as the input PyRanges. The returned Series is assigned to the new column name.
• apply(func): Use this when your function returns a DataFrame that can be converted back into a PyRanges object (i.e., it contains Chromosome, Start, End, and Strand columns).

PyRanges objects support element-wise operations similar to pandas Series. You can create new columns by performing arithmetic on existing columns (like End and Start) to calculate properties such as interval length.

# Create a new 'Length' column by subtracting Start from End
prom_in_cds.Length = prom_in_cds.End - prom_in_cds.Start

PyRanges are collections of genomic intervals that support comparison operations (such as overlap and intersection) and other methods useful for genomic analyses.

Key characteristics:

• Metadata Support: Intervals can have an arbitrary number of metadata fields (columns).
• Pandas Integration: Data is stored in a pandas.DataFrame, making it compatible with the high-performance scientific computing ecosystem.

PyRanges objects are categorized as either Stranded or Unstranded:

• Stranded: An object where a Strand column is present and all values are either + or -.
• Unstranded: An object where the Strand column is absent or contains invalid values (e.g., .).

You can check the status of an object using the .stranded property. Many PyRanges methods require a Stranded input. If your data contains invalid strand values, use .make_stranded() to transform them to + or remove them.

# Check if the object is stranded
is_stranded = cds.stranded

# Transform invalid strand values to '+' or remove them
cds = cds.make_stranded()

PyRanges objects represent sequence intervals (genomic regions, protein domains, etc.). It is critical to note that PyRanges follows standard Python conventions for coordinates:

• 0-based indexing.
• Start is included, end is excluded (half-open intervals).

While formats like GFF and GTF use 1-based, inclusive coordinates, PyRanges automatically handles the conversion between these conventions when loading and writing files in those formats.

Be careful when using the terms merge and join, as they have different meanings depending on the object type:

1. In pandas: merge and join refer to database-style operations (joining tables based on common column values).
2. In PyRanges: merge and join refer to genomic interval manipulation based on spatial overlap.

A PyRanges object is a collection of DataFrames, where data is partitioned into separate tables for each chromosome/strand pair (e.g., one table for chr1 + strand, one for chr1 - strand, etc.).

Sorting Behavior

• PyRanges .sort(): Sorts each internal chromosome/strand table independently. Because intervals on different chromosomes are never mixed in the same table, they have no relative order to each other. When printing, PyRanges displays the tables ordered by Chromosome and Strand.
• Pandas .sort_values(): Sorts the entire dataset globally, which can mix rows from different chromosomes and strands together.

Indexing

Unlike pandas, PyRanges objects do not have a user-facing index. While the internal tables have indices, they are hidden from the user and should not be queried or relied upon for data access.