cutadapt Documentation

repository·main·Indexed 18 days ago

https://github.com/marcelm/cutadapt

A bioinformatics tool for adapter trimming and preprocessing of high-throughput sequencing reads. It supports single-end and paired-end reads, demultiplexing, and the removal of unwanted sequences such as adapters, primers, and poly-A tails. The tool features semiglobal alignment for adapter discovery, quality trimming, and read filtering based on expected errors or average error rates.

Tokens
32.8K
Snippets
105
Records
150
Agent score
69%

What's inside cutadapt

  1. What is Cutadapt

    main

    Cutadapt is a tool designed to find and remove unwanted sequences from high-throughput sequencing reads. This includes:

    • Adapter sequences: Common in small-RNA sequencing when the read is longer than the molecule.
    • Primers: Often found at the start of amplicon reads.
    • Poly-A tails: Often removed to clean up RNA sequencing data.

    Key features include:

    • Error-tolerant searching: Finds sequences even with mismatches.
    • Sequence modification: Can modify and filter both single-end and paired-end reads.
    • IUPAC support: Adapter sequences can contain IUPAC wildcard characters.
    • Demultiplexing: Can be used to demultiplex reads based on barcodes.
  2. Filter paired-end reads with --pair-filter

    main

    When using filtering options (like --minimum-length or --discard-trimmed) on paired-end data, Cutadapt always discards both reads of a pair to keep them in sync. The --pair-filter option determines the logic used to decide if a pair should be discarded:

    • --pair-filter=any (default): Discard the pair if at least one read fulfills the criterion.
    • --pair-filter=both: Discard the pair only if both reads fulfill the criterion.
    • --pair-filter=first: Discard the pair based only on the first read (R1), ignoring the second.

    Example behavior with --minimum-length=20:

    • any: Pair is discarded if R1 < 20 OR R2 < 20.
    • both: Pair is discarded only if R1 < 20 AND R2 < 20.
  3. How the quality trimming algorithm works

    main

    The quality trimming algorithm (similar to the one used in BWA) removes low-quality bases from the ends of a read.

    Process:

    1. Subtract the user-provided cutoff from all quality scores.
    2. Compute partial sums from the end of the sequence towards the beginning.
    3. The sequence is cut at the index where the partial sum is minimal.

    This method allows for some high-quality bases to be preserved even if they are surrounded by lower-quality bases, preventing premature trimming of useful data.

  4. Handling ambiguous sequences in indexed demultiplexing

    main

    When using an index for multiple anchored adapters, Cutadapt checks for 'ambiguous sequences'—sequences that match two or more adapters equally well.

    • Default behavior (v5.0+): Cutadapt prints a WARNING and does not trim reads with ambiguous sequences to avoid incorrect assignments.
    • With --no-index: Cutadapt will not print the warning and will assign the ambiguous read to the first matching adapter in the list.
  5. Speed up demultiplexing with adapter indexing

    main

    Cutadapt can significantly speed up demultiplexing by building an index of adapter sequences instead of checking them one by one. To enable index creation, the following conditions must be met:

    1. Anchoring: Adapters must be anchored.
      • 5' adapters: -g ^ADAPTER or -g ^file:adapters.fasta
      • 3' adapters: -a ADAPTER$ or -a file$:adapters.fasta
    2. Error Rate: The maximum error rate (-e) must be between 0 and 3.
    3. No Wildcards: No IUPAC wildcards can be used, and --match-read-wildcards must not be used.

    Optimization Tips:

    • Providing --no-indels makes index creation faster and uses less memory.
    • You can disable indexing with --no-index.
    • If an index is built, you will see Building index of X adapters ... in the output.
  6. How the Poly-A trimming algorithm works

    main

    The --poly-A trimming algorithm identifies and removes a suffix of the read consisting of a high concentration of A nucleotides.

    Algorithm Logic:

    1. It evaluates all possible suffixes of the read.
    2. It excludes any suffix where non-A nucleotides exceed 20% of the suffix length.
    3. For valid suffixes, it calculates a score: +1 for A nucleotides and -2 for non-A nucleotides.
    4. The suffix with the highest score is selected for removal. In the event of a tie, the shorter suffix is chosen.
    # Conceptual implementation of Poly-A trimming
    n = len(s)
    best_index = n
    best_score = score = errors = 0
    for i, nuc in reversed(list(enumerate(range(n)))):
        if nuc == "A":
            score += 1
        else:
            score -= 2
            errors += 1
        if score > best_score and errors <= 0.2 * (n - i):
            best_index = i
            best_score = score
    
    s = s[:best_index]
  7. How Cutadapt handles paired-end read validation

    main

    When processing paired-end reads, Cutadapt validates that the read IDs of R1 and R2 match. If they do not match, Cutadapt prints an error and aborts.

    Matching Logic:

    • Comments in the FASTQ/FASTA header are ignored.
    • If the read ID ends with 1, 2, or 3, these characters are ignored for the comparison.

    Examples:

    • Properly paired: @my_read/1 a comment and @my_read/2 another comment (the /1 and /2 are ignored).
    • Improperly paired: @my_read/1;1 and @my_read/2;1 (the ;1 is considered part of the name and is not ignored, causing a mismatch).
    @my_read/1 a comment
    @my_read/2 another comment
    
    # This is considered properly paired because /1 and /2 are ignored at the end.
    
    @my_read/1;1
    @my_read/2;1
    
    # This is considered improperly paired because ;1 is part of the name.
  8. How minimum overlap reduces random matches

    main

    To prevent erroneously trimming bases due to short random matches, Cutadapt requires a minimum number of bases to align.

    • Default: 3 bases.
    • Global setting: Use -O or --overlap <value>.
    • Adapter-specific setting: Use min_overlap=<value> within the adapter string.
    • Constraint: The minimum overlap cannot be set for anchored adapters, as they require full-length matches by definition.
    • Linked adapters: The minimum overlap is applied separately to the 5' and 3' components of a linked adapter.
    cutadapt -O 5 -a "ADAPTER;min_overlap=10"
  9. Use wildcards in adapter sequences

    main

    Cutadapt supports all IUPAC nucleotide codes (degenerate bases) in adapter sequences. For example, N matches any nucleotide. This is particularly useful for trimming adapters containing variable barcodes.

    Key behaviors:

    • Inosine: The character I is automatically replaced with N in the adapter sequence.
    • The X wildcard: Using X (e.g., -a ADAPTERX) gives it a special meaning in the matching algorithm that disallows internal adapter matches.
    • Read wildcards: By default, wildcards are only interpreted in the adapter, not the read. This prevents accidental matches against low-quality N bases in reads. To allow wildcards in reads, use the --match-read-wildcards flag.
    • Disabling wildcards: Use the -N option to disable wildcard interpretation entirely. When -N is used (and --match-read-wildcards is not), Cutadapt performs a literal ASCII comparison, allowing you to use arbitrary strings like SEQUENCE or ADAPTER for testing.
    # Example: Trimming an adapter with an embedded variable barcode
    cutadapt -a ACGTAANNNNTTAGC -o output.fastq input.fastq
  10. How the adapter alignment algorithm works

    main

    Cutadapt uses a semiglobal alignment (also known as free-shift, ends-free, or overlap alignment) to find adapters. Unlike global alignment, semiglobal alignment allows sequences to shift relative to each other, only penalizing differences within the overlapping region.

    To find the optimal alignment, Cutadapt uses a hybrid approach combining edit distance (unit costs) and alignment scores:

    1. Edit Distance (Unit Costs): Used to fill the dynamic programming matrix. Mismatches, insertions, and deletions are each counted as one error. This allows users to specify a maximum error rate using the -e flag.
    2. Alignment Scores: A second matrix is filled simultaneously using a scoring function:
      • Match: +1
      • Mismatch: -1
      • Indel: -2
    3. Selection Criteria: The algorithm identifies the best overlap by looking at the last row and column of the score matrix. It prioritizes alignments that maximize the score, ensuring more intuitive results (e.g., avoiding alignments that maximize matches by ignoring large gaps/indels).

    In cases of multiple adapter occurrences (like adapter dimers), the algorithm is designed to reliably prefer the leftmost (earlier) occurrence.

  11. The order of read modifications in Cutadapt

    main

    Cutadapt applies modifications to each read in a specific, deterministic order. Steps not requested via command-line options are skipped. The sequence is:

    1. Unconditional base removal: --cut
    2. Quality trimming: -q
    3. Adapter trimming: -a, -b, -g (and uppercase versions)
    4. Poly-A/poly-T trimming: --poly-a
    5. Read shortening: --length
    6. N-end trimming: --trim-n
    7. Length tag modification: --length-tag
    8. Read name suffix removal: --strip-suffix
    9. Name prefix/suffix addition: -x/--prefix and -y/--suffix
    10. Read renaming: --rename
    11. Zero capping: Replacing negative quality values with zero
  12. JSON report schema and structure

    main

    The Cutadapt JSON report provides a comprehensive summary of the run.

    Key Concepts

    • Completeness: All keys are included in the JSON. If a key is not applicable to the specific run, its value is set to null.
    • Single-end vs Paired-end: Single-end data is represented as "paired-end data without read 2". In these cases, values for read 1 are populated, and values for read 2 are set to null.

    Top-level Keys

    • tag: Always "Cutadapt report".
    • schema_version: A list of two integers [major, minor] representing the schema version.
    • cutadapt_version: The version of Cutadapt used.
    • python_version: The Python version used.
    • command_line_arguments: The exact arguments used for the invocation (intended for information, not for parsing).
    • cores: Number of CPU cores used.
    • input: A dictionary containing path1, path2 (null if single-end), and paired (boolean).
    • read_counts: Statistics on input, filtered, and output reads. Filtered reads are categorized by keys like too_short, too_long, too_many_n, too_many_expected_errors, casava_filtered, discard_trimmed, and discard_untrimmed.
    • basepair_counts: Statistics on total basepairs for input, output, and those removed via quality trimming or poly-A trimming.
    • adapters_read1 / adapters_read2: Lists of dictionaries containing detailed statistics for each matched adapter.
    • poly_a_trimmed_read1 / poly_a_trimmed_read2: Histograms of the lengths of poly-A/poly-T sequences removed.