sourmash

sourmash is a command line tool and Python library designed for sketching collections of DNA, RNA, and amino acid k-mers. It is used for biological sequence search, comparison, and analysis.

Key capabilities include:

• Fast and accurate sequence comparisons: Uses FracMinHash sketching to support datasets of different sizes.
• Profiling: Supports taxonomic profiling and functional profiling.
• Large-scale search: Capable of petabase-scale sequence search.
• Similarity Metrics: Supports estimation of overlap, bidirectional containment, and Jaccard similarity.
• Biological Context: As of v4.4, it can convert similarity metrics to estimated Average Nucleotide Identity (ANI) values.
• Applications: Identifying samples, finding similar samples, identifying datasets with shared sequences, and building phylogenetic trees.

sourmash is a command-line tool and Python/Rust library designed for metagenome analysis and genome comparison using k-mers. It is optimized for sequences 30kb or larger, including bacterial and viral genomes.

Key Use Cases:

• Reference Identification: Identify which reference genomes to use for metagenomic read mapping.
• Genome Searching: Search large sequence databases (like Genbank) with a sequence query.
• Clustering: Cluster hundreds or thousands of genomes based on similarity.
• Taxonomic Profiling: Classify genomes or metagenomes against NCBI and GTDB taxonomies.
• Metagenome Searching: Search thousands of metagenomes using a query genome or sequence.
• SRA Querying: Query the 1 million publicly available metagenomes in the Sequence Read Archive via the Branchwater plugin.

sourmash is a command-line tool used to create FracMinHash sketches (saved as "signature files") from DNA and protein sequences. These signatures allow for quick and accurate estimation of sequence similarity and containment within large collections.

Key capabilities include:

• Sketching: Creating signatures from sequence data.
• Comparison: Comparing multiple signatures to estimate similarity.
• Visualization: Plotting results like dendrograms and similarity matrices.
• Metagenomics: Genome search in metagenomes, metagenome decomposition, and taxonomic classification.

The tool uses a subcommand syntax where all commands begin with sourmash followed by a specific subcommand (e.g., sketch, compare, plot, search).

If you are using private databases, you can provide taxonomic information by creating a taxonomy file.

Required Columns:

• ident: The genome accession (e.g., GCA_1234567.1).
• Taxonomic rank columns: One column for each rank from superkingdom down to species.
• strain: (Optional) If provided, this column will also be used.

LIN (Lineage Identifier) Support: As of v4.8, you can use LIN taxonomic information with commands that accept the --lins flag. For these commands, the taxonomy file must include a lin column containing ;-separated LINs (ideally with consistent lengths, such as all 10 or all 20 positions).

LINgroups: Some taxonomy commands also accept a lingroups file, which is a two-column file with the format: name <tab/space> lin (where lin is the LIN prefix used for taxonomic summarization).

RocksDB indexes are treated as full equivalents to any other sourmash collection. They support all standard sourmash sig collection commands, such as sourmash sig summarize and sourmash sig grep.

sourmash sig summarize gtdb-rs226-reps.k31.rocksdb

When comparing abundance-weighted signatures using sourmash search or sourmash compare, sourmash defaults to angular similarity (a distance metric based on cosine similarity). This is particularly suitable for clustering metagenomic data.

Important Limitation: Angular similarity searches cannot be performed on SBT or LCA databases. You must provide collections of signature files or zip file collections directly to the command. If you attempt to run sourmash search on an LCA or SBT with an abundance-weighted query, sourmash will issue a warning and automatically apply --ignore-abundance.

The sourmash gather algorithm identifies the best match in a database based on containment, subtracts that match from the query, and repeats the process until no more significant matches are found.

CSV Output Column Definitions:

• f_unique_to_query: The fraction of the database match that is unique with respect to the original query. The sum of this column across all rows represents the total fraction of query k-mers hit (unweighted). Use this column to avoid double-counting matches.
• f_match_orig: How much of the match is in the original query.
• f_match: How much of the match is in the remaining query after previous matches have been removed.
• f_orig_query: How much of the original query belongs to the match (equivalent to running sourmash search --containment <metagenome> <match>).

sourmash is a k-mer analysis multitool designed for searching, comparing, and analyzing genomic and metagenomic datasets. Key features include:

• FracMinHash sketching: A sketching method that enables accurate comparisons (including Average Nucleotide Identity, or ANI) between datasets of different sizes.
• sourmash gather: A combinatorial k-mer approach designed for more accurate metagenomic profiling.

When interacting with sourmash SQLite databases directly, be aware of how hashes are stored. Hashes are stored as signed 64-bit integers.

To correctly interpret these values, you must convert them to unsigned 64-bit numbers internally. In the SQLite table, negative numbers represent unsigned integers that are larger than $2^{63} - 1$.

Recommendation: While the SQLite schema is accessible for lineage and manifest querying via external scripts, it is highly recommended to perform hash value querying or searching via the sourmash Python API to avoid manual conversion errors.