vg

repository·master·Indexed 23 days ago

A suite of tools for variation graph data structures, interchange formats, alignment, genotyping, and variant calling. vg allows for the succinct encoding of multiple genomes using nodes, edges, and paths, supporting formats such as PackedGraph (.vg), GFA (.gfa), and GBZ (.gbz). It includes specialized tools like vg giraffe for short-read mapping, vg map for k-mer based mapping, and vg mpmap for transcriptomic analysis with spliced graphs.

Tokens: 16.8K
Snippets: 26
Records: 82
Agent score: 79%

What's inside vg

Overview of vg subcommands
master
The vg toolkit is used for variation graph data structures, interchange formats, alignment, genotyping, and variant calling. Subcommands are categorized by their primary function:
Graph construction and indexing
Used to build graphs and various index types (xg, distance, GCSA, etc.).
- vg autoindex: Automatically construct a graph and indexes for specific workflows (e.g., giraffe, rpvg).
- vg construct: Manually construct a graph from a reference and variants.
- vg index: Manually build individual indexes.
- vg gbwt: Manually build and manipulate GBWTs and indexes (GBWTgraph, GBZ, r-index).
- vg minimizer: Manually build a minimizer index for mapping.
- vg haplotypes: Haplotype sample a graph (recommended for mapping with giraffe).
Read mapping
Used to align reads to a variation graph.
- vg giraffe: Fast haplotype-aware short read alignment.
- vg mpmap: Splice-aware multipath alignment of short reads.
- vg map: MEM-based read alignment.
Downstream analyses
Used for variant calling and graph-specific analysis.
- vg pack: Convert alignments to a compact coverage index (used with vg call).
- vg call: Call or genotype VCF variants (requires vg pack).
- vg rna: Construct splicing graphs and pantranscriptomes.
- vg deconstruct: Create a VCF from variation in the graph.
Working with read alignments
Tools for manipulating and converting alignment files.
- vg gamsort: Sort a GAM/GAF file or index a sorted GAM file.
- vg filter: Filter alignments by properties.
- vg surject: Project alignments from a graph onto a linear reference (gam/gaf -> bam/sam/cram).
- vg inject: Project alignments from a linear reference onto a graph (bam/sam/cram -> gam/gaf).
- vg sim: Simulate reads from a graph.
Graph and read statistics
- vg stats: Get statistics about the graph.
- vg paths: Get statistics about the paths.
- vg gbwt: Get statistics about a GBWT.
- vg filter: Get statistics about alignments (using --tsv-out).
Manipulating a graph
- vg mod: Filter, transform, and edit the graph.
- vg prune: Prune the graph for GCSA2 indexing.
- vg ids: Manipulate graph node IDs.
- vg combine: Merge graphs into a combined graph.
- vg paths: Manipulate paths in a graph.
- vg gbwt: Manipulate GBWTs and associated indexes.
- vg annotate: Annotate a graph or alignments.
Conversion between formats
- vg convert: Convert between handle graph formats and GFA, or between alignment formats.
- vg view: Convert between non-handle graph formats and alignment formats (dot, json, turtle...).
- vg surject: Project alignments from a graph onto a linear reference.
- vg inject: Project alignments from a linear reference onto a graph.
- vg paths: Extract a FASTA from a graph.
Subgraph extraction
- vg chunk: Split a graph and/or alignment into smaller chunks.
- vg find: Use an index to find nodes, edges, kmers, paths, or positions.
https://github.com/vgteam/vg/blob/master/doc/vgmanmd.desc.md
Overview of the vg tool suite
master
:code:vg is a tool suite designed for working with variation graphs, which are graph-based data structures used to represent genomic information. The suite includes capabilities for variation graph data structures, interchange formats, alignment, genotyping, and variant calling methods.
https://github.com/vgteam/vg/blob/master/doc/index.rst
Understand the Graph Alignment Format (GAF) v1.0
master
GAF is a tab-delimited, UTF-8 encoded file format used for sequence alignments to bidirected sequence graphs. It is a superset of the original GFA/PAF formats and uses SAM-style conventions for sequence names and optional fields. A GAF file consists of optional header lines followed by alignment lines. Each line is split into fields separated by TAB (\t) characters.
https://github.com/vgteam/vg/blob/master/doc/static/GAF.md
What is a variation graph in vg?
master
A variation graph in vg provides a succinct encoding of sequences from many genomes. It is composed of three primary elements:
- nodes: Labeled by sequences and IDs.
- edges: Connect two nodes via their respective ends.
- paths: Describe genomes, sequence alignments, and annotations (like gene models) as walks through nodes connected by edges.
Paths are critical because they provide coordinate systems relative to genomes encoded in the graph. This allows for stable mappings even if the graph structure changes. vg tools maintain paths as immutable during transformations, using them to project graph-relative data into reference-relative coordinate spaces.
https://github.com/vgteam/vg/blob/master/README.md

Annotate Pantograph in RDF

master

Pantograph format provides a hierarchical way to describe path information in RDF using zoom levels, components, bins, and cells. This structure is useful for multi-scale genomic representations.

Key abstractions include:

vg:ZoomLevel: Defines the scale (e.g., vg:zoomFactor).
vg:Component: A structural unit within a zoom level, ordered by vg:componentRank.
vg:Bin: A subdivision of a component, containing vg:cells.
vg:Cell: The finest granularity, containing positional information (vg:positionPercent, vg:inversionPercent) and a vg:cellRegion (using faldo:Region to define boundaries).
vg:Link: Represents non-linear connections between vg:Bins, specifying vg:arrival and vg:departure points.

<vg/zoom10> a vg:ZoomLevel ;
   vg:components <vg/zoom10/component1>, <vg/zoom10/component2> ;
   vg:zoomFactor 10 .

<vg/zoom1000/component1> a vg:Component ;
   vg:componentRank 1 ;
   vg:forwardComponentEdge <vg/zoom1000/component2> ;
   vg:bins <vg/zoom1000/component1/bin1>, <vg/zoom1000/component1/bin2> .

<vg/zoom1000/component1/bin1> a vg:Bin ;
   vg:forwardBinEdge <vg/zoom1000/component2/bin2> ;
   vg:binRank 1 ;
   vg:cells <vg/zoom1000/component1/bin1/cell1>, <vg/zoom1000/component1/bin1/cell2> .

<vg/zoom1000/component1/bin1/cell1> a vg:Cell ;
   vg:positionPercent 0.04 ;
   vg:inversionPercent 0.98 ;
   vg:cellRegion <path1/region/6-100> .

<vg/zoom1000/link1> a vg:Link ;
   vg:arrival <vg/zoom1000/component1/bin1> ;
   vg:departure <vg/zoom1000/component2/bin2> ;
   vg:forwardLinkEdge <vg/zoom1000/link2> ;
   vg:linkRank 1 ;
   vg:linkPaths <path1> <path2> ;
   vg:linkZoomLevel <vg/zoom10> .

<path1/region/6-100> a faldo:Region ;
   faldo:begin <path1/position/6>  ;
   faldo:end <path1/position/100>  .

Use bdsg::HashGraph for HandleGraph implementations
master
For most use cases involving a MutablePathMutableHandleGraph, the recommended implementation is bdsg::HashGraph from the libbdsg library.
Header Location: deps/libbdsg/bdsg/include/bdsg/hash_graph.hpp
https://github.com/vgteam/vg/blob/master/BOTS.md
Interfacing with vg via protobuf and stream patterns
master
The vg toolset is built on a common data model called the variation graph, which is defined by a protobuf schema located in vg.proto. Data objects defined in this schema can be serialized using a stream pattern implemented in stream.hpp.
Developers do not need to write C++ code within the vg repository to interface with its algorithms. Instead, you can write external algorithms that read and write the same data formats defined by the vg.proto schema and the stream.hpp serialization pattern.
https://github.com/vgteam/vg/blob/master/README.md
Supported graph formats and their use cases
master
The following are the primary graph formats supported by vg:
- PackedGraph (.vg): The native vg format. Supports all types of edits (topology and paths) but can be inefficient at large scales with many paths.
- GFA (.gfa): A standard text-based format ideal for exchanging graphs between different pangenome tools. vg can operate on uncompressed GFA files directly using an in-memory PackedGraph representation.
- GBZ (.gbz): A highly-compressed format that uses significantly less space for paths. However, it does not allow general edits to the graph.
You can check the format of any graph using vg stats -F.
https://github.com/vgteam/vg/blob/master/README.md
GAF Header Lines and Graph Relationships
master
Header lines are optional and must appear before the first alignment line. They start with a three-character tag @ followed by [A-Za-z][A-Za-z0-9].
File Headers (@HD)
Used to specify file-level metadata. The VN tag (type Z) specifies the version number (e.g., 1.0).
Reference Name (@RN)
Identifies the graph used for alignment. The second field contains the pggname (a SHA-256 hash of the canonical GFA representation). There can only be one @RN line per file.
Graph Relationships (@SG, @TL)
Used to describe relationships between different reference graphs:
- Subgraph (@SG): Contains the pggname of the subgraph as the second field and the pggname of the supergraph as the third field.
- Translation (@TL): Contains the name of the source graph as the second field and the name of the destination graph as the third field.
```
@HD	VN:Z:1.0
@RN	7f4b28c71ceb808aebd8b8e9fe85e79d0d208ee263ffe9fcdef5ade20534ceb5
@SG	7f4b28c71ceb808aebd8b8e9fe85e79d0d208ee263ffe9fcdef5ade20534ceb5	e10f3b362d8a4273059d9aea38a78bd71913418c3f3c9a2b5ea44e86de2c1181
@TL	e10f3b362d8a4273059d9aea38a78bd71913418c3f3c9a2b5ea44e86de2c1181
1f133f116e8dd98fc07a647a8954038c2bcf07a45759ba94718471fe34ed7a7c
```
https://github.com/vgteam/vg/blob/master/doc/static/GAF.md
Understand the VG RDF conceptual model
master
The VG RDF representation is built upon three core components that map to the variation graph structure:
1. Node: Corresponds directly to the Node concept in the VG protobuf serialization. It represents a biological sequence segment.
2. Path: A sequence of Steps that represent a linear biological sequence (e.g., a reference genome, a patient genome, or a specific feature like a gene).
3. Step: The individual connections that link Nodes together to form a Path.
https://github.com/vgteam/vg/blob/master/ontology/README.md

How the HandleGraph API works

master

The libhandlegraph interfaces model a bidirected sequence graph. In this model, nodes contain DNA sequences, and edges connect to either the start or the end of the involved nodes.

Core Types

handle_t: An opaque 64-bit value representing a handle.
nid_t: The node ID type.
edge_t: Defined as pair<handle_t, handle_t>.

Key HandleGraph Methods

Method	Signature	Description
`get_handle`	`get_handle(nid_t, bool is_reverse=false)`	Returns a `handle_t` for a given node ID.
`get_id`	`get_id(handle_t)`	Returns the `nid_t` for a handle.
`get_is_reverse`	`get_is_reverse(handle_t)`	Returns whether the handle is in reverse orientation.
`flip`	`flip(handle_t)`	Toggles the orientation of the handle.
`get_sequence`	`get_sequence(handle_t)`	Returns the DNA sequence in the handle's orientation.
`follow_edges`	`follow_edges(handle_t, bool go_left, iteratee)`	Iterates over neighboring handles.
`for_each_handle`	`for_each_handle(iteratee, bool parallel=false)`	Iterates over all handles in the graph.
`for_each_edge`	`for_each_edge(iteratee, bool parallel=false)`	Iterates over all edges in the graph.
`has_edge`	`has_edge(handle_t left, handle_t right)`	Checks if an edge exists between two handles.

MutableHandleGraph Methods

Use these methods to modify the graph structure:

create_handle(string seq) or create_handle(string seq, nid_t id): Creates a new handle.
create_edge(handle_t left, handle_t right): Creates an edge between two handles.
destroy_handle(handle_t): Removes a handle.
destroy_edge(handle_t, handle_t): Removes an edge.

Core graph data model components in vg
master
The vg Protobuf graph data model is built around four primary classes that define the structure of a variation graph:
- vg::Node: Represents a single node in the graph.
- vg::Edge: Represents a connection between nodes.
- vg::Path: Represents a sequence of nodes (a path through the graph).
- vg::Graph: Defines the collection of nodes, edges, and paths.
These classes form the fundamental building blocks for representing variation graph data.
https://github.com/vgteam/vg/blob/master/doc/rst/api.rst

vg

What's inside vg

Overview of vg subcommands

Graph construction and indexing

Read mapping

Downstream analyses

Working with read alignments

Graph and read statistics

Manipulating a graph

Conversion between formats

Subgraph extraction

Overview of the vg tool suite

Understand the Graph Alignment Format (GAF) v1.0

What is a variation graph in vg?

Annotate Pantograph in RDF

Use bdsg::HashGraph for HandleGraph implementations

Interfacing with vg via protobuf and stream patterns

Supported graph formats and their use cases

GAF Header Lines and Graph Relationships

File Headers (`@HD`)

Reference Name (`@RN`)

Graph Relationships (`@SG`, `@TL`)

Understand the VG RDF conceptual model

How the HandleGraph API works

Core Types

Key HandleGraph Methods

MutableHandleGraph Methods

Core graph data model components in vg

vg

What's inside vg

Graph construction and indexing

Read mapping

Downstream analyses

Working with read alignments

Graph and read statistics

Manipulating a graph

Conversion between formats

Subgraph extraction

File Headers (@HD)

Reference Name (@RN)

Graph Relationships (@SG, @TL)

Core Types

Key HandleGraph Methods

MutableHandleGraph Methods

File Headers (`@HD`)

Reference Name (`@RN`)

Graph Relationships (`@SG`, `@TL`)