Spektral

repository·master·Indexed 25 days ago

https://github.com/danielegrattarola/spektral

A Python library for graph deep learning based on the Keras API and TensorFlow 2. Spektral provides a framework for implementing Graph Neural Network (GNN) architectures, including convolutional layers (GCN, GraphSAGE, GAT, GIN), pooling layers (DiffPool, Top-K, SAG), and utilities for graph manipulation. It supports various data modes (Single, Disjoint, Batch, Mixed) and includes standardized data containers for graphs and datasets.

Tokens
6.7K
Snippets
15
Records
29
Agent score
81%

What's inside spektral

  1. Overview of Spektral features and capabilities

    master

    Spektral is a Python library for graph deep learning built on the Keras API and TensorFlow 2. It provides a framework for creating Graph Neural Networks (GNNs) for tasks such as social network user classification, molecular property prediction, graph generation (GANs), node clustering, and link prediction.

    Key components include:

    Convolutional Layers

    Spektral implements several popular graph convolution operations, including:

    • Graph Convolutional Networks (GCN)
    • Chebyshev convolutions
    • GraphSAGE
    • ARMA convolutions
    • Edge-Conditioned Convolutions (ECC)
    • Graph attention networks (GAT)
    • Approximated Personalized Propagation of Neural Predictions (APPNP)
    • Graph Isomorphism Networks (GIN)
    • Diffusional Convolutions

    Pooling Layers

    For graph pooling tasks, Spektral provides:

    • MinCut pooling
    • DiffPool
    • Top-K pooling
    • Self-Attention Graph (SAG) pooling
    • Global pooling
    • Global gated attention pooling
    • SortPool

    Utilities

    The library includes utilities for representing, manipulating, and transforming graphs.

  2. Understand Spektral data modes

    master

    Spektral uses four distinct data modes to represent one or more graphs by grouping their x (node features), a (adjacency matrix), and e (edge features) into single tensors X, A, and E. Choosing the right mode depends on whether you are working with a single graph, a collection of graphs of different sizes, or multiple signals on a single graph.

    ModeA.shapeX.shapeE.shape
    Single[nodes, nodes][nodes, n_feat][edges, e_feat]
    Disjoint[nodes, nodes][nodes, n_feat][edges, e_feat]
    Batch[batch, nodes, nodes][batch, nodes, n_feat][batch, nodes, nodes, e_feat]
    Mixed[nodes, nodes][batch, nodes, n_feat][batch, edges, e_feat]

    Note: batch is the batch size, nodes is the number of nodes, edges is the number of edges, n_feat is the number of node features, and e_feat is the number of edge features.

  3. Use Disjoint mode for batches of graphs

    master

    Disjoint mode represents a batch of graphs as a single large graph via their

    from spektral.datasets import TUDataset
    from spektral.data import DisjointLoader
    
    dataset = TUDataset('PROTEINS')
    loader = DisjointLoader(dataset, batch_size=3)
    
    # Inspecting a batch
    batch = loader.__next__()
    inputs, target = batch
    x, a, i = inputs
    # x.shape will be [cumulative_nodes, n_feat]
    # a.shape will be [cumulative_nodes, cumulative_nodes]
    # i.shape will be [cumulative_nodes] (identifies which node belongs to which graph)
  4. Representing graphs with `spektral.data.Graph`

    master

    In Spektral, graphs are represented using spektral.data.Graph objects. A graph is defined by four main attributes:

    • a: The adjacency matrix. Can be a dense np.array or a Scipy sparse matrix of shape [n_nodes, n_nodes]. Sparse matrices are generally preferred for memory efficiency.
    • x: The node features. Represented as a dense np.array of shape [n_nodes, n_node_features].
    • e: The edge features. Represented as a dense np.array of shape [n_edges, n_edge_features].
      • Important: Spektral assumes edge features are sorted in row-major ordering (sorted by row, then by column). Use spektral.utils.sparse.reorder to ensure your edge features match the order of the edges in the adjacency matrix.
    • y: The labels. Can be graph-level (scalars or 1D arrays of shape [n_labels, ]) or node-level (1D arrays of shape [n_nodes, ] or 2D arrays of shape [n_nodes, n_labels]).
  5. How the MessagePassing interface works

    master

    The MessagePassing class is a flexible base class for creating Graph Neural Network (GNN) layers. It operates on a scheme where node features are updated based on information from their neighbors. The core logic follows this mathematical pattern:

    x_out[i] = update(x[i], aggregate([message(x[j]) for j in neighbours(i)]))

    To define a custom layer, you override three main methods:

    1. message(self, ...): Computes a transformation of the neighbors of each node.
    2. aggregate(self, messages): Aggregates messages in an order-independent way (e.g., sum, mean, max).
    3. update(self, embeddings): Takes the aggregated messages and decides how to transform the target node's features.

    To execute this scheme, you call the propagate method, which orchestrates the flow between these three functions.

  6. Use Single mode for single graph tasks

    master

    Single mode is used when you have only one graph (e.g., node classification on a single citation network). In this mode, the matrices represent the single graph's structure and features.

    • A: [nodes, nodes]
    • X: [nodes, n_feat]
    • E: [edges, e_feat] (sorted in row-major ordering)

    To train a model in single mode, use the SingleLoader to extract matrices into a tf.data.Dataset.

    from spektral.datasets import Cora
    from spektral.data import SingleLoader
    
    dataset = Cora()
    loader = SingleLoader(dataset)
    # Returns a tf.data.Dataset
    loader.load()
  7. What are Loaders in Spektral?

    master

    Loaders are abstractions that iterate over a graph dataset to create mini-batches, hiding the complexity of handling graphs with different numbers of nodes and edges.

    Key characteristics:

    • Each loader has a .load() method that returns a data generator compatible with Keras.
    • Loaders provide a .steps_per_epoch attribute used to tell Keras how many batches constitute one epoch.
    • Different loaders exist for different data modes (e.g., BatchLoader for graph-level tasks, SingleLoader for node-level tasks on a single large graph, or DisjointLoader for more memory-efficient batching).
  8. How to create a custom Dataset

    master

    To create a custom dataset in Spektral 1.0, you must subclass spektral.data.Dataset. The implementation relies on two primary methods:

    1. read(): This is the core method. It is called during instantiation and must return a list of spektral.data.Graph objects. This is where you load your data into memory (from files or generated on the fly).
    2. download(): This method is called by __init__ if the dataset's path does not exist on disk. Use this to define operations for saving raw data to a directory.

    Key Lifecycle Rules:

    • __init__ calls download() (if needed) and then read().
    • If you override __init__, you must call super().__init__(**kwargs) (ideally as the last line) to ensure the dataset is correctly initialized.
    • The path property defaults to ~/spektral/datasets/[ClassName].
    from spektral.data import Dataset, Graph
    
    class MyDataset(Dataset):
        def __init__(self, nodes, feats, **kwargs):
            self.nodes = nodes
            self.feats = feats
            super().__init__(**kwargs)
    
        def download(self):
            # Logic to save data to self.path
            pass
    
        def read(self):
            # Must return a list of Graph objects
            return [Graph(x=..., a=..., y=...) for _ in range(n)]
  9. Key changes and features in Spektral 1.0

    master

    The 1.0 release introduced several architectural improvements to standardize data handling and model building:

    • Standardized Data Containers: New Graph and Dataset containers standardize how data is handled (this does not impact existing models but improves data usability).
    • Simplified Batching: The new Loader class manages the complexity of creating graph batches, allowing users to focus on training logic whether using custom loops or model.fit().
    • Graph Transformations: A new transforms module allows applying common graph operations via the apply() method on datasets.
    • General Architectures: New GeneralConv and GeneralGNN classes provide flexible model building with sensible defaults based on state-of-the-art literature.
    • Expanded Datasets: Includes QM7, ModelNet10/40, and a new wrapper for OGB datasets.
    • API Change Note: The primary breaking change for existing users is the new datasets API.
  10. Node-level learning vs Graph-level learning

    master

    Spektral supports two primary learning paradigms:

    1. Graph-level learning: Predicting a label for an entire graph (e.g., classifying a molecule). This typically uses loaders like BatchLoader or DisjointLoader to process multiple graphs in a batch.
    2. Node-level learning: Predicting labels for individual nodes within a graph (e.g., classifying users in a social network). This is often performed on a single large graph using a SingleLoader and layers like GCNConv.
  11. Train a GNN using Loaders

    master

    Because graphs have variable sizes (number of nodes/edges), they cannot be reshaped into standard fixed-size tensors. Therefore, you cannot use model.fit() with a standard dataset directly. Instead, you must use a Spektral Loader to create mini-batches.

    To train a model:

    1. Instantiate and compile your model using Keras methods.
    2. Create a loader (e.g., BatchLoader) from your dataset.
    3. Call model.fit() using loader.load() as the data source. You must provide steps_per_epoch=loader.steps_per_epoch and omit the batch_size argument in fit().
    model = MyFirstGNN(32, dataset.n_labels)
    model.compile('adam', 'categorical_crossentropy')
    
    from spektral.data import BatchLoader
    loader = BatchLoader(dataset_train, batch_size=32)
    
    # Use loader.load() and specify steps_per_epoch
    model.fit(loader.load(), steps_per_epoch=loader.steps_per_epoch, epochs=10)
  12. Node-level prediction examples

    master

    Spektral provides several templates for node-level prediction tasks, such as citation network analysis. These examples demonstrate different GNN architectures and training approaches:

    • GCN (Graph Convolutional Networks): Standard implementation and a version using a custom training loop.
    • ChebConv: Using Chebyshev polynomials for graph convolutions.
    • GAT (Graph Attention Networks): Standard implementation and a version using a custom training loop.
    • ARMA: Using Autoregressive Moving Average models.
    • SimpleGCN: Demonstrating how to use custom transforms.
    • Large Datasets: Implementation for the Open Graph Benchmark (OGB) ogbn-arxiv dataset.