Signatory

Signatory provides differentiable computations of the signature and logsignature transforms on both CPU and GPU.

The signature transform is a universal nonlinearity used to extract information about order and area from a stream of data (such as a time series). It is useful in machine learning because any continuous function of the input stream can be approximated by a linear function of its signature. It is also robust to missing or irregularly sampled data and offers optional translation and sampling invariance.

The signatory library provides a suite of tools for working with signatures, log-signatures, and paths. The API is organized into several functional areas:

• Signatures: Core functions for creating, combining, and extracting terms from signatures (signature, Signature, signature_channels, extract_signature_term, signature_combine, multi_signature_combine).
• Log-signatures: Tools for working with log-signatures (logsignature, LogSignature, logsignature_channels, signature_to_logsignature, SignatureToLogSignature).
• Paths: Working with Path objects.
• Signature Inversion: Tools for inverting signatures (invert_signature).
• Utilities: Helper functions and types (Augment, all_words, lyndon_words, lyndon_brackets).

The signature transform is a feature transformation for streams of data (such as time series). It is analogous to the Fourier transform, but instead of extracting frequency information, it extracts information about order and area.

Key properties include:

• Universal Nonlinearity: Every continuous function of the input stream can be approximated arbitrarily well by a linear function of its signature. This makes it highly effective for machine learning tasks.
• Robustness: It is robust to missing or irregularly sampled data.
• Invariance: It supports optional translation invariance and optional sampling invariance.
• Utility: It can be used to encode physical quantities and for data compression.

When deciding between Signatory and iisignature, consider these key differences:

In contrast, iisignature is limited to the CPU and is NumPy-based. Note that while they share similar goals, they have different functional offerings; some operations available in one may not be present in the other.

By default, the signature transform is translation invariant. This means the signature of a path $x_1,
ightdots, x_n$ is identical to the signature of the path shifted by a constant $y$ ($x_1 + y, \dots, x_n + y$).

If your application requires the signature to be sensitive to absolute position (i.e., you want to detect translations), you must add a 'basepoint'. Setting basepoint=True in the signatory.signature function adds a zero vector ($0 \in \mathbb{R}^c$) to the start of the path, allowing the signature to capture the translation relative to the origin.

import torch
import signatory

# path shape: (batch_size, sequence_length, dimension)
path = torch.rand(2, 10, 5)

# Setting basepoint=True adds a 0 vector to the start of the path
# to make the signature sensitive to translations.
sig = signatory.signature(path, 3, basepoint=True)

When choosing between Signatures and Logsignatures for your data representation, consider the following trade-offs:

• Signatures: These can become very large because they are designed to linearize all possible functions of their input. They are generally faster to compute.
• Logsignatures: These are more information-dense and modestly sized. They are ideal if you need to capture high-order interactions between input channels without the massive size of a full signature. However, they are slower to compute because they are derived from the signature.

Important Performance Note: Computing a logsignature is not more memory-efficient than computing a signature, as the logsignature is computed by first calculating the signature.

The signature transform is sampling invariant, meaning the signature of a path is approximately the same regardless of how many times the underlying path is measured or how finely it is sampled.

This property makes the signature transform useful for:

• Dealing with missing data.
• Handling irregularly-sampled data.
• Processing sequences of different lengths without changing the mathematical model.

Important Note on Time: The signature transform does not use the index of an element in a sequence. If your machine learning model needs to depend on the passing of time (e.g., speed or velocity), you must explicitly include time as a dimension in your input stream of data. If you do not include time, the signature will only capture the geometric shape of the path, regardless of how fast it was traversed.

The signatory.Augment class is used to enhance the input stream before it reaches the signature transform. It is recommended for two specific tasks:

1. Applying stream-preserving neural networks: Placing these before the signature transform to increase the transform's expressive power.
2. Adding a 'time' dimension: Augmenting the input stream with an extra dimension representing time to improve the signature's representation of the path.

Choosing the right approach for signature computation depends on your data access pattern:

1. signatory.signature(path, order): Use this if you only need the signature of the entire path once. It is the simplest method but does not support efficient subinterval queries.
2. signatory.Path(path, order): Use this if you need to compute signatures for many different sub-intervals of the same path or if data arrives incrementally via .update(). Note that Path must hold the entire stream in memory.
3. Combining Signatures: If you need signatures for adjacent disjoint intervals and want to find the signature of their union, look for the combine pattern (see examples-combine). This can be more efficient than Path if you want to avoid the memory overhead of storing the whole stream.

Incremental signature calculation allows you to compute the signature of a concatenated path $XY$ using only the signature of $X$, the signature of $Y$, and the connection point between them.

In signatory.signature(Y, depth, basepoint=basepoint, initial=initial):

• initial: The signature of the preceding path segment.
• basepoint: The last point of the preceding path segment (the point where the new segment $Y$ begins).

This approach is memory-efficient because once the signature and the last point of a segment are stored, the rest of the segment's data can be discarded.

Because an infinite signature cannot fit in memory, you should truncate the signature to a finite collection of statistics and embed it within a nonlinear model like a neural network. In this setup, the signature transform acts as a pooling function that extracts information from the stream.

Best Practices:

• Manage Complexity: The number of terms in signatures grows rapidly with depth and the number of channels. Experiment to find an acceptable balance between information and computational cost.
• Use Stream-Preserving Networks: Placing small, stream-preserving neural networks before the signature transform can significantly enhance its power. You can implement this using the signatory.Augment class.
• Augment with Time: It is often beneficial to augment the input stream with an extra 'time' dimension. This can also be achieved using the signatory.Augment class.