TorchSig: Signal Processing Machine Learning Toolkit

repository·main·Indexed 43 days ago

https://github.com/torchdsp/torchsig

An open-source signal processing machine learning toolkit built on PyTorch for generating, augmenting, and transforming real and complex-valued signals, specifically targeting radio frequency (RF) domains for ML research. It features a unified dataset architecture including TorchSigIterableDataset for synthetic generation and StaticTorchSigDataset for disk-based loading.

Tokens
43K
Snippets
106
Records
147
Agent score
56%

What's inside TorchSig

  1. Explore TorchSig examples and tutorials

    main

    The examples/ directory contains Jupyter Notebooks and Markdown files demonstrating various TorchSig capabilities. Key usage patterns covered include:

    • Data Ingestion: Reading custom NumPy (.npz) or SigMF files into TorchSig datasets.
    • Dataset Management: Creating, customizing, and saving/loading datasets using YAML configuration files.
    • Model Training: Training PyTorch models for modulation recognition (IQ samples) and YOLO models for energy detection (spectrograms).
    • Customization: Implementing custom file handlers for specific disk write formats and applying advanced signal transforms.
    • Reproducibility: Using random number generator (RNG) seeding to ensure reproducible datasets and dataloaders.
  2. Use Metadata Transforms to modify dataset labels and bounding boxes

    main

    Metadata Transforms in torchsig.transforms.metadata_transforms allow you to alter the metadata returned by a dataset without modifying the underlying signal data. They are used to manipulate information such as:

    • Class labels
    • Class names
    • Bounding box information

    Note that these transforms are strictly for metadata; the actual signal (audio/RF data) remains unchanged.

  3. How to implement a Signal Builder

    main

    Signal builders combine modulation functions with a subclass of torchsig.signals.builder.BaseSignalGenerator.

    Implementation Steps:

    1. Modulation Functions: Place these in focused modules under torchsig/signals/builders/. They should perform the waveform math, accept explicit parameters (sample counts, rates, etc.), and return a 1D NumPy array (ideally TorchSigComplexDataType). Avoid global random state.
    2. Generator Class: Inherit from BaseSignalGenerator.
      • Define required_metadata_fields.
      • Use set_default_class_name("name") to set a stable class name.
      • Implement generate(), which must return a torchsig.signals.signal_types.Signal containing waveform data and accurate metadata.
      • Access metadata via self["field_name"] to allow inheritance from parent datasets.

    Registration: To make a builder selectable by name in torchsig.datasets.datasets.TorchSigIterableDataset, you must update:

    • torchsig/utils/signal_building.py: Add to signal_generator_lookup_table using _add_signal_generator.
    • _family_name and family_names for new families.
    • torchsig/signals/signal_lists.py: Add to CLASS_FAMILY_DICT and extend TorchSigSignalLists if necessary.
    • Export the class/functions in the module's __all__.
    class ExampleSignalGenerator(BaseSignalGenerator):
        """Generate an example waveform."""
    
        def __init__(self, **kwargs) -> None:
            super().__init__(**kwargs)
            self.required_metadata_fields = [
                "signal_duration_in_samples_min",
                "signal_duration_in_samples_max",
            ]
            self.set_default_class_name("example")
    
        def generate(self) -> Signal:
            num_samples = self.random_generator.integers(
                self["signal_duration_in_samples_min"],
                self["signal_duration_in_samples_max"] + 1,
            )
            data = example_modulator(num_samples, rng=self.random_generator)
            return Signal(data=data, center_freq=0, bandwidth=1)
  4. Choose between TorchSigIterableDataset and StaticTorchSigDataset

    main

    TorchSig provides two primary dataset types depending on whether you are generating data on-the-fly or loading pre-saved data from disk:

    1. TorchSigIterableDataset: Used for generating synthetic data in memory. It produces data infinitely.
    2. StaticTorchSigDataset: Used for loading datasets that have already been saved to disk. Unlike the iterable version, samples in a static dataset can be accessed in any order, and previously generated samples remain accessible.

    To bridge these two, you can use torchsig.utils.writer.DatasetCreator to take a TorchSigIterableDataset and write it to disk as a StaticTorchSigDataset.

  5. Use HomogeneousHDF5Reader to read datasets

    main

    The HomogeneousHDF5Reader is designed to read datasets where all top-level arrays share a single NumPy dtype and shape. These arrays are stored as a single native (number_of_signals, *signal_shape) HDF5 dataset.

    This format is suitable for:

    • Fixed-length IQ signals
    • Wideband IQ signals
    • Spectrograms
    • Other fixed-shape non-object NumPy arrays

    Note that while top-level arrays must be homogeneous, component counts, component shapes, and component dtypes are allowed to vary.

    from torchsig.utils.file_handlers.homogeneous_hdf5 import HomogeneousHDF5Reader
    
    reader = HomogeneousHDF5Reader("/path/to/homogeneous-dataset")
    try:
        # Access signal data here
    finally:
        reader.teardown()
  6. How to implement a Signal Transform

    main

    Transforms in TorchSig operate on torchsig.signals.signal_types.Signal objects. Implementation is split into two parts to separate numerical logic from state management:

    1. Functional Layer (torchsig/transforms/functional.py): Contains pure, array-level signal processing. Functions should accept explicit values, return an array, and use an optional numpy.random.Generator for randomness instead of global state. They must validate inputs with exceptions and preserve shape and NumPy dtypes.
    2. Wrapper Layer (torchsig/transforms/transforms.py): A stateful class inheriting from torchsig.transforms.transforms.SignalTransform. This layer handles parameter sampling, validation, metadata updates, and dtype enforcement. You implement the logic inside the __apply__ method.

    Randomness in Transforms: Transforms use torchsig.utils.random.Seedable. Use self.random_generator for direct draws or self.get_distribution(value_or_range) for configurable values. Supported distribution conventions:

    • List: Uniform selection from entries.
    • Tuple: Uniform sampling between two bounds.
    • Scalar: Uniform sampling from zero to the value.
    • scaling="log10": Applied to a tuple for logarithmic sampling.
    def scale_amplitude(data: np.ndarray, gain: float) -> np.ndarray:
        """Scale complex IQ samples by a linear gain."""
        if not np.isfinite(gain) or gain < 0:
            raise ValueError("gain must be finite and nonnegative")
        return (data * gain).astype(data.dtype, copy=False)
    
    class ScaleAmplitude(SignalTransform):
        """Scale a signal by a fixed linear gain."""
    
        def __init__(self, gain: float, **kwargs) -> None:
            super().__init__(data_dtype=TorchSigComplexDataType, **kwargs)
            self.gain = gain
    
        def __apply__(self, signal: Signal) -> Signal:
            signal.data = F.scale_amplitude(signal.data, self.gain)
            return signal
  7. What are Structured Signals in TorchSig

    main

    Unlike standard signal builders (e.g., FSK, PSK, OFDM) which emit streams of random symbols, Structured Signals model explicit protocol frame layouts. They include recognizable temporal and spectral structures such as fixed sync words, training preambles, signaling headers, and pilots.

    Key Characteristics:

    • Structural Models: They provide faithful framing, preambles, modulation, pulse shaping, and per-frame symbol counts.
    • RFML Optimized: They intentionally skip bit-exact field encoding (like FEC or scrambling) and use random payload symbols. This makes them ideal for RF Machine Learning (RFML) datasets where temporal structure is more important than bit-level decodability.
    • Parameter Constraints: Structured signals may inherently break standard parameter contracts for burst lengths or bandwidths. Always verify the specific signal builder implementation and its output for your use case.
  8. Select the correct HDF5 reader for your dataset

    main

    TorchSig provides three HDF5 readers. You must use the reader that matches the specific layout used when the dataset was written; layouts are not interchangeable.

    ReaderTop-level arraysComponents and metadataRecommended use
    HDF5ReaderShapes and dtypes may vary.Preserves component and parent hierarchy using the legacy object-per-record layout.Existing legacy datasets.
    PackedHDF5ReaderShapes and dtypes may vary.Variable components and hierarchical metadata are supported.New datasets with heterogeneous top-level arrays.
    HomogeneousHDF5ReaderEvery top-level array has one shared shape and dtype.Variable components are supported; inherited metadata is flattened.Fixed-size model inputs and efficient contiguous batches.

    Note on array content: While Packed and Homogeneous files can contain narrowband IQ, wideband IQ, spectrograms, and other non-object NumPy arrays, HomogeneousHDF5Reader requires that all top-level arrays share the same shape and dtype. Component counts, shapes, and dtypes within those components may still vary.

  9. Apply Dataset Impairments to model environments

    main

    Dataset Impairments are specialized sequences of Transforms designed to emulate specific channel environments. They are applied within SignalBuilders in a strict order to represent realistic signal processing effects.

    • Level 0 (Perfect): No transforms; represents a pure computer simulation.
    • Level 1 (Cabled): Moderate impairments; represents a benchtop experiment.
    • Level 2 (Wireless): Heavy impairments; includes models of radio frequency hardware and wireless channel effects.
  10. Metadata format and validation in Packed HDF5

    main

    Packed HDF5 files use the versioned torchsig-json codec for metadata.

    Supported Metadata Types

    • JSON scalar values
    • bytes
    • Complex values
    • Tuples
    • NumPy scalars and NumPy arrays
    • Lists
    • Dictionaries with string keys

    Validation

    Before serving records, the reader validates:

    • Embedded schema
    • Required datasets
    • Physical dtypes
    • Record ranges
    • Component links and parent links
    • File completion marker

    Format Identifiers:

    • Frozen format identifier: torchsig-packed
    • Current schema version: 1.0
  11. Core TorchSig Dataset Abstractions

    main

    TorchSig's dataset system is built around several key classes that handle signal generation, augmentation, and storage:

    • TorchSigIterableDataset: A unified class for synthetic signal generation. It supports both classification (single signal) and detection (multiple signals) tasks.
      • If num_signals_max is 1, it performs classification.
      • If num_signals_max > 1, it performs detection.
      • If num_samples=None, it generates samples infinitely.
    • DatasetCreator: A utility to write a DataLoader (containing a TorchSigIterableDataset) to disk. It includes progress tracking and memory optimization.
    • StaticTorchSigDataset: A class for loading previously generated datasets from disk back into memory efficiently. It provides a unified interface for both classification and detection datasets.
  12. Understand raw vs processed datasets

    main

    When saving a TorchSigIterableDataset to disk, the resulting dataset is categorized based on whether transforms were applied during the writing process:

    • raw dataset: Created if the TorchSigIterableDataset was written with no transforms and no target transforms. When loading a raw dataset using StaticTorchSigDataset, you can define and apply new transforms and target transforms.
    • processed dataset: Created if transforms or target transforms were applied during the writing process. When loading a processed dataset using StaticTorchSigDataset, you cannot define or apply additional transforms or target transforms.