Neo

repository·master·Indexed 18 days ago

https://github.com/neuralensemble/python-neo

A Python package for representing electrophysiology data with a standardized, hierarchical object model. Neo provides an interoperability layer for neurophysiology tools and supports reading a wide range of file formats, including Spike2, NeuroExplorer, AlphaOmega, Axon, Blackrock, Plexon, Tdt, and Igor Pro. It integrates with NumPy via the quantities package and organizes data into data objects (e.g., AnalogSignal, SpikeTrain), container objects (Segment, Block), and grouping/linking objects (Group, ChannelView).

Tokens
19.9K
Snippets
62
Records
97
Agent score
61%

What's inside neo

  1. What is Neo?

    master

    Neo is a Python package designed for working with electrophysiology data. It provides a common, shared object model to improve interoperability between various Python tools used for analyzing, visualizing, and generating neurophysiology data.

    Key Features:

    • Broad Format Support: Reads a wide range of neurophysiology formats including Spike2, NeuroExplorer, AlphaOmega, Axon, Blackrock, Plexon, Tdt, and Igor Pro.
    • Data Writing: Supports writing to a subset of the above formats, as well as non-proprietary formats like Kwik and HDF5.
    • Hierarchical Data Model: Adapted for intracellular, extracellular, and EEG data, including support for multi-electrode setups (e.g., tetrodes).
    • NumPy Integration: Neo objects are built on the quantities package and NumPy. They behave like normal NumPy arrays but include metadata, dimensional consistency checks, and automatic unit conversion.
  2. Overview of Neo

    master

    Neo is a Python package designed for working with electrophysiology data. It provides a common, shared object model to improve interoperability between different Python tools used for analyzing, visualizing, and generating electrophysiology data.

    Key Features:

    • Broad Format Support: Reads a wide range of neurophysiology formats including Spike2, NeuroExplorer, AlphaOmega, Axon, Blackrock, Plexon, and Tdt. It can write to a subset of these and non-proprietary formats like HDF5.
    • Hierarchical Data Model: Adapted for intracellular, extracellular, and EEG data, with support for multi-electrode configurations (e.g., tetrodes).
    • NumPy-Compatible Objects: Neo objects are built on the quantities package and NumPy. They behave like standard NumPy arrays but include metadata, dimensional consistency checks, and automatic unit conversion.
    • Lightweight Design: Neo focuses strictly on data representation and does not include built-in functions for data analysis or visualization.
  3. Overview of Neo for electrophysiology data

    master

    Neo is a Python package designed for working with electrophysiology data, including intracellular, extracellular, and EEG data. It provides a hierarchical data model aimed at improving interoperability between different Python tools for analyzing, visualizing, and generating neurophysiology data by offering a common, shared object model.

    Key features include:

    • Wide Format Support: Reading many neurophysiology formats (e.g., Spike2, NeuroExplorer, AlphaOmega, Axon, Blackrock, Plexon, Tdt, Igor Pro) and writing to open formats like NWB and NIX.
    • NumPy-like Behavior: Neo objects behave like standard NumPy arrays but include additional metadata, dimensional consistency checks, and automatic unit conversion.
    • Lightweight Design: Neo focuses strictly on the representation of data; it does not include built-in functions for data analysis or visualization to keep dependencies minimal.
  4. Metadata and annotations in Neo data objects

    master

    Neo data objects support three levels of metadata:

    1. Required Metadata: Attributes that must exist for the object to be valid. For an AnalogSignal, this includes sampling_rate.
    2. Recommended/Optional Metadata: Attributes like name that appear directly on the object.
    3. Fully Optional Metadata: Stored in two specific attributes:
      • annotations: A dictionary for general metadata.
      • array_annotations: A dictionary for metadata that is array-like (e.g., mapping channel indices to specific values). Slicing the data object automatically slices these annotations.

    When slicing a Neo object, the array_annotations are updated to match the slice.

    # Accessing metadata
    print(signal.name)
    print(signal.sampling_rate)
    print(signal.annotations)
    print(signal.array_annotations)
    
    # Slicing preserves array_annotations
    sliced_signal = signal[100:110, 1:3]
    print(sliced_signal.array_annotations)
  5. How Neo data objects and the data hierarchy work

    master

    Neo organizes neurophysiology data in a hierarchical tree structure:

    1. Block: Represents a recording session. A file can contain multiple blocks.
    2. Segment: Contained within a Block. Each segment contains data recorded during the same time interval.
    3. Data Types: Contained within a Segment. Common types include AnalogSignal (continuous time series) and others like SpikeTrain or ImageSequence (discussed in later sections).

    Example hierarchy for a single file:

    • Block (N=1)
      • Segment (N=1)
        • AnalogSignal (N=1)
          • Data (channels, sampling rate, units, etc.)
    # Example of navigating the hierarchy
    # Assuming 'data' was loaded via an IO module
    signal = data[0].segments[0].analogsignals[0]
  6. How Neo data objects work with NumPy and quantities

    master

    Neo implements a hierarchical data model where data objects are built on top of the quantities package, which in turn builds on NumPy.

    This design means that Neo objects behave like normal NumPy arrays but provide enhanced capabilities:

    1. Physical Dimensions: Support for physical units and dimensions.
    2. Metadata: Inclusion of additional metadata alongside the raw data.
    3. Consistency Checks: Automatic checks for dimensional consistency during operations.
    4. Unit Conversion: Automatic conversion of units when performing calculations or interacting with different data scales.
  7. Use Neo objects with NumPy

    master

    Neo data objects inherit from Quantity, which in turn inherits from numpy.ndarray. This allows you to use Neo objects directly in most NumPy functions (e.g., passing a SpikeTrain to numpy.histogram or an AnalogSignal to numpy.std).

    To convert a Neo object back to a standard numpy.ndarray, use the .magnitude attribute after rescaling the units to your desired target.

    # Example: converting an AnalogSignal to a numpy array
    np_sig = neo_analogsignal.rescale('mV').magnitude
    np_times = neo_analogsignal.times.rescale('s').magnitude
  8. Use the annotations dictionary for object customization

    master
    Neo objects include an annotations dictionary that allows for arbitrary extensions and customization. While you can use any name for keys within this dictionary, it is recommended to follow consistent naming conventions to improve interoperability with other tools and modules. This dictionary is the standard way to attach metadata or domain-specific information to any Neo object.
  9. How signal streams and signal buffers work

    master

    The neo.rawio API uses two concepts to manage analog signal data:

    1. signal_stream: A logical group of channels that can be read together via get_analog_signal_chunk(). Channels in a stream are guaranteed to share the same sampling rate and duration per segment (e.g., channels from the same headstage).

    2. signal_buffer: A group of channels that share the same physical data layout in a file (e.g., allowing for np.memmap or HDF5 access). A signal_buffer can contain one or more signal_streams.

    To check if a specific RawIO implementation supports the buffer API, inspect the has_buffer_description_api class attribute.

  10. Use the `neo.rawio` module for high-performance data access

    master

    When building applications that handle large-scale datasets where speed and memory consumption are critical, you can bypass the standard Neo object model in favor of the neo.rawio module.

    The neo.rawio layer provides a low-level, read-only API that uses only dictionaries and NumPy buffers. This minimizes the overhead typically associated with Neo objects and reduces memory footprint. This capability is available for a large subset of the file formats supported by Neo.

  11. Accessing SpikeTrain data by trial, neuron, or tetrode

    master

    When a Block contains SpikeTrain objects, you can access them using three different organizational strategies depending on your analysis goal:

    • By Trial (Segment): Access all spike trains within a specific segment to calculate metrics like a Peristimulus Time Histogram (PSTH) for a single trial.
    • By Neuron (Group): Access spike trains within a specific group to analyze the activity of an individual unit across all trials.
    • By Tetrode (Filtering): Use the block.filter() method to find groups associated with a specific hardware identifier (e.g., a tetrode_id stored in annotations) to blend activity from all units on that tetrode.
    # By trial: PSTH for each segment
    for seg in block.segments:
        stlist = [st - st.t_start for st in seg.spiketrains]
        # ... calculate histogram ...
    
    # By neuron: PSTH for each group (unit)
    for group in block.groups:
        stlist = [st - st.t_start for st in group.spiketrains]
        # ... calculate histogram ...
    
    # By tetrode: Blend all units on a specific tetrode
    for tetrode_id in block.annotations["tetrode_ids"]:
        stlist = []
        for unit in block.filter(objects=Group, tetrode_id=tetrode_id):
            stlist.extend([st - st.t_start for st in unit.spiketrains])
        # ... calculate histogram ...
  12. Understand relationships between Neo objects

    master

    Neo's core architecture is built on a hierarchy of objects. Most core objects inherit from the Quantity class. When navigating the object model, pay attention to the following relationship types and attribute requirements:

    • Inheritance: Objects marked with a star (*) inherit from Quantity.
    • Attributes:
      • Required: Attributes marked in red are mandatory for the object to function.
      • Recommended: Attributes marked in white are suggested for best practice or completeness.
    • Cardinality: Cyan indicators represent one-to-many relationships.

    Note that the core object diagram may not explicitly include ChannelView and RegionOfInterest in all simplified representations.