DOCSEARCHbyAlgolia
Submit Docs

NeuroAI

repository·main·Indexed 43 days ago

https://github.com/facebookresearch/neuroai

A suite of specialized Python packages for NeuroAI research, including neuralbench (v0.3.1) for unified benchmarking of EEG, MEG, and fMRI models, neuralfetch for neuroimaging dataset downloading, neuralset for building AI-ready data pipelines, and neuraltrain for scalable deep learning tailored to neuroimaging modalities.

Tokens
66.6K
Snippets
214
Records
337
Agent score
55%

What's inside NeuroAI

  1. Overview of the NeuroAI Python suite

    main

    NeuroAI is a Python suite designed for Neuro-AI research, providing specialized tools for data processing, dataset retrieval, model training, and benchmarking. The suite is composed of several modular packages:

    • NeuralSet: Converts neural and stimulation data into AI-ready tensors. This is the recommended starting point for users.
    • NeuralFetch: Used to fetch curated Neuro-AI datasets.
    • NeuralTrain: Provides deep learning capabilities specifically for Neuro-AI.
    • NeuralBench: A unified benchmark for evaluating brain foundation models.
    • Exca: Provides caching and remote compute infrastructure.

  2. Overview of neuraltrain features

    main

    neuraltrain is a deep learning library tailored for neuroimaging data (M/EEG, fMRI, etc.). Key capabilities include:

    • Scalable Training: Uses PyTorch and PyTorch Lightning to support multi-GPU training with distributed backends.
    • Specialized Architectures: Includes models designed specifically for MEG, EEG, fMRI, and multi-modal neuroimaging data.
    • Neuroscience-specific logic: Provides domain-specific loss functions and evaluation metrics.

  3. Overview of the EEG/EMG Foundation Challenge 2026

    main

    The EEG/EMG Foundation Challenge 2026 is a competition focused on model generalization in neurotechnology. The challenge evaluates whether models can maintain performance across new stimuli, sessions, or users. It consists of four distinct tracks:

    • Track 1 -- EEG-to-Image (Cross-stimulus): Retrieve a viewed image from an EEG epoch against a gallery of unseen images.
    • Track 2 -- BCI decoding (Cross-session): Decode one of three cued mental commands using data from a session where the user was not calibrated.
    • Track 3 -- Sleep onset (Cross-user): Predict seconds to the first stable N2 stage from wearable EEG data on sleepers not present in the training set.
    • Track 4 -- EMG-to-Pose (Cross-user): Regress 20 hand-joint angles from wrist sEMG on new users and new movement stages.

    Important Notes:

    • Prerequisites: These tutorials assume neuralbench is already installed and configured. If you haven't run a task yet, refer to the neuralbench installation guide and the quickstart guide.
    • Competition Details: Registration, submission windows, rules, and prizes are managed by the official competition website.

  4. Overview of Neuro AI packages

    main

    The Neuro AI ecosystem provides a modular suite of tools for the NeuroAI research lifecycle:

    PackagePurpose
    NeuralSetEfficient data loading for Neuro AI
    NeuralFetchAccessing curated Neuro AI datasets
    NeuralTrainLarge-scale model training
    NeuralBenchUnified benchmarking for NeuroAI models

    The backbone of these packages is powered by exca, an execution and caching framework.

  5. Overview of neuralset

    main

    neuralset is a Python toolkit designed for building neuroimaging data pipelines. It converts raw data into AI-ready batched tensors using:

    • Event-driven processing: Utilizes typed and validated DataFrames.
    • Multi-modal extractors: Supports MEG, EEG, fMRI, EMG, iEEG, text, image, audio, and video.
    • Caching and remote compute: Integrated with exca for efficient data handling.

  6. Overview of neuralfetch capabilities

    main

    neuralfetch is a utility library designed for downloading backends and managing neuroimaging datasets.

    Key features include:

    • Download backends: Support for various platforms including OpenNeuro, DANDI, S3, Physionet, OSF, Zenodo, Figshare, Hugging Face, Synapse, and Dryad.
    • Helper utilities: Tools for study data management, BIDS (Brain Imaging Data Structure) export, and file scanning.

    neuralfetch is built on top of the neuralset package.

  7. Understand the sleep_arousal task details

    main

    The sleep_arousal task is a binary classification benchmark for detecting cortical arousal from EEG segments.

    Task Specifications:

    • Category: sleep
    • Dataset: neuralset.studies.Ghassemi2018 (PhysioNet Challenge 2018)
    • Objective: Binary classification
    • Split Strategy: Leave-subjects-out (Train/Val/Test splits are drawn at the subject level: 60/20/20)
    • Segment Length: Non-overlapping 15s segments

    Labeling Logic: A segment is labeled positive if it overlaps any scored SleepArousal event (including respiratory disturbances like apnea/hypopnea or non-respiratory arousals like RERA/PLM). It is labeled negative otherwise. Note that SleepArousal events tagged as noise (e.g., coughing, movement) are filtered out and do not trigger a positive label.

    Metrics: Due to the strong imbalance toward "no arousal" segments, the benchmark uses a weighted sampler and reports:

    • bal_acc: Balanced accuracy (the principal metric for benchmark consistency).
    • AUPRC: Area Under the Precision-Recall Curve.

  8. Available downstream tasks in NeuralBench

    main

    NeuralBench provides downstream tasks for evaluating NeuroAI models, categorized by the type of brain imaging or physiological device used. Currently supported task categories include:

    • EEG (Electroencephalography)
    • MEG (Magnetoencephalography)
    • fMRI (Functional magnetic resonance imaging)
    • EMG (Electromyography)

    Note: fNIRS (Functional near-infrared spectroscopy) tasks are planned but not yet available.

  9. Overview of the LRP classification task

    main

    The Lateralized readiness potential (LRP) classification task is a binary classification objective within the brain-computer interfacing category.

    • Objective: Binary classification (distinguishing left- vs. right-hand responses).
    • Dataset: Kappenman2020Lrp (also known as ErpCore2021_LRP).
    • Split Strategy: Leave-subjects-out.
    • Description: The task uses EEG data from 40 subjects performing a flanker task. It aims to identify the LRP, an asymmetric motor preparation component that reflects differential activation of the contralateral motor cortex before voluntary movement.

  10. Overview of the EEG age regression task

    main

    The age task is a regression objective within the eeg category of neuralbench.

    Task Details:

    • Objective: Regression
    • Dataset: neuralset.studies.Shirazi2024 (HBN)
    • Split Strategy: Leave-subjects-out (one cohort is held out for testing)
    • Data Usage: While the HBN dataset contains various tasks (resting-state, contrast change detection, etc.), this specific benchmark only utilizes resting-state data for age prediction.

    This task is designed to estimate brain age from EEG signals, which can serve as a biomarker for neurological and psychiatric disorders.

  11. Core components of NeuralBench

    main

    The NeuralBench framework is built around several key abstractions:

    • Data: Handled via neuralbench.data, including utilities like get_default_dataloaders to prepare datasets for benchmarking.
    • Experiment: The primary orchestration unit for running benchmarks.
    • BenchmarkAggregator: Used to collect and aggregate results across different runs or models.
    • BrainModule: A specialized PyTorch module (from neuralbench.pl_module) designed for NeuroAI tasks.
    • TrainerConfig: A utility from neuralbench.utils to manage training hyperparameters and settings.

  12. Understand the NeuralSet core components

    main

    A NeuralSet pipeline is composed of four modular building blocks that can be used independently or composed into a reproducible pipeline. Each component is independently cacheable and configurable:

    • Study: Acts as the interface to an external dataset. It handles downloading, iterating through timelines, and producing Events.
    • Transform: Operates on the events DataFrame to modify it (e.g., filtering, splitting, or enriching data).
    • Extractor: Responsible for converting events within a specific time window into a tensor.
    • Segmenter: Creates time-locked segments and wires Extractors to a Dataset.