DOCSEARCHbyAlgolia
Submit Docs

DeepTrack2

repository·develop·Indexed 42 days ago

https://github.com/deeptrackai/deeptrack2

A modular Python library for generating, manipulating, and analyzing image data pipelines for machine learning and digital microscopy. It supports image simulation, particle tracking, and optical system modeling, including specialized modules for holography and aberrations. DeepTrack2 integrates with PyTorch for automatic differentiation and optimization, and provides tools like LodeSTAR for label-free tracking and MAGIK for graph neural network-based tracking. Version 2.0 and later do not support TensorFlow.

Tokens
25.6K
Snippets
83
Records
110
Agent score
52%

What's inside DeepTrack2

  1. Compose a dataset using the pipe operator

    develop

    DeepTrack uses the >> operator to chain components together (e.g., particle >> noise >> optics). This creates a pipeline where the output of one component becomes the input of the next. You can also use the & operator to combine a dataset with a label-generating function for training.

    # Chaining particle, noise, and optics
dataset = optics(particle >> noise) >> complex_to_float

# Combining dataset with a label function for training
training_data = dataset & (dataset >> get_label)

  2. Getting Started with DeepTrack2 tutorials

    develop

    DeepTrack2 provides a series of Jupyter notebooks to help beginners understand core features, including image simulation, data loading, and training neural networks. Key tutorials include:

    • Introduction to DeepTrack2 (DTGS101): Overview of creating images, extracting properties, and training neural networks.
    • Simulating Image Modalities (DTGS106): Simulating spherical particles with different modalities and Brownian motion.
    • Loading Image Files (DTGS111): Using sources to load image files for training.
    • Particle Tracking (DTGS121, DTGS131, DTGS141): Using CNNs and U-Nets for tracking point particles, multiple particles, or distinguishing particles by size.
    • Characterizing Aberrations (DTGS126, DTGS127): Using CNNs or the Optuna optimization framework to characterize optical aberrations.
    • Advanced Techniques: Unsupervised object detection with LodeSTAR (DTGS151), fitting using PyTorch gradients (DTGS161), and creating custom scatterers (DTGS171, DTGS172).

  4. Access Developer Tutorials for DeepTrack2

    develop

    Developers looking to contribute to or understand the internal workings of DeepTrack2 can access specialized developer notebooks:

    • Overview of Code Base: A high-level walkthrough of the repository structure and logic.
    • Style Guide: Guidelines for maintaining code quality and consistency.
    • Using Different Computational Backends: Instructions on how to work with and implement various computational backends.

  5. Install deeptrack

    develop

    To use DeepTrack2, install the package via pip. If you are working in a notebook environment like Colab or Kaggle, ensure the installation is performed in your current session.

    !pip install deeptrack

  6. Explore Advanced DeepTrack2 Module Tutorials

    develop

    DeepTrack2 provides advanced tutorials designed to demonstrate the functionality of individual modules in isolation. These tutorials help users understand the specific roles and capabilities of various sub-packages within the ecosystem.

    Key module tutorials include:

    • deeptrack.features: Feature extraction and manipulation.
    • deeptrack.properties: Handling object properties.
    • deeptrack.sequences: Working with sequences.
    • deeptrack.statistics: Statistical analysis tools.
    • deeptrack.elementwise: Element-wise operations.
    • deeptrack.sources: Data sourcing (base and folder-based).
    • deeptrack.pytorch: Integration with PyTorch for data and features.
    • deeptrack.backend: Core backend operations including units, polynomials, and Mie scattering.

  7. Explore DeepTrack2 examples and tutorials

    develop

    DeepTrack2 provides a variety of practical examples for real-world datasets, ranging from basic neural network training to advanced particle tracking and feature extraction. These examples are available as Jupyter notebooks and can often be run directly in Google Colab.

    General DeepTrack2 Examples

    • MNIST (DTEx211): Training a fully connected neural network to identify handwritten digits.
    • Single Particle Tracking (DTEx212): Tracking experimental videos of a single particle.
    • Multi-Particle Tracking (DTEx213): Detecting quantum dots in low SNR (Signal-to-Noise Ratio) images.
    • Particle Feature Extraction (DTEx214): Extracting physical properties like radius and refractive index.
    • Cell Counting (DTEx215): Counting cells in fluorescence images.
    • 3D Multi-Particle Tracking (DTEx216): Tracking multiple particles in 3D for holography.
    • GAN Image Generation (DTEx217): Using Generative Adversarial Networks to create cell images from masks.

    LodeSTAR Examples (Label-free particle tracking)

    LodeSTAR is a specific application/methodology within DeepTrack2 for label-free tracking. Examples include:

    • Particle Detection (DTEx231A, DTEx231B): Detecting particles and particles of various shapes.
    • Mass Measurement (DTEx231C, DTEx231I, DTEx231J): Measuring mass in holography, simulated particles, and cells.
    • Dataset Specific Detection (DTEx231D, DTEx231E, DTEx231F): Detecting cells in BF-C2DT-HSC, Fluo-C2DT-Huh7, and PhC-C2DT-PSC datasets.
    • Biological/Environmental Detection (DTEx231G, DTEx231H): Detecting plankton and particles in 3D holography.

    MAGIK Examples (Graph Neural Network based tracking)

    MAGIK uses graph neural networks for particle linking and trace characterization. Examples include:

    • Cell Migration (DTEx241A): Tracing migrating cells.
    • HeLa Cells (DTEx241B): Tracking HeLa cells.

  8. Install DeepTrack for Colab/Kaggle

    develop

    If you are running in a notebook environment like Google Colab or Kaggle, you must install the deeptrack package using pip.

    !pip install deeptrack

  9. Explore Advanced Tutorials for Optical Microscopy

    develop

    For users working with optical microscopy, DeepTrack2 offers specialized advanced tutorials covering:

    • deeptrack.optical.scatterers: Modeling scatterers.
    • deeptrack.optical.optics: Optical system modeling.
    • deeptrack.optical.holography: Holographic techniques.
    • deeptrack.optical.aberrations: Modeling optical aberrations.
    • deeptrack.optical.noises: Noise modeling.
    • deeptrack.optical.augmentations: Optical-specific data augmentations.
    • deeptrack.optical.math: Mathematical foundations for optical modeling.

  10. Install DeepTrack2

    develop

    DeepTrack2 requires Python 3.10 or higher. You can install it using pip to automatically handle required dependencies.

    Note on TensorFlow Compatibility: DeepTrack2 version 2.0 and subsequent versions do not support TensorFlow. If your workflow requires TensorFlow, you must install the legacy version 1.7.

    pip install deeptrack
# or
python -m pip install deeptrack

  11. Install MAGIK via PyPi

    develop

    MAGIK is a geometric deep learning approach for analyzing dynamical properties from time-lapse microscopy. To use MAGIK, you must first install the deeptrack framework.

    Prerequisites:

    • Python 3.6 or higher.
    • Note for recent Python versions: You may need to install numpy manually before installing deeptrack to avoid known issues with scikit-image.
    pip install deeptrack

  12. What is deeptrack.sources.base?

    develop

    The deeptrack.sources.base module provides utilities for manipulating data sources in DeepTrack2. It is used when data needs to be dynamically manipulated, filtered, or combined.

    Key components include:

    • Source and SourceItem: Classes representing structured data streams where SourceItem triggers callbacks upon access.
    • SourceDeepTrackNode: A subclass of DeepTrackNode that allows dynamic field access using attribute notation (e.g., source.a()) to track dependencies.
    • Manipulation Utilities: Tools like Product, Subset, Sources, and random_split() for constructing complex datasets through combination, filtering, or partitioning.