DOCSEARCHbyAlgolia
Submit Docs

Space2Vec

repository·master·Indexed 40 days ago

https://github.com/gengchenmai/space2vec

A multi-scale representation learning framework for spatial feature distributions using grid cells. It is designed for tasks such as Point of Interest (POI) type classification and geo-aware fine-grained image classification, featuring tools for training spatio-temporal priors and extracting features from datasets like iNat2017, iNat2018, BirdSnap, and NABirds.

Tokens
5.6K
Snippets
18
Records
32
Agent score
50%

What's inside Space2Vec

  1. Overview of geo_prior/pre_process scripts

    master

    The geo_prior/pre_process directory provides a suite of scripts designed for the preprocessing stage of the spatial feature distribution pipeline. These scripts are used for:

    1. Creating data splits: Partitioning datasets for training, validation, and testing.
    2. Training image classifiers: Building models to classify image-based features.
    3. Extracting image features: Generating feature representations from images for downstream tasks.

  3. Train and evaluate geographical priors

    master

    The geo_prior package provides tools for training spatio-temporal priors and evaluating them for fine-grained image classification tasks.

    • Use train_geo_net.py to train the spatio-temporal prior on various datasets.
    • Use run_evaluation.py to evaluate different priors for image classification tasks.

  5. Setup the Point of Interest (POI) Type Classification Task

    master

    The POI Type Classification task (located in the spacegraph/ folder) is used for location and context modeling. It requires Python 2.7+ and Torch 1.0.1+.

    To set up the environment, run the provided setup script:

    python spacegraph/setup.py

    Additional required packages are listed in spacegraph/requirements.txt.

  6. Train and Evaluate Geo-Aware Image Classification models

    master

    The image classification task is implemented in Python 3. The main logic resides in geo_prior/geo_prior/.

    1. Train the location encoder model: Run train_geo_net.py.

    2. Evaluate the model: Run run_evaluation.py to evaluate the location encoder by combining it with pre-trained CNN features (e.g., using the BirdSnap† dataset).

    # Train the location encoder
python3 train_geo_net.py

# Evaluate the model
python3 run_evaluation.py

  8. Pre-process BirdSnap images using ImageMagick

    master

    To resize and convert raw BirdSnap images to JPG format with a maximum dimension of 800px, use the mogrify command. This command should be executed from within the directory containing the raw data. It will output the processed images to the ../images_sm directory.

    mogrify -geometry "800x800>" -format jpg -path ../images_sm *

  9. Train Location Modeling models for POI Type Classification

    master

    Location modeling (Sections 5.1.1 and 5.1.2 of the ICLR 2020 paper) is implemented in Python 2.7. You can train different models by executing specific bash scripts located in the spacegraph/ directory.

    Available models:

    • direct: bash Place2Vec_2_enc_dec_global_naive.sh
    • tile: bash Place2Vec_2_enc_dec_global_gridlookup.sh
    • wrap: bash Place2Vec_2_enc_dec_global_aodha.sh
    • rbf: bash Place2Vec_2_enc_dec_global_rbf.sh
    • gird: bash Place2Vec_2_enc_dec_global_grid.sh
    • theory: bash Place2Vec_2_enc_dec_global_theory.sh

    Outputs:

    • Trained models are saved to the directory specified by the --model_dir flag.
    • Evaluation logs are saved to the directory specified by the --log_dir flag.
    # Example: running the 'direct' model
bash Place2Vec_2_enc_dec_global_naive.sh

  11. Visualize spatio-temporal prior outputs with figure generation scripts

    master

    This directory provides a collection of Python scripts used to visualize the outputs of a trained spatio-temporal prior. Each script corresponds to a specific figure or supplementary visualization presented in the research paper.

    Main Paper Figures

    • Fig 1: Use plot_pair_of_classes.py to visualize pairs of classes.
    • Fig 3a: Use plot_model_on_map.py to plot the model results on a map.
    • Fig 3b: Use plot_photographer_embedding.py to visualize photographer embeddings.
    • Fig 4: Use plot_class_embedding.py to visualize class embeddings.
    • Fig 5: Use plot_photographer_embedding.py to visualize photographer embeddings.
    • Fig 6: Use plot_class_over_time.py to visualize class changes over time.

    Supplementary and Video Visualizations

    • Supplementary:
      • plot_class_improvement.py for class improvement metrics.
      • plot_metadata_stats.py for metadata statistics.
    • Video:
      • plot_video_frames.py to visualize video frames.

  12. Set up the geo_prior environment

    master

    Follow these steps to prepare the environment for training and evaluation:

    1. Data Preparation: Download required datasets and metadata from the project website and place them in the ../data/ directory. This should include features, predictions from trained image classifiers, and location metadata. Additionally, download the training and validation annotation files for each iNat dataset.
    2. Model Files: If evaluating existing trained models, ensure they are placed in the ../models/ directory.
    3. Configuration: Update the paths in paths.py to match the absolute or relative locations of your data and models on your local system.
    4. Dependencies: Install the package versions specified in requirements.txt. The project was developed using Python 3.7.