CTR-GCN: Channel-wise Topology Refinement Graph Convolution

To evaluate a model saved in a specific <work_dir>, use the --phase test flag. You must provide the configuration file from the work directory and the path to the weights file (.pt).

python main.py --config <work_dir>/config.yaml --work-dir <work_dir> --phase test --save-score True --weights <work_dir>/xxx.pt --device 0

To set up the development environment, ensure you have Python >= 3.6 and PyTorch >= 1.1.0 installed. You can install the required dependencies using the provided requirements.txt file and the torchlight package.

1. Create an Anaconda virtual environment.
2. Install dependencies via pip.
3. Install torchlight in editable mode.

pip install -r requirements.txt
pip install -e torchlight

CTR-GCN supports NTU RGB+D 60, NTU RGB+D 120, and NW-UCLA datasets.

NTU RGB+D 60 and 120

1. Request the dataset from the official source.
2. Download the skeleton-only files:
   • nturgbd_skeletons_s001_to_s017.zip (for NTU 60)
   • nturgbd_skeletons_s018_to_s032.zip (for NTU 120)
3. Extract these files into ./data/nturgbd_raw.

NW-UCLA

1. Download the dataset from Dropbox.
2. Move the all_sqe folder to ./data/NW-UCLA.

Required Directory Structure

Ensure your data follows this layout:

- data/
  - NW-UCLA/
    - all_sqe
  - ntu/
  - ntu120/
  - nturgbd_raw/
    - nturgb+d_skeletons/     # from s001_to_s017.zip
    - nturgb+d_skeletons120/  # from s018_to_s032.zip

After downloading the raw NTU RGB+D files, you must run a three-step processing pipeline to generate the usable dataset. Navigate to the appropriate data directory (./data/ntu or ./data/ntu120) and execute the following scripts in order:

1. Get raw skeletons: python get_raw_skes_data.py
2. Denoise: python get_raw_denoised_data.py (removes bad skeletons)
3. Transform: python seq_transformation.py (centers the skeleton to the first frame)

cd ./data/ntu
python get_raw_skes_data.py
python get_raw_denoised_data.py
python seq_transformation.py

When loading weights via the Processor.load_model() method, the system supports:

1. Format Support: Both .pt (PyTorch) and .pkl (Pickle) files.
2. Weight Stripping: It automatically removes the module. prefix (common when loading DataParallel models) from keys.
3. Selective Loading: Using the --ignore-weights flag, you can specify substrings of weight names to be excluded from the state dict. This is useful for fine-tuning models where certain layers (like the final classification head) need to be re-initialized.
4. Partial Matching: If the loaded weights do not perfectly match the model's state_dict, the system attempts to update the model's state with the available weights rather than failing.

The main.py script serves as the primary entrypoint for training and testing the CTR-GCN model. It uses a configuration-first approach where parameters can be defined in a YAML file, overridden by command-line arguments.

Parameter Priority: Command Line Arguments > YAML Config File > Default Values.

To run the script, you must specify the --phase (either train or test) and provide a --config path. If testing, you must also provide --weights pointing to a .pt or .pkl model file.

# Example: Training
python main.py --phase train --config ./config/nturgbd-cross-view/test_bone.yaml --work-dir ./my_experiment

# Example: Testing a specific model
python main.py --phase test --config ./config/nturgbd-cross-view/test_bone.yaml --weights ./work_dir/runs/model-epoch-10.pt

To combine results from different modalities (joint, bone, motion, and bone-motion) for a final ensemble, use the ensemble.py script. You must provide the dataset type and the directory paths where each modality's results were stored.

# Example: ensemble four modalities of CTRGCN on NTU RGB+D 120 cross subject
python ensemble.py --datasets ntu120/xsub --joint-dir work_dir/ntu120/csub/ctrgcn --bone-dir work_dir/ntu120/csub/ctrgcn_bone --joint-motion-dir work_dir/ntu120/csub/ctrgcn_motion --bone-motion-dir work_dir/ntu120/csub/ctrgcn_bone_motion

The Processor class is the core orchestration engine. It handles model initialization, data loading, optimizer setup, and the execution of training and evaluation loops.

Key Methods:

• start(): The main entrypoint. If phase is train, it runs the training loop and then evaluates the best model. If phase is test, it runs evaluation on the provided weights.
• train(epoch, save_model=False): Executes one epoch of training, calculates loss/accuracy, and optionally saves the model.
• eval(epoch, save_score=False, loader_name=['test'], wrong_file=None, result_file=None): Executes evaluation. It can optionally output a wrong_file (containing indices and incorrect predictions) and a result_file (containing correct/incorrect labels) for error analysis.

# Conceptual usage within the script
processor = Processor(arg)
processor.start()