ConvNeXt V2 Documentation

To set up the environment for ConvNeXt V2 ImageNet classification experiments, create a new conda environment and install the required Python packages and system dependencies.

1. Create and activate environment:

   conda create -n convnextv2 python=3.8 -y
   conda activate convnextv2

2. Install PyTorch and torchvision: Ensure you install PyTorch >= 1.8.0 and torchvision >= 0.9.0. Example for CUDA 11.1:

   pip install torch==1.8.0+cu111 torchvision==0.9.0+cu111 -f https://download.pytorch.org/whl/torch_stable.html

3. Install core packages: Clone the repository and install timm, tensorboardX, six, submitit, and openblas-devel:

   git clone https://github.com/facebookresearch/ConvNeXt-V2.git
   pip install timm==0.3.2 tensorboardX six
   pip install submitit
   conda install openblas-devel -c anaconda -y

To reproduce the results from the paper using a SLURM cluster, you can use multi-node training via the submitit library. Ensure you have installed submitit before running the training scripts.

pip install submitit

The project requires specific directory structures for ImageNet datasets.

ImageNet-1K Structure

For classification experiments, organize the data into train and val folders, each containing subdirectories for every class:

/path/to/imagenet-1k/
  train/
    class1/
      img1.jpeg
    class2/
      img2.jpeg
  val/
    class1/
      img3.jpeg
    class2/
      img4.jpeg

ImageNet-22K Structure

For pre-training, organize the data into class-named subdirectories directly under the root:

/path/to/imagenet-22k/
  class1/
    img1.jpeg
  class2/
    img2.jpeg
  class3/
    img3.jpeg
  class4/
    img4.jpeg

Evaluation can be performed using the main_finetune.py script. You can run evaluation on a single GPU or across multiple GPUs using torch.distributed.launch.

When evaluating different model variants, ensure you update the --model, --resume (path to checkpoint), and --input_size flags. Note that while --drop_path is required during training, it is not strictly required during evaluation as the DropPath module in timm behaves consistently.

### Single-GPU Evaluation
```bash
python main_finetune.py \
--model convnextv2_base \
--eval true \
--resume /path/to/checkpoint \
--input_size 224 \
--data_path /path/to/imagenet-1k \

Multi-GPU Evaluation

python -m torch.distributed.launch --nproc_per_node=8 main_finetune.py \
--model convnextv2_base \
--eval true \
--resume /path/to/checkpoint \
--input_size 224 \
--data_path /path/to/imagenet-1k \

You can perform FCMAE pre-training on ImageNet-1K using either a multi-node SLURM cluster or a single machine with PyTorch distributed launch.

Multi-node (SLURM): Use submitit_pretrain.py. Single-machine: Use main_pretrain.py with torch.distributed.launch.

# Multi-node example
python submitit_pretrain.py --nodes 8 --ngpus 8 \
--model convnextv2_base \
--batch_size 64 \
--blr 1.5e-4 \
--epochs 1600 \
--warmup_epochs 40 \
--data_path /path/to/imagenet-1k \
--job_dir /path/to/save_results

# Single-machine example
python -m torch.distributed.launch --nproc_per_node=8 main_pretrain.py \
--model convnextv2_base \
--batch_size 64 --update_freq 8 \
--blr 1.5e-4 \
--epochs 1600 \
--warmup_epochs 40 \
--data_path /path/to/imagenet-1k \
--output_dir /path/to/save_results

Install NVIDIA apex from source with C++ and CUDA extensions enabled:

git clone https://github.com/NVIDIA/apex
cd apex
pip install -v --disable-pip-version-check --no-cache-dir --global-option="--cpp_ext" --global-option="--cuda_ext" ./
cd ..

ConvNeXt V2 requires a customized version of MinkowskiEngine that supports depth-wise convolutions. You must initialize submodules before installing.

1. Update submodules:

   git submodule update --init --recursive
   git submodule update --recursive --remote

2. Build and install: Navigate to the MinkowskiEngine directory and install using openblas:

   cd MinkowskiEngine
   python setup.py install --blas_include_dirs=${CONDA_PREFIX}/include --blas=openblas