DOCSEARCHbyAlgolia
Submit Docs

RadImageNet Documentation

repository·main·Indexed 19 days ago

https://github.com/bmeii-ai/radimagenet

An open-access medical imaging database containing 1.35 million annotated CT, MRI, and ultrasound images across 3 modalities, 11 anatomies, and 165 pathologic labels. It provides pretrained models for ResNet50, DenseNet121, InceptionResNetV2, and InceptionV3 in TensorFlow and PyTorch to improve transfer learning for downstream medical imaging tasks.

Tokens
2.2K
Snippets
4
Records
7
Agent score
17%

What's inside RadImageNet

  1. Access the RadImageNet dataset

    main

    The RadImageNet database is an open-access medical imaging database containing 1.35 million annotated CT, MRI, and ultrasound images across 3 modalities, 11 anatomies, and 165 pathologic labels.

    To obtain the dataset, you must submit a request via the official website. If you do not receive a response, you can contact the researchers directly via email.

  2. Download pretrained RadImageNet models

    main

    Pretrained models (ResNet50, DenseNet121, InceptionResNetV2, and InceptionV3) are available for both TensorFlow and PyTorch. These models are trained on RadImageNet medical images and are intended for use as starting points for transfer learning on downstream medical imaging applications.

    Note: Swin Transformer weights are planned for a future release.

  3. Train a model with RadImageNet features

    main

    To train a classifier on top of RadImageNet features, compose a nn.Sequential model containing the Backbone and a Classifier. The training loop should include standard PyTorch boilerplate: switching between .train() and .eval() modes, zeroing gradients, and saving the best model based on validation loss.

    Example setup:

    1. Define a Backbone (feature extractor).
    2. Define a Classifier (linear layers).
    3. Combine them: model = nn.Sequential(backbone, classifier).
    4. Use torch.save(model.state_dict(), ...) to persist the best performing weights.
    # Model composition
model = nn.Sequential(backbone, classifier)
device = torch.device("cuda")
model = model.to(device)

# Training loop snippet
for e in range(num_epochs):
    model.train()
    for i_batch, info_batch in enumerate(train_loader):
        data, labels = info_batch['image'].to(device), info_batch['label'].to(device)
        optimizer.zero_grad()
        target = model(data)
        loss = criterion(target, labels)
        loss.backward()
        optimizer.step()

    model.eval()
    # ... validation logic ...
    if min_valid_loss > valid_loss:
        torch.save(model.state_dict(), 'best_model.pth')

  4. Cite RadImageNet in research

    main

    If you use the RadImageNet dataset or models in your research, please cite the following paper:

    @article{doi:10.1148/ryai.210315,
author = {Mei, Xueyan and Liu, Zelong and Robson, Philip M. and Marinelli, Brett and Huang, Mingqian and Doshi, Amish and Jacobi, Adam and Cao, Chendi and Link, Katherine E. and Yang, Thomas and Wang, Ying and Greenspan, Hayit and Deyer, Timothy and Fayad, Zahi A. and Yang, Yang},
title = {RadImageNet: An Open Radiologic Deep Learning Research Dataset for Effective Transfer Learning},
journal = {Radiology: Artificial Intelligence},
volume = {0},
number = {ja},
pages = {e210315},
year = {0},
doi = {10.1148/ryai.210315},

URL = { 
        https://doi.org/10.1148/ryai.210315
    
},
eprint = { 
        https://doi.org/10.1148/ryai.210315
}
}
    @article{doi:10.1148/ryai.210315,
author = {Mei, Xueyan and Liu, Zelong and Robson, Philip M. and Marinelli, Brett and Huang, Mingqian and Doshi, Amish and Jacobi, Adam and Cao, Chendi and Link, Katherine E. and Yang, Thomas and Wang, Ying and Greenspan, Hayit and Deyer, Timothy and Fayad, Zahi A. and Yang, Yang},
title = {RadImageNet: An Open Radiologic Deep Learning Research Dataset for Effective Transfer Learning},
journal = {Radiology: Artificial Intelligence},
volume = {0},
number = {ja},
pages = {e210315},
year = {0},
doi = {10.1148/ryai.210315},

URL = { 
        https://doi.org/10.1148/ryai.210315
    
},
eprint = { 
        https://doi.org/10.1148/ryai.210315
}
}

  6. Load pre-trained RadImageNet ResNet50 weights

    main

    To use the pre-trained RadImageNet weights with a custom PyTorch model, you can load the state dictionary into a Backbone module. The example uses a modified ResNet50 architecture where the encoder layers are extracted to serve as the feature extractor.

    Note: Ensure the architecture of your Backbone class matches the structure of the saved .pt file (e.g., the number of layers extracted via resnet50(pretrained=False).children()).

    # Define the backbone structure
class Backbone(nn.Module):
    def __init__(self):
        super().__init__()
        base_model = resnet50(pretrained=False)
        encoder_layers = list(base_model.children())
        self.backbone = nn.Sequential(*encoder_layers[:9])

    def forward(self, x):
        return self.backbone(x)

# Initialize and load weights
backbone = Backbone()
backbone.load_state_dict(torch.load("resnet50_torch.pt"))

  7. Implement a custom Dataset for medical images

    main

    When working with RadImageNet datasets, you can implement a torch.utils.data.Dataset to handle image loading, normalization, and resizing. The following pattern uses cv2 for loading and applies a specific normalization: (image - 127.5) * 2 / 255.

    Expected DataFrame columns:

    • img_dir: Path to the image file.
    • label: Integer label for classification.
    class createDataset(Dataset):
    def __init__(self, dataframe, transform=None):
        self.dataframe = dataframe
        self.transform = transforms.Compose([transforms.ToTensor()])

    def __len__(self):
        return self.dataframe.shape[0]

    def __getitem__(self, index):
        image = self.dataframe.iloc[index]["img_dir"]
        image = cv2.imread(image)
        # Normalization and resizing
        image = (image - 127.5) * 2 / 255
        image = cv2.resize(image, (224, 224))
        
        if self.transform is not None:
            image = self.transform(image)
            
        label = self.dataframe.iloc[index]["label"]
        return {"image": image, "label": torch.tensor(label, dtype=torch.long)}