How to determine the source model for transfer learning?

I wish to use a model trained on ImageNet dataset (ResNet, Inception, DenseNet, EfficientNet, etc.) for trasnfer learning. I wish to train a CNN model on medical x-ray images with approximately 300 thousand images. I’m confused as to how to determine the model which would be best for my case. I don’t have time to try and use each model one by one and then compare the accuracy to figure out which model would be best for me. Is there any way how I could determine the best source model for my case?

Hi @Harshit1097,

welcome to the community!

Maybe Huggingface’s transformers docs and models are relevant for you. In my opinion they are definitely worth a closer look:

Of course you can also use there the search and filter function to get a good overview on state of the art pretrained transformers which you can use for transfer learning afterwards…

Afterwards there are several ways how to proceed but I guess this would be a good starting point, see also this similar thread.

Best regards
Christian

Thanks Christian. Appreciate the help!

1 Like

My pleasure, @Harshit1097!

Feel free to leave an update if your search was successful. Perhaps other follow learners could benefit from your experiences.

Have fun and good luck with transfer learning!

Best regards
Christian

Beyond which model to fine tune, you may wish to consider if ImageNet is the best starting point for your transfer learning task. For this,
https://openaccess.thecvf.com/content_cvpr_2018/papers/Cui_Large_Scale_Fine-Grained_CVPR_2018_paper.pdf might be helpful? Specifically the section on domain similarity:

Under this assumption, transfer learning can be viewed as moving a set of images from the source domain S to the target domain T… the distance between two domains can be defined as the least amount of total work needed. This definition of domain similarity can be calculated by the Earth Mover’s Distance (EMD) [41, 43].

Good luck!

@alexandershepard thanks for sharing the information.