I’m training a model using image-text pair based contrastive learning approach using Pytorch framework. The purpose of training is to use self-supervised learning for chest x-ray image classification.
My workstation has 2 Nvidia 3080 ti GPUs with 12 GB dedicated memory each. I’m using Distributed Data Parallel for training purpose. With an image size of 320x320, I can use a maximum batch size of 32 which fills up all the available vRAM. However, the GPU utilization is nearly 10% for each GPU. Why this is so? How can I increase this utilization percentage?
2 Likes
More vRam, better data busses to connect the Ram with GPU, also keep in mind that some computing power RAM and GPU is used by the operating system and its processes too!
Thanks for your response. Is there some other technique which can help me reduce the memory footprint and at the same time increase the GPU utilization? I came across a technique called Fully Sharded Data Parallel but I’m unable to implement that.
1 Like
I am not familiar with it sorry!
Hello Harshit,
I am not sure if the below link will help you, but you can refer as it is related to pointer to make better GPU utilisation.
Regards
DP
2 Likes
Try lightning ?
1 Like