For those who want to run the code where there is multiple devices (GPUs) to practice distributed training, I have been using kaggle runtime.
Kaggle accelerator GPU has two devices, and it is good for practise.
2 Likes
Hi @Akingbeni_David ,
Very good Advice!
I have a notebook in kaggle, that is a short and easy tutorial about how to use multiple GPUs
https://www.kaggle.com/code/peremartramanonellas/using-multiple-gpu-s-with-tensorflow-on-kaggle
I hope it can be useful to someone 
1 Like