your explanation of your issue lacks information what kind of model you are working on. Kindly give a brief detail about your model without sharing codes like your batch size or steps per epoch if any used, model compile parameters.
If your each epoch time is taking 5-6 minutes it can be surely tpu related but it can also be model algorithm related.
So to know if it is not model related, one needs to have a bit understanding about your model, to give you hints on improving the training time.
P.S. note even for me the last two assignment of course 4 took pretty long time like 2 hours as in whole but I created the simplest model by referring to different threads in discourse community (sharing one link where I had asked for help)
Also if you feel it is GPU related, refer the below link