Basic Recipe for machine learning Question

In the video, towards the end, Andrew says that training a bigger network almost never hurts, but if we keep adding layers and keep training on bigger network would that not introduce overfitting? In the previous course it was mentioned that too many layers is a bad thing for a network.

@bhagyashree_desai
What Andrew meant when he says training a bigger network almost never hurts is that with a sufficiently large amount of data training a bigger network would not hurt the performance.

But if you are training a bigger network from scratch on a smaller dataset, it would definitely overfit.

1 Like