What does it mean? " Large neural networks are low bias machines?

Course Q&A Machine Learning Specialization Advanced Learning Algorithms
1

Large neural networks are low bias machines. That quote said by Andrew,
in content: Bias/variance and neural networks | Coursera

Help me understand that quote? Thanks a lot

2

Hello @Nhat_Minh, let me quote the narration:

And it turns out that large neural networks when trained on small to moderate sized datasets are low bias machines. And what I mean by that is, if you make your neural network large enough, you can almost always fit your training set well. So long as your training set is not enormous. And what this means is this gives us a new recipe to try to reduce bias or reduce variance as needed without needing to really trade off between the two of them.

How do you think about the bolded line?

Raymond

2 Likes
3

does it mean small neural network are high bias?

4

Hi @Dafa_Aqilla_M322DSX0, not necessary. If you have just one sample, a small neural network can still be high bias. We need to take the sample size into consideration.

Raymond