Hello, in this lecture, it was mentioned that for a NN system, if you find it has a bias problem, you increase the model, and if you find variance problem, you increase the data size (under assumption that you have both enough computation power and data). It was also mentioned that this is what makes NN so powerful.
But as far as I understand, isn’t this the same for other models. For other models, if they have enough complexity, they don’t suffer from bias problem, and if they have enough data, they don’t (almost always) suffer from variance problem. I quite didn’t understand why this characteristics makes NN so powerful compared to other models.
Or is it that, while for other models you have to manually create new features or apply completely different algorithm to increase complexity, for NN, you just have to increase the NN size to increase complexity, which makes NN so powerful?
It would be a great help if further explanation provided. Thanks.
This doesn’t always work 100% of the time, but, in general, it works pretty well. The idea is that it’s easy to scale up a neural network by adding more neurons/layers.
It depends on the model. Each non-NN has their own characteristic.
Yea, I think this is a good way to think about it.
For many non-NN models, you usually need to do more feature engineering to get better results.
For NN models, if you increase the size just the right amount, they tend to be able to discover some of these hidden features automatically.