After knowing the Human level performance, train,dev error. Suppose we have Avoidable bias, and we modify the model to reduce the avoidable bias, during this process, will the variance also vary? I mean, will be be able to reduce both the avoidable bias as well as variance, rather than concentrating on only one of the techniques?
Reducing bias means the model better fits the training data. Hopefully, the features learnt by the model helps reduce loss in the validation set. This behavior is observed only when training and validation sets come from the same distribution.
1 Like
In most cases, yes. The point is that variance and bias are (at a high level anyway) the inverses of each other. Most methods for reducing bias involve increasing variance (making your model more complex and better able to make fine distinctions). But the difficulty is that we always need to apply the Goldilocks Principle: just enough variance, but not too much.
1 Like