Hi!
I have a question.
What happen to weights values in vanishing gradient and exploding gradient problem.
Does the weights reduce slowly, reduce fast or increase, I am a bit confuse. So please explain what happens to weights exactly during both problems.
With vanishing gradients, the weight values no longer change, and they are stuck at whatever values they learned when the gradients were larger.
With exploding gradients, the weight values diverge to +Inf or -Inf.
1 Like