I suppose there might be a bit confusion to your understanding,
Read through the comment which explain when gamma u is 0, the initial value is updated and not the new one.
Also the significance of tanh provides a nonlinear response which can capture higher-level patterns or features in the input data when comes to training a neural network.