Hi there,
I had an interesting result with gradient descent on the logistic regression model in the C1_W3_Lab06_Gradient_Descent_Soln, where my values of w & b ended up off the edge of the surface plot. I thought it was always meant to end up in the shaded area - any explanation of what’s going on?
I’ve managed to get a similar result in 2 different ways:
- Clicking ‘Run Gradient Descent’ after manually choosing a good w & b on the contour plot
- Running Gradient Descent twice. The first result was about where I expected, but when I clicked again the cross moved off the surface plot.
Is it possible to overdo gradient descent?
In context:
Hi @Nightingale
This can happen due to a high learning rate that causes the updates to overshoot the optimal region. Another possibility is that running gradient descent multiple times without resetting the learning rate or initialization can amplify updates and push the parameters outside the plotted range.
Try reducing the learning rate \alpha to ensure smaller, more controlled updates. Also, confirm that the surface plot covers the full parameter range relevant to your model and that the stopping criteria for gradient descent are appropriate.
Hope it helps! Feel free to ask if you need further assistance.
1 Like
Thanks @Alireza_Saei, that’s really helpful. I imagine in this case it was the amplified updates due to not resetting. Good to know that can happen!
You’re welcome, happy to help!
1 Like