Cost increasing after more instances of gradient desecen

Course Q&A Machine Learning Specialization Supervised ML: Regression and Classification
1

Week 1: C1_W1_Lab04_Gradient_Descent_Soln

image
image1075×242 9.64 KB

As you can see the cost is increasing in iterations 2000, 4000, 5000, 7000, 9000
Therefore the final output may have a cost higher than in other point of gradient decent. I also built a model based on this. It converged on 4.287… (cost) but at one point it was able to reach 0.000…(cost).

How Do I prevent this??

2

The cost is decreasing, not increasing, 10 to -2 is bigger than 10 to -4!

3

Indeed. Look at the exponents.