Hi. I’m not sure whether this has not been already reported, but, in my opinion, there’s an incorrect code in the C2_W3_Lab_1_regression_with_perceptron Jupyter notebook.
Normalization is done by adv_norm = (adv - np.mean(adv))/np.std(adv), but np.mean gives mean for all values in the dataset, i.e. for TV and Sales column values combined, which means the normalization is done incorrectly. np.mean(adv) returns scalar 80.5325, but we need means for both columns. That can be done by using adv.mean(), which returns
TV 147.0425
Sales 14.0225
dtype: float64
If we plot the dataset now, we can see that z-score normalization was not performed, since the values do not have mean = 0 and std = 1.
This can be tracked further down, where the cost function increases with every iteration, even when the notebook states otherwise (“You can see that after a few iterations the cost function does not change anymore (the model converges).”).
Cost after iteration 0: 82.198751
Cost after iteration 1: 130.739329
Cost after iteration 2: 305.245531
Cost after iteration 3: 741.580765
Cost after iteration 4: 1807.749252
Cost after iteration 5: 4408.245575
Cost after iteration 6: 10750.238833
Cost after iteration 7: 26216.684591
Cost after iteration 8: 63935.226298
Cost after iteration 9: 155920.696905
Cost after iteration 10: 380248.742860
Cost after iteration 11: 927325.056636
Cost after iteration 12: 2261498.373272
Cost after iteration 13: 5515191.494311
Cost after iteration 14: 13450081.807199
Cost after iteration 15: 32801164.387933
Cost after iteration 16: 79993297.143455
Cost after iteration 17: 195082330.778827
Cost after iteration 18: 475753809.047071
Cost after iteration 19: 1160236736.958894
Cost after iteration 20: 2829508162.375940
Cost after iteration 21: 6900416256.774491
Cost after iteration 22: 16828276076.755072
Cost after iteration 23: 41039680097.593567
Cost after iteration 24: 100084841420.330750
Cost after iteration 25: 244080252534.341705
Cost after iteration 26: 595246681034.021606
Cost after iteration 27: 1451648003487.149414
Cost after iteration 28: 3540182571648.848633
Cost after iteration 29: 8633561724674.523438
W = [[3665169.03360114]]
b = [[2508550.18541392]]
If adv_norm is computed as adv_norm = (adv - adv.mean())/adv.std(), then the plot looks OK (0 in the middle of the axis) and there’s a real convergence of the cost function:
Cost after iteration 0: 0.494112
Cost after iteration 1: 0.204421
Cost after iteration 2: 0.193519
Cost after iteration 3: 0.193108
Cost after iteration 4: 0.193093
Cost after iteration 5: 0.193092
Cost after iteration 6: 0.193092
Cost after iteration 7: 0.193092
Cost after iteration 8: 0.193092
Cost after iteration 9: 0.193092
Cost after iteration 10: 0.193092
Cost after iteration 11: 0.193092
Cost after iteration 12: 0.193092
Cost after iteration 13: 0.193092
Cost after iteration 14: 0.193092
Cost after iteration 15: 0.193092
Cost after iteration 16: 0.193092
Cost after iteration 17: 0.193092
Cost after iteration 18: 0.193092
Cost after iteration 19: 0.193092
Cost after iteration 20: 0.193092
Cost after iteration 21: 0.193092
Cost after iteration 22: 0.193092
Cost after iteration 23: 0.193092
Cost after iteration 24: 0.193092
Cost after iteration 25: 0.193092
Cost after iteration 26: 0.193092
Cost after iteration 27: 0.193092
Cost after iteration 28: 0.193092
Cost after iteration 29: 0.193092
W = [[0.78222442]]
b = [[-2.55795385e-16]]
It’s actually also suggested if you run 