Hi the Expected values are:
W1 =
[[ 1.63942428 -0.6268425 -0.54320974]
[-1.08782943 0.85036983 -2.2865723 ]]
W2 =
[[ 0.33356139 -0.26425199 1.47707772]
[-2.04538458 -0.30744933 -0.36903141]
[ 1.14873036 -1.09256871 -0.15734651]]
b1 =
[[ 1.75854357]
[-0.74616067]]
b2 =
[[-0.89228024]
[ 0.02707193]
[ 0.56782561]]
I don't get the dimensions because it should be
W1 -- weight matrix of shape (n_h, n_x)
b1 -- bias vector of shape (n_h, 1)
W2 -- weight matrix of shape (n_y, n_h)
b2 -- bias vector of shape (n_y, 1)
in this case n_h = 2 n_x = 3 n_y = 3 and W2 is 3 x 3 instead of 3 x 2
1 Like
Interesting. So the dimensions are actually not consistent. Thanks for your sharp eyes! It’s kind of amazing that no-one else has noticed that up to this point (it’s been that way for literally years). This is just a synthetic test case to check your update logic, so in that sense it doesn’t matter. But I totally agree that it’s “bad form”. I will file a git issue about this.
Thanks again for your observational skills!
But wait, there’s more! I added a new cell and copied the test cell for the update_parameters_with_gd function and added a “Before” print statement. Here’s what I see:
Before
W1 =
[[ 1.62434536 -0.61175641 -0.52817175]
[-1.07296862 0.86540763 -2.3015387 ]]
b1 =
[[ 1.74481176]
[-0.7612069 ]]
W2 =
[[ 0.3190391 -0.24937038 1.46210794]
[-2.06014071 -0.3224172 -0.38405435]
[ 1.13376944 -1.09989127 -0.17242821]]
b2 =
[[-0.87785842]
[ 0.04221375]
[ 0.58281521]]
After:
W1 =
[[ 1.63535156 -0.62320365 -0.53718766]
[-1.07799357 0.85639907 -2.29470142]]
b1 =
[[ 1.74604067]
[-0.75184921]]
W2 =
[[ 0.32171798 -0.25467393 1.46902454]
[-2.05617317 -0.31554548 -0.3756023 ]
[ 1.1404819 -1.09976462 -0.1612551 ]]
b2 =
[[-0.88020257]
[ 0.02561572]
[ 0.57539477]]
All test passed
Notice that my “After” values are different than yours. An error in the second or third decimal place is definitely not just a rounding error. So I think there is something wrong with your update logic.
There really aren’t that many moving parts here. I tried doing += instead of -= on the updates, since that’s a common mistake, but that doesn’t generate the same wrong values as yours. The only other thing I can think of is that you hard-coded the learning rate somehow and are ignoring the actual value being passed in.
Also note that the routine that generates the test case data sets the random seed, so adding that cell does not change the results.
Oh, sorry, I may be off track here. I just realized that there are three separate test cells for three different “update parameters” functions. You don’t say which one you are referring to. So perhaps your logic is not really incorrect. You’ll know from the test cases in the notebook.
Interestingly all 3 of the test cases have the same dimensional flaw that you pointed out. The authors of the code were obviously in “copy/paste” mode …
1 Like