C2, W2, Programming Task / Update_parameters_with_momentum

Course Q&A Deep Learning Specialization Improving Deep Neural Networks: Hyperparameter tun
1

Hello there!

I am stuck at the “update parameters with momentum” programming exercise. The Autograder marks 2 out of 3 assertion as correct.

I have implemented the formula as per the assignment:

  • The velocity of both W(l) and b(l) have been calculated as follows:
    — Two derivatives have been used (dW(l) and db(l)) have been used to calculate the velocity by multiplying them by (beta) and by (1-beta) respectively.

  • The W(l) and the b(l) parameters are being updated by subtracting the newly calculated velocity times the learning rate.

Somehow this just gets 2 out of 3 assertions right and I cannot find what is wrong with the formulas. The error message is too generic for me.

Does anybody have any pointers? Thank you for any help!

Log for completeness:

W1 = 
[[ 1.63535156 -0.62320365 -0.53718766]
 [-1.07799357  0.85639907 -2.29470142]]
b1 = 
[[ 1.74604067]
 [-0.75184921]]
W2 = 
[[ 0.32171798 -0.25467393  1.46902454]
 [-2.05617317 -0.31554548 -0.3756023 ]
 [ 1.1404819  -1.09976462 -0.1612551 ]]
b2 = 
[[-0.88020257]
 [ 0.02561572]
 [ 0.57539477]]
v["dW1"] = 
[[-1.10061918  1.14472371  0.90159072]
 [ 0.50249434  0.90085595 -0.68372786]]
v["db1"] = 
[[-0.12289023]
 [-0.93576943]]
v["dW2"] = 
[[-0.26788808  0.53035547 -0.69166075]
 [-0.39675353 -0.6871727  -0.84520564]
 [-0.67124613 -0.0126646  -1.11731035]]
v["db2"] = v[[0.2344157 ]
 [1.65980218]
 [0.74204416]]
 2  Tests passed
 1  Tests failed
---------------------------------------------------------------------------
AssertionError                            Traceback (most recent call last)
<ipython-input-22-f37bbbdbd581> in <module>
     11 print("v[\"db2\"] = v" + str(v["db2"]))
     12 
---> 13 update_parameters_with_momentum_test(update_parameters_with_momentum)

~/work/release/W2A1/public_tests.py in update_parameters_with_momentum_test(target)
    155     ]
    156 
--> 157     multiple_test(test_cases, target)
    158 
    159 def initialize_adam_test(target):

/opt/conda/lib/python3.7/site-packages/dlai_tools/testing_utils.py in multiple_test(test_cases, target)
    162         print('\033[91m', len(test_cases) - success, " Tests failed")
    163         raise AssertionError(
--> 164             "Not all tests were passed for {}. Check your equations and avoid using global variables inside the function.".format(target.__name__))

AssertionError: Not all tests were passed for update_parameters_with_momentum. Check your equations and avoid using global variables inside the function.
2

Fixed by @Jonathanlugo.

It was some error in the computation of the velocity formulas.

3 Likes
3

Thank you, @kampamocha.

The formula from the notebook is a bit small. This is the part that confused @Jonathanlugo, in case it helps others (v was capitalized for clarity):

image

3 Likes
4

Thank you @kampamocha and @nramon for your help! Problem solved.

Jonathan

2 Likes
5

Hey !! I’m facing the same issue . Have spent quite a time but couldn’t figure out what’s the bug in there. could you please tell me how did you resolved it ?
Thank you

6

Hi @Maaz , I had the formula wrong, as I was reading it wrong. I resolved it by multiplying

beta * v[derivative of W + …]

Just implement the formula as it shows in the assignment. I hope this pointer helps!

1 Like