I failed some of tests about gradient descent function but can't find the reason

Course Q&A Natural Language Processing NLP with Probabilistic Models
1

Hi~
I passed all the tests of the functions used in the gradient_descent function, including backprop. I implemented the weight update formula correctly, W_new = W_old - alpha*W_old. However, 7 tests failed in the test cell, while 9 passed. Does anyone have any idea why?

name default_check
iters: 10 cost: 9.090380
Wrong output values for W1 matrix.
	 Expected: [[0.3715731  0.39385588 0.12117024 ... 0.21485169 0.8417732  0.4013149 ]
 [0.14116653 0.54929948 0.39651013 ... 0.64207701 0.6203919  0.97065011]
 [0.04290103 0.38236303 0.41687347 ... 0.18933903 0.40646899 0.71397686]
 ...
 [0.22607702 0.20055185 0.18807501 ... 0.39607305 0.78038406 0.6061392 ]
 [0.47582915 0.14843496 0.21027327 ... 0.87438456 0.75839127 0.97442377]
 [0.14481384 0.24052239 0.05433805 ... 0.71292201 0.93679829 0.72879085]] 
	Got: [[0.37117565 0.39288078 0.12117024 ... 0.21485169 0.8417732  0.4013149 ]
 [0.13991714 0.54941463 0.39651013 ... 0.64207701 0.6203919  0.97065011]
 [0.04100031 0.382276   0.41687347 ... 0.18933903 0.40646899 0.71397686]
 ...
 [0.22690606 0.2005237  0.18807501 ... 0.39607305 0.78038406 0.6061392 ]
 [0.47454242 0.14638964 0.21027327 ... 0.87438456 0.75839127 0.97442377]
 [0.14643882 0.24282176 0.05433805 ... 0.71292201 0.93679829 0.72879085]].
Wrong output values for W2 matrix.
	 Expected: [[1.04355856 0.26318299 0.84914897 ... 0.75019496 0.94294152 0.1368148 ]
 [0.98069515 0.65404213 0.1068009  ... 1.07724596 0.88271816 0.14056072]
 [0.74306881 0.69342346 0.77137288 ... 0.46450717 0.16699485 0.19468519]
 ...
 [0.61897234 0.52198013 0.44595451 ... 0.91218032 0.84877489 0.93974697]
 [0.46573302 0.27582731 0.8577078  ... 0.55740859 0.76747007 0.59148898]
 [0.88165068 0.52037408 0.49309489 ... 0.17477077 0.81721016 0.49459079]] 
	Got: [[1.04355696 0.26318383 0.84911302 ... 0.75020904 0.94290574 0.13686753]
 [0.98067898 0.65403833 0.10678908 ... 1.07726221 0.88269575 0.14058947]
 [0.74306882 0.69342346 0.77137294 ... 0.46450717 0.16699492 0.19468514]
 ...
 [0.61897232 0.5219801  0.44595459 ... 0.91218029 0.84877504 0.93974686]
 [0.46573306 0.27582735 0.85770793 ... 0.55740866 0.76747021 0.59148895]
 [0.88165068 0.52037409 0.4930949  ... 0.17477077 0.81721017 0.49459078]].
Wrong output values for b1 vector.
	 Expected: [[0.35306266]
 [0.36809944]
 [0.62566526]
 [0.4766034 ]
 [0.36000409]
 [0.60847943]
 [0.04476555]
 [0.84473931]
 [0.28527232]
 [0.19606767]] 
	Got: [[0.35306351]
 [0.36809621]
 [0.62567814]
 [0.47660443]
 [0.3600132 ]
 [0.60847978]
 [0.04475389]
 [0.84473729]
 [0.28529365]
 [0.19605157]].
Wrong output values for gradient of b2 vector.
	 Expected: [[0.8511688 ]
 [0.07720283]
 [0.77574835]
 ...
 [0.26060613]
 [0.67110795]
 [0.83251134]] 
	Got: [[0.85116895]
 [0.07720285]
 [0.77574836]
 ...
 [0.26060611]
 [0.67110801]
 [0.83251135]].
name small_check
iters: 10 cost: 8.652134
Wrong output values for W1 matrix.
	 Expected: [[0.22146088 0.87091686 0.20671916 ... 0.07752275 0.77457191 0.67174422]
 [0.27796801 0.53216618 0.30534023 ... 0.38543118 0.28671259 0.13483498]
 [0.98969129 0.47077882 0.28648156 ... 0.8206767  0.03792429 0.60534254]
 [0.34361261 0.59263915 0.27437007 ... 0.24937364 0.37304776 0.02650941]
 [0.86246255 0.86536007 0.9106285  ... 0.30000818 0.2322273  0.7837576 ]] 
	Got: [[0.22152875 0.87099662 0.20671916 ... 0.07752275 0.77457191 0.67174422]
 [0.27777287 0.53187974 0.30534023 ... 0.38543118 0.28671259 0.13483498]
 [0.9892534  0.47081179 0.28648156 ... 0.8206767  0.03792429 0.60534254]
 [0.34426091 0.59201743 0.27437007 ... 0.24937364 0.37304776 0.02650941]
 [0.86267728 0.86539586 0.9106285  ... 0.30000818 0.2322273  0.7837576 ]].
Wrong output values for W2 matrix.
	 Expected: [[0.76340197 0.5452355  0.93398193 0.73610848 1.01578978]
 [0.12184119 1.0054892  0.93145991 0.05333345 0.8317053 ]
 [0.10183728 0.04003453 0.99304243 0.70575225 0.2790009 ]
 ...
 [0.44888194 0.68064171 0.25140396 0.82963448 0.9178064 ]
 [0.02947694 0.82557034 0.36649496 0.49664268 0.41883318]
 [0.64371722 0.51447755 0.43217078 0.32061603 0.31237661]] 
	Got: [[0.76340131 0.54523548 0.93398319 0.73610873 1.01578774]
 [0.12184067 1.00548815 0.93145875 0.05333274 0.83170586]
 [0.10183728 0.04003453 0.99304243 0.70575225 0.27900089]
 ...
 [0.44888194 0.68064171 0.25140396 0.82963447 0.9178064 ]
 [0.02947695 0.82557034 0.36649496 0.49664268 0.41883318]
 [0.64371723 0.51447755 0.43217078 0.32061602 0.31237661]].
Wrong output values for b1 vector.
	 Expected: [[0.27139757]
 [0.68261082]
 [0.30245577]
 [0.02300601]
 [0.31519436]] 
	Got: [[0.27139864]
 [0.68261116]
 [0.30245569]
 [0.02300409]
 [0.31519471]].
 9  Tests passed
 7  Tests failed
2

Hi @Nadle

First thing - alpha should be multiplied by the gradient (grad_W but not W_old).
Second thing - are you re-setting W_old = W_new later after the update?

These are some obvious points to make from what I can see.

Cheers

3

I accidentally wrote W_old instead of grad_W. ^^;
Fortunately, the problem I was asking for advice on above was resolved.
Thank you for your answer!

4

Hello, I am currently facing the same issue where the code is failing 9 of the 16 tests despite seeming correct prior to this test. Could you kindly share with me how you managed to resolve this issue? Any assistance would be greatly appreciated. Thank you in advance!