C2, WK1, 3rd Programming Assignment. Exercise 4 - gradient_check_n

Course Q&A Deep Learning Specialization Improving Deep Neural Networks: Hyperparameter tun
1

Hello!

I am battling with this code for several hours now and I just cannot figure it out.

This is the last programming exercise in the Gradient Checking programming assignment.

I am trying to calculate the approximated gradient using the given formula:

š‘”š‘Ÿš‘Žš‘‘š‘Žš‘š‘š‘Ÿš‘œš‘„[š‘–] = (š½+š‘–āˆ’š½āˆ’š‘–)/2šœ€

While doing so, in the first iteration the values are calculated (shown below). No problem. On the second iteration, the program crashes and throws the following error:

ValueError: could not broadcast input array from shape (12) into shape (1)

  • I already took a look at the shapes: they seem fine (see below)
  • The formula seems fine: Subtracting the j+[i] minus the J-[i] and then division by 2 epsilon. ok

Could anybody give me a pointer what I may be doing wrongly? I am also showing below the print out of my program (I have been told that the printout does not violates the honor code. If this were to be incorrect, please let me know and I will remove it)

Thanks for any help
Jonathan

value of i: 0
X:
[[ 1.62434536 -0.61175641 -0.52817175]
 [-1.07296862  0.86540763 -2.3015387 ]
 [ 1.74481176 -0.7612069   0.3190391 ]
 [-0.24937038  1.46210794 -2.06014071]]
(4, 3)
W1:
[[-0.3224171  -0.38405435  1.13376944 -1.09989127]
 [-0.17242821 -0.87785842  0.04221375  0.58281521]
 [-1.10061918  1.14472371  0.90159072  0.50249434]
 [ 0.90085595 -0.68372786 -0.12289023 -0.93576943]
 [-0.26788808  0.53035547 -0.69166075 -0.39675353]]
(5, 4)
W2:
[[ 0.2344157   1.65980218  0.74204416 -0.19183555 -0.88762896]
 [-0.74715829  1.6924546   0.05080775 -0.63699565  0.19091548]
 [ 2.10025514  0.12015895  0.61720311  0.30017032 -0.35224985]]
(3, 5)
W3:
[[0.58662319 0.83898341 0.93110208]]
(1, 3)
b1:
[[-0.6871727 ]
 [-0.84520564]
 [-0.67124613]
 [-0.0126646 ]
 [-1.11731035]]
(5, 1)
b2:
[[-1.1425182 ]
 [-0.34934272]
 [-0.20889423]]
(3, 1)
b3:
[[0.28558733]]
(1, 1)
J_plus:
2.4078333524866014
J_minus:
2.4078334271821493
value of i: 1
X:
[[ 1.62434536 -0.61175641 -0.52817175]
 [-1.07296862  0.86540763 -2.3015387 ]
 [ 1.74481176 -0.7612069   0.3190391 ]
 [-0.24937038  1.46210794 -2.06014071]]
(4, 3)
W1:
[[-0.3224172  -0.38405425  1.13376944 -1.09989127]
 [-0.17242821 -0.87785842  0.04221375  0.58281521]
 [-1.10061918  1.14472371  0.90159072  0.50249434]
 [ 0.90085595 -0.68372786 -0.12289023 -0.93576943]
 [-0.26788808  0.53035547 -0.69166075 -0.39675353]]
(5, 4)
W2:
[[ 0.2344157   1.65980218  0.74204416 -0.19183555 -0.88762896]
 [-0.74715829  1.6924546   0.05080775 -0.63699565  0.19091548]
 [ 2.10025514  0.12015895  0.61720311  0.30017032 -0.35224985]]
(3, 5)
W3:
[[0.58662319 0.83898341 0.93110208]]
(1, 3)
b1:
[[-0.6871727 ]
 [-0.84520564]
 [-0.67124613]
 [-0.0126646 ]
 [-1.11731035]]
(5, 1)
b2:
[[-1.1425182 ]
 [-0.34934272]
 [-0.20889423]]
(3, 1)
b3:
[[0.28558733]]
(1, 1)
J_plus:
(array([[ 1.45368313, -3.29348827,  2.99468224],
       [-0.25505587, -0.6794216 ,  0.07907767],
       [-2.23948552,  1.04112122, -3.47212244],
       [ 2.20318799, -2.43012334,  2.97376478],
       [-3.22938712, -0.54805397, -1.59975179]]), array([[1.45368313, 0.        , 2.99468224],
       [0.        , 0.        , 0.07907767],
       [0.        , 1.04112122, 0.        ],
       [2.20318799, 0.        , 2.97376478],
       [0.        , 0.        , 0.        ]]), array([[-0.3224172 , -0.38405425,  1.13376944, -1.09989127],
       [-0.17242821, -0.87785842,  0.04221375,  0.58281521],
       [-1.10061918,  1.14472371,  0.90159072,  0.50249434],
       [ 0.90085595, -0.68372786, -0.12289023, -0.93576943],
       [-0.26788808,  0.53035547, -0.69166075, -0.39675353]]), array([[-0.6871727 ],
       [-0.84520564],
       [-0.67124613],
       [-0.0126646 ],
       [-1.11731035]]), array([[-1.22440184, -0.36996027, -0.87973818],
       [-2.83889529, -0.29644569, -4.34728424],
       [ 3.50554288,  0.43368902,  6.98284034]]), array([[0.        , 0.        , 0.        ],
       [0.        , 0.        , 0.        ],
       [3.50554288, 0.43368902, 6.98284034]]), array([[ 0.2344157 ,  1.65980218,  0.74204416, -0.19183555, -0.88762896],
       [-0.74715829,  1.6924546 ,  0.05080775, -0.63699565,  0.19091548],
       [ 2.10025514,  0.12015895,  0.61720311,  0.30017032, -0.35224985]]), array([[-1.1425182 ],
       [-0.34934272],
       [-0.20889423]]), array([[3.54960559, 0.68939608, 6.7873245 ]]), array([[0.97206672, 0.66583257, 0.99887329]]), array([[0.58662319, 0.83898341, 0.93110208]]), array([[0.28558733]]))
J_minus:
(array([[ 1.45368335, -3.29348844,  2.9946827 ],
       [-0.25505587, -0.6794216 ,  0.07907767],
       [-2.23948552,  1.04112122, -3.47212244],
       [ 2.20318799, -2.43012334,  2.97376478],
       [-3.22938712, -0.54805397, -1.59975179]]), array([[1.45368335, 0.        , 2.9946827 ],
       [0.        , 0.        , 0.07907767],
       [0.        , 1.04112122, 0.        ],
       [2.20318799, 0.        , 2.97376478],
       [0.        , 0.        , 0.        ]]), array([[-0.3224172 , -0.38405445,  1.13376944, -1.09989127],
       [-0.17242821, -0.87785842,  0.04221375,  0.58281521],
       [-1.10061918,  1.14472371,  0.90159072,  0.50249434],
       [ 0.90085595, -0.68372786, -0.12289023, -0.93576943],
       [-0.26788808,  0.53035547, -0.69166075, -0.39675353]]), array([[-0.6871727 ],
       [-0.84520564],
       [-0.67124613],
       [-0.0126646 ],
       [-1.11731035]]), array([[-1.22440179, -0.36996027, -0.87973808],
       [-2.83889545, -0.29644569, -4.34728459],
       [ 3.50554333,  0.43368902,  6.98284131]]), array([[0.        , 0.        , 0.        ],
       [0.        , 0.        , 0.        ],
       [3.50554333, 0.43368902, 6.98284131]]), array([[ 0.2344157 ,  1.65980218,  0.74204416, -0.19183555, -0.88762896],
       [-0.74715829,  1.6924546 ,  0.05080775, -0.63699565,  0.19091548],
       [ 2.10025514,  0.12015895,  0.61720311,  0.30017032, -0.35224985]]), array([[-1.1425182 ],
       [-0.34934272],
       [-0.20889423]]), array([[3.54960601, 0.68939608, 6.7873254 ]]), array([[0.97206673, 0.66583257, 0.99887329]]), array([[0.58662319, 0.83898341, 0.93110208]]), array([[0.28558733]]))
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-147-0b57cb811a44> in <module>
      3 cost, cache = forward_propagation_n(X, Y, parameters)
      4 gradients = backward_propagation_n(X, Y, cache)
----> 5 difference = gradient_check_n(parameters, gradients, X, Y, 1e-7, True)
      6 assert not(type(difference) == np.ndarray), "You are not using np.linalg.norm for numerator or denominator"
      7 

<ipython-input-146-813e93f8db34> in gradient_check_n(parameters, gradients, X, Y, epsilon, print_msg)
     89         #print(J_plus[i].shape)
     90         print(J_minus[i])
---> 91         gradapprox[i] = np.subtract(J_plus[i], J_minus[i]) / (2 * epsilon)
     92 
     93         # YOUR CODE ENDS HERE

ValueError: could not broadcast input array from shape (12) into shape (1)
2

Fixed by @Jonathanlugo.

If someone else gets this error, be careful not to alter the shape of J_plus, which is initialized as:

J_plus = np.zeros((num_parameters, 1))

Think how it relates to the loop:

for i in range(num_parameters):

Good luck with the rest of the course :slight_smile:

3 Likes