Need help with DL specialization Course 5 Lab1 rnn_backward function

Course Q&A Deep Learning Specialization Sequence Models
1

Hello,
My result for the rnn_backward function is not matching with the expected output.

Below is the code from the time step iteration, what am I doing wrong here?
[snippet removed by mentor]

2

cache is a tuple of size 4 where the 0th index contains a_next. Please read the signature of rnn_cell_backward and fix the 1st parameter in the call.

3

Hello Balaji,
Thank you for your response.

I still have some doubts regd fixing the 1st parameter.

Given rnn_cell_backward signature as:
rnn_cell_backward(da_next, cache)
da_next – Gradient of loss with respect to next hidden state
cache – python dictionary containing useful values (output of rnn_cell_forward())

Now coming to the function call,
As per the below image, da_next is defined as follows:

image
image1035×626 62.1 KB

i.e. da_next = da[:, :, t] + a_next

So as per my understanding, I came up with the below code for da_next
Where, a_next = caches[t][0], cache at time step ‘t’ and indexing 0 to get ‘a_next’ at time step ‘t’

Code:
gradients = rnn_cell_backward( da[:,:,t] + caches[t][0], caches[t])

Is my understanding correct?
Am I still missing something here?

Regards,
Ankit S

4

Please refer the markdown for the exercise:

  • Note that this notebook does not implement the backward path from the Loss ‘J’ backwards to ‘a’.
    • This would have included the dense layer and softmax which are a part of the forward path.
    • This is assumed to be calculated elsewhere and the result passed to rnn_backward in ‘da’.

Going back to the picture, for timestep T_x - 1, the value of {da}_{next} is equal to da for the last timestep (based on the last point in the note). Computing the gradients using rnn_cell_backward will return {da}_{prev} which corresponds to the value at left side of he figure you uploaded.

At T_x - 2, use da for that timestep and the above value to compute gradient of the parameters via rnn_cell_backward. Keep the process going till you iterate over all timesteps.

Hope this helps.

Don’t forget to remove code from your replies on this topic since posting code in public could get your account suspended. It’s okay to share images / stacktraces and images though.

5

Hello Balaji,
Thanks a lot for your advice, I was able to solve this.

Best Regards,
Ankit S