DLS C5 Week 1 Ex7 - LSTM Cell Backward

Course Q&A Deep Learning Specialization Sequence Models
1

In the notes, equations 19 and 21 are the same (see attached screenshot). IS this correct?

image
image1277×620 54.1 KB

This is my output:
gradients[“dxt”][1][2] = 0.3695862892930779
gradients[“dxt”].shape = (8, 10)
gradients[“da_prev”][2][3] = -0.06396214197109236
gradients[“da_prev”].shape = (8, 10)
gradients[“dc_prev”][2][3] = 0.7975220387970015
gradients[“dc_prev”].shape = (5, 10)
gradients[“dWf”][3][1] = -0.1479548381644968
gradients[“dWf”].shape = (5, 8)
gradients[“dWi”][1][2] = 1.0574980552259903
gradients[“dWi”].shape = (5, 8)
gradients[“dWc”][3][1] = 2.3045621636876668
gradients[“dWc”].shape = (5, 8)
gradients[“dWo”][1][2] = 0.3313115952892109
gradients[“dWo”].shape = (5, 8)
gradients[“dbf”][4] = [0.18864637]
gradients[“dbf”].shape = (5, 1)
gradients[“dbi”][4] = [-0.40142491]
gradients[“dbi”].shape = (5, 1)
gradients[“dbc”][4] = [0.25587763]
gradients[“dbc”].shape = (5, 1)
gradients[“dbo”][4] = [0.13893342]
gradients[“dbo”].shape = (5, 1)

I think my weights aren’t set correctly, but not sure how to do this.

2

No, they are not the same. We have this in the notebook:

Here, to account for concatenation, the weights for equations 19 are the first n_a, (i.e. W_f = W_f[:,:n_a] etc…) …where the weights for equation 21 are from n_a to the end, (i.e. W_f = W_f[:,n_a:] etc…).

3

Hello @Chris_Dandre,

See the two lines underlined in green. They talk about the difference.

image
image704×332 52.4 KB

Such difference comes from the following equation:

image

See that only a part of W_f has been multiplied to a^{<t-1>}, and only the other part of W_f has been multiplied to x^{<t>}.

Good luck on getting the correct results!

Cheers,
Raymond

4

Thanks everyone - I fixed up the weights and achieved the correct output.