DLS course 5 LSTM backward ungraded hw

Hello,

For the ungraded section of the first assignment I was getting results that didn’t match the expected results. I’m not getting the same results as any of the other people I’ve noticed post on here. I went back through the equations for dot, dcct, dit, and dft and made a couple small corrections and was able to get my results for the first few gradients to be quite similar to expected results. But the last 4 are still a bit off.

These results come from summing all of the ‘m’ examples. An example of one I did is here:
{Moderator’s Edit: Solution Code Removed}

If this is right then perhaps my dWf was calculated incorrectly.

This was simply a dot product of dft with the transposed concatenation of aprev and xt.
{Moderator’s Edit: Solution Code Removed}

If I go back any further I’m back at the equations that I’ve picked over a bunch. Here is the specific one for dft for example.
{Moderator’s Edit: Solution Code Removed}

At this point I suppose it could back track even further into these other variables but at that point I feel like a mistake even earlier would have propagated throughout all the calculations I’ve done.
Thanks

gradients[“dbf”][4] = [1.23112653]
gradients[“dbf”].shape = (5, 1)
gradients[“dbi”][4] = [-1.22362657]
gradients[“dbi”].shape = (5, 1)
gradients[“dbc”][4] = [1.13641227]
gradients[“dbc”].shape = (5, 1)
gradients[“dbo”][4] = [-0.18974623]
gradients[“dbo”].shape = (5, 1)

|gradients[dbf][4] = |[ 0.18864637]|
|gradients[dbf].shape = |(5, 1)|
|gradients[dbi][4] = |[-0.40142491]|
|gradients[dbi].shape = |(5, 1)|
|gradients[dbc][4] = |[ 0.25587763]|
|gradients[dbc].shape = |(5, 1)|
|gradients[dbo][4] = |[ 0.13893342]|
|gradients[dbo].shape = |(5, 1) |

Hey @zac_builta,
For computing dbf, you have summed over dWf, but the equation for this is as follows:

db_f = \sum_{batch} d\gamma_f^{<t>}

Additionally, it’s written that d\gamma_f^{<t>} is represented by dft. I suppose you would have figured out your error in this.

Cheers,
Elemento

1 Like

Thank you, I appreciate the help. That worked.