Derivation of backpropagation of RNN

Course Q&A Deep Learning Specialization Sequence Models
1

Hi all!

I’m currently trying to derive the equations that are presented in the ungraded section of the first programming assignment for Course 5 Week 1. I’m struggling to come up with the derivations on my own. Is there an additional resource that steps through the derivation? I think this will be beneficial to my understanding of how the RNN backprop works.

Thanks again!

Ricardo

2

Here’s a thread with some links, but I think they only cover the derivations for FC nets and ConvNets. The thread links to a bibliography thread which points to books like Goodfellow et al which might cover this, but I have not actually looked at it to check that.

Note that Stanford has been very generous about putting material from their ML related courses online. The previous linked thread points to CS231, which covers ConvNets. You might check CS224n, which is their intro NLP course. That would involve RNNs, so might include those derivations.

3

Maybe you can post a picture of your derivations with comments on where you feel stuck and I (or we) can see if we can help.

1 Like