Queries regarding backpropagation in RNNS

Course Q&A Deep Learning Specialization Sequence Models
1

Can anyone please help me with my queries:

  1. Even in DNN back propagation and here as well what always confused me how to decide on what to use during multiplication like element wise or dot product?(is it has to deal with derivatives of matrix using Jacobian matrix…can you please explain in a detailed and intuitive way)
    image
    image1202Γ—617 97.9 KB

    image
    image1142Γ—530 30 KB

    2.It is regarding the dz that was mentioned in the lab notebook, isnt the dz same as dtanh?
    Z=π‘Šπ‘Žπ‘₯π‘₯βŸ¨π‘‘βŸ©+π‘Šπ‘Žπ‘Žπ‘ŽβŸ¨π‘‘βˆ’1⟩+π‘π‘Ž , then dJ/dZ=π‘‘π‘Žπ‘›π‘’π‘₯π‘‘βˆ—(1βˆ’tanh2(π‘Šπ‘Žπ‘₯π‘₯βŸ¨π‘‘βŸ©+π‘Šπ‘Žπ‘Žπ‘ŽβŸ¨π‘‘βˆ’1⟩+π‘π‘Ž))?
2

At the risk of stating the obvious, we’re taking derivatives of functions, so it depends on what the functions are, right? Some of them are β€œelementwise” operations (e.g. the application of an activation function) and some of them involve dot products.

Matrix calculus is really beyond the scope of these courses, but here’s a thread with links to background material if you want to go deeper.

Note that in all cases beyond Course 2, we are using TF so we don’t actually have to deal with any of the derivatives ourselves: backprop is all just magically handled for us β€œunder the covers” by TF (or PyTorch or whatever your ML platform of choice is).