Emojify Lab vs Jazz Music Generation Lab (LSTM)

Course Q&A Deep Learning Specialization Sequence Models
,
1

In week 1, lab 3 (jazz music generation) we used a for loop over the time steps. We explicitly got the values of a and c from the LSTM and passed it to the next time step. However, in week 2, lab 2 (emojify) we used vectorization and did not have a for loop over the time steps. Also, we did not explicitly handle a and c.

Was this done just to show us students different ways we can go about building a neural network model? Or is there a reason why the 2 RNNs were built differently?

Any insight is appreciated!

Thanks,
Adi

1 Like
2

It’s a good point that the code does look quite a bit different between the two cases, but the situations are pretty different. In the Jazz case, we have an output for each timestep that we need to deal with by generating the note and then appending it to the “tune” sequence that is the output of the model. In the Emojify case, it’s one of those situations where we take a sequence of inputs (here the “timesteps” are individual words in the input sentence) and we only produce a single output, being the (we hope) appropriate emoji to represent the sentiment of the sentence.

Prof Ng did give lots of examples in the lectures of different types of RNNs. If you take a look at the TF docpage for the Keras LSTM layer, you’ll see that it can handle the a and c states by generating a_0 and c_0 as zero tensors of the appropriate shape by default. They don’t explicitly say it, but apparently the passing of the state between the timesteps can also happen automatically. It might be an interesting exercise to see if we could rewrite the Jazz model to use that “implicit state” technique as well.

Thanks for pointing this out. I don’t think what I said above really counts as an “answer” to your actual question, but at least some food for thought. The TF/Keras website has a lot of tutorials as well as the low level documentation for the APIs. It might be a worthwhile use of some time to go have a look for any LSTM tutorials they have. E.g. here is their top level tutorial about using RNNs.

2 Likes
3

Thanks @paulinpaloalto. This answers my question!

1 Like