Problem with understanding tl.Serial

Course Q&A Natural Language Processing NLP with Sequence Models
1

I have one problem with understanding tl.Serial and how LSTM and Dense layers are connected. This is regarding C3W3 Assignment. I had the same question for Week 2 with GRU but I thought it would be explained later on. In the code part we see:

model = tl.Serial(
tl.Embedding(vocab_size, d_model), # Embedding layer
tl.LSTM(d_model), # LSTM layer
tl.Dense(len(tags)), # Dense layer with len(tags) units
tl.LogSoftmax() # LogSoftmax layer
)

Where we chain LSTM later with dense layer. Since there are 3 outputs per one LSTM cell, hidden state c, h and y, when we connect Dense Layer which one of these are fed in the network? I think the y output is hidden to Dense Layer. So LSTM predicts whether there is a named entity in the sentence and then Dense with LogSoftmax figures out which named entity it is. Did I get it correctly or am I terribly wrong?

2

The default value of return_state parameter to LSTM is False. As a result, only the output is returned.

3

So only the output is returned and then those outputs (y’s of each LSTM cell) are fed into Dense layer right? Kind of like on this picture:

lstm_plus_dense
lstm_plus_dense1516×740 18.5 KB

Sorry for bad drawing, I just want to make sure I understand it exactly the way it is.

4

Please see this thread to understand the difference between an LSTM cell and an LSTM layer. That should help revise the picture.