Hi everyone,
I have a small question about how our definition of our model in EmojifyV2 relates to the input and output size Tx and Ty.
One thing I would like to understand from this picture:
What happens if the Tx is smaller than the number of units as defined by the number of units ?
Does Keras automatically pads it ?
I can’t really wrap my head around about how Keras handles the different sizes of inputs.
We fix the Tx size to the max length of our sentences, then we pad up to that length with zeros. We only care about the last output from the second layer of LSTM cells, so we have Ty = 1 here (it is a classification problem). Keras does not automatically pad, but we do it in sentences_to_indices(X, word_to_index, max_len). How? We initialize X_indices as a numpy matrix of zeros and the correct shape. Now we have the problem that some of our sentences are padded, we can use a padding mask to skip computations for padded entries. In the lab, we don’t skip padded entries, but use them in our model. If you want to skip padded entries, you could pass mask_zero=True to the embedding layer. You might have better performance if you do. Please try it out and compare.
@jonaslalin Thank you so much for the answer and the reading.
I’ve had a look and I already know that I’ll need to dig deeper in this topic.
