How do RNNs manage variable lengths?

Course Q&A Deep Learning Specialization Sequence Models
1

I just went through the 1st week of the sequence models course and I was wondering, how do RNNs manage different lengths of an input sentence?
For example: in the sentence Teddy Roosevelt → president and Teddy bears → sale, the 2 sentences were of different lengths, how exactly does the same RNN model manage predictions for both?
Do you pad the sentences? In which case the RNN model must be built to have a large number of sequential cells to handle to longest of sentences or there is some other way to do that?
Or you don’t pad the sentences and there is some other way?
Please help me out with this. Thanks in advance.

2

I believe Andrew discusses this later in the course. There are two immediate options:

  • Pad all the sentences to the same length. This makes the RNN quite heavy to train.
  • Truncate the word sequences to a common length. This is particularly effective if you’re trying to understand the context of a sequence of words.
1 Like