RNN Model Purpose

In the “Language Model and Sequence Generation” lecture, the RNN model is mentioned. What is the purpose of the RNN model? Why are we inputting the sentence “Cats average 15 hours of sleep a day.” into the RNN? Are we trying to train the RNN with the sentence “Cats average 15 hours of sleep a day.” being one training example? Or are we inferencing/predicting using the RNN with the above sentence?

Another more fundamental question: Why are we wanting the RNN to predict the next word in the above sentence? What sort of black magic are we trying to accomplish?

Thanks in advance.

DLS Course 5 is about “Sequence Models”, which are models specifically designed to process sequences of inputs. There are lots of different types of sequential inputs and different cases for what you want to derive from a sequence of inputs. Some examples are language translation (taking an input sentence in one language and outputting the equivalent sentence in a different language), sentiment classification (take a sentence as input and predict whether it is positive or negative), music generation.

My suggestion is that you “hold that thought” and just listen to all the lectures that Prof Ng gives in Week 1 of DLS Course 5.

@hungng777 Personally, I think, of all the things we do, despite its majesty, language actually turns out to be one of the easier ones (on the surface) to solve. I mean I used to teach ESL, so you expect to have subject, object, verb, and generally you expect to see them in that order.

This is highly ‘temporal’, thus the RNN, but also contextual. I mean if someone were to tell you “Sleep a day, 15 hours, Cats!” it would sound really strange, but you could still understand what they are trying to say.

However you can’t mess with this order too much, because if you said “Sleep, Cats, 15, average, hours a day”… It starts to make no sense what you are talking about-- Does ‘15’ relate to the number of the cats ? Their age ? How much they sleep ?

I mean, we presume we do not speak (or think) in complete riddles thus as a starting point an RNN makes sense.