Hello, I finished the quiz but still don’t understand one question about sequence sampling/generating.
When sampling/generating random sentences, output probabilities of a trained RNN at each time step are used to pick the highest probability word for that time step.
Is there anything wrong with that assessment?
Thanks!
There are lots of different versions of the various quiz questions and I’m not sure I understand the exact question from what you have said here. So I will have to guess a bit, but I think that answer is probably wrong. The point is that RNNs are trained to output not probabilities but the actual type of values that you want. E.g. if you’re building an RNN to guess the next word in a sentence, the output is not a probability, but an actual word (typically expressed as either an index into the vocabulary or the “one hot” equivalent of that).
Another possibility is that if you always pick the highest probability, you won’t get any variation in the output. So you might randomly select using the outputs as a distribution.
The idea being that the question asks about “random sentences”, not just “what is the single most likely sentence”.
Thanks! I think both explanation make sense @TMosh @paulinpaloalto