Why Sampling Novel Sequences?

I’m still unclear on the actual purpose of Sampling novel sequences. Why we need to do it ? What we need to do in order to process this?

Hey @Akshay_Singh_Thakur,
Welcome to the community. Apologies for the delayed response. Let’s say you have trained a RNN which can create new music tracks. For this model, in the training phase, you will be feeding in notes of the existing music tracks, but as you might have learnt in the lecture videos, for inference, we only need to feed in a single starting input (for instance, 0 vector), and then iteratively, we can feed in the output of the previous LSTM cell as the input to the next LSTM cell.

And this is what the concept of Sampling Novel Sequences is all about. In other words, this concept is used to perform inference on sequence models which doesn’t require any input, for instance, the one I described here. And based on these sampled outputs, you can evaluate your sequence model. I hope this helps, still if you have any further query, please feel free to ask.

Regards,
Elemento

1 Like

Thanks, this helps me.

Sequence model inference is very different to what we learned in previous lectures, in which we simply called a “predict” method.

In sampling novel sequence, Andrew said that the first layer is initialised with “zero” vector - so wouldn’t the sampling be same for a given model for a given dictionary every time as the probabilities of finding a word doesn’t change (as there is no external input to start with?) or Am I confused with what has been said?

It’s a good point, but there are ways to introduce randomness in which token you select at a given iteration, even if the initialization is deterministic. You’ll see an example of how to do that in the Dinosaur Name Assignment, which is the second assignment in DLS C5 W1. Stay tuned for that! :nerd_face:

1 Like