In C5W1, video “Language Model and Sequence Generation”, Andrew mentions that at each time step model output y (y hat) is the probability of the output given all previous time steps outputs. RNN gives the output with highest probability at each time step. Is it possible instead of getting one single output at each time step, we get the whole probability distribution of the output at each time step?
Please see return_sequences
Thanks @balaji.ambresh for your response. If I set return_sequence=True, then my RNN layer will generate the full sequence rather than only the sequence with the highest probability. correct? In the W1A2-Ex 2, we specified the probability of the output sequence (16%, 50%, 90%,…). How can I specify the probability of the output sequence in Keras RNN layer such as GRU, LSTM,… ?
I would like my Keras RNN layer (GRU, LSTM) generate output sequences with different probabilities that I specify. I would be grateful if you could advise how to do that.
Get the sequences for each timestep and then use np.random.choice to sample output at each timestep.
Edit based on @ElhamS correction:
See course 5 week 1 assignment 2 exercise 2 (sample) for an example.
I think you are referring to W1A2-Ex 2. In that exercise we built the layer from scratch. How can I do sampling in Keras built in GRU layer?
Please confirm if generate_output() in shakespeare_utils.py helps. Do check model.summary() to see where the recurrent layer is located.