#C5 3 How to obtain the probability value in Error Analysis in Beam Search

Course Q&A Deep Learning Specialization Sequence Models
,
1

Hi,
I have a question after watching the video for Error Analysis in Beam Search.
I learned to decide error is more on RNN or Beam Search by comparing P(y*|x) and P(y_hat|x). However, I did not get how to get the P value.

2

Output of each timestep of an RNN corresponds to the probability you’re looking for.

See image from assignment:

image
image1355×542 69.4 KB

>>> import tensorflow as tf
>>> inputs = tf.random.normal([32, 10, 8])
>>> lstm = tf.keras.layers.LSTM(4)
>>> output = lstm(inputs)
>>> output.shape
TensorShape([32, 4])
>>> probabilities = tf.math.softmax(output)
>>> # Sanity check
>>> tf.math.reduce_sum(probabilities, axis=-1)
<tf.Tensor: shape=(32,), dtype=float32, numpy=
array([1.        , 1.        , 1.        , 1.        , 1.        ,
       0.9999999 , 0.9999999 , 1.0000001 , 0.9999998 , 0.99999994,
       0.9999999 , 1.0000001 , 1.        , 1.        , 0.99999994,
       0.9999999 , 1.        , 1.        , 1.        , 1.        ,
       1.        , 1.        , 1.        , 1.        , 0.99999994,
       0.9999999 , 1.        , 1.        , 1.        , 1.        ,
       1.        , 0.9999999 ], dtype=float32)>
>>>