W4E8- confusing about the argument called output_setence

Course Q&A Deep Learning Specialization Sequence Models
1

def call(self, input_sentence, output_sentence, training, enc_padding_mask, look_ahead_mask, dec_padding_mask):

I am kind of confused about the argument here, For the transformer exercise, there is an argument called output_sentence, but I don’t know where to put it, is it a part in the decoder? but We already have an input_sentence which is supposed to be the same representation as the output_sentence? or did I miss something?

2

“final_output” is the prediction for the next output in the sequence.

3

I think this assignment makes a bit more sense if you continue with the two ungraded labs.

4

Here is an overview.

image
image2746×1988 382 KB

“input_sentence” is a source to be translated, and fed into the Encoder. In this picture, it is French sentence.
“output_sentence” is a sentence of “Ground Truth” to be used for training in the Decoder. In this picture, it is English sentence.

In the Decoder, an overview of a process flow is;

  1. create a self-attention by using “output_sentence” followed by normalization
  2. calculate similarity of query (Q) by using output (V,K), which is “enc_output”, from the Encoder
  3. find the best word sequence to be translated.

Hope this helps.

5

One addition about the behavior at the ‘prediction time’. In the training time, we feed “grand truth” to the Decoder. In the prediction (inference) time, output_sentence is actually output from the Decoder. When the Decoder predicts the translated word at time t, the Decoder refers a past sequence of “translated” words until time t-1. In the other words, at time t, output_sequence is actual output sequence from Decoder at time t-1.
Again, this is for the prediction time. Hope this clarifies behaviors at the training time and the prediction time.

6

That makes it much clear! Thanks for your help!