C5_W4_A1 Exercise 5

Course Q&A Deep Learning Specialization Sequence Models
1

reading the instruction, what is the meaning is step 3 of self.pos_encoding [:, :seq_len, :] t

in particular, [:, :seq_len, :]

Exercise 5 - Encoder

Complete the Encoder() function using the call() method to embed your input, add positional encoding, and implement multiple encoder layers.

In this exercise, you will initialize your Encoder with an Embedding layer, positional encoding, and multiple EncoderLayers. Your call() method will perform the following steps:

  1. Pass your input through the Embedding layer.
  2. Scale your embedding by multiplying it by the square root of your embedding dimension. Remember to cast the embedding dimension to data type tf.float32 before computing the square root.
  3. Add the position encoding: self.pos_encoding [:, :seq_len, :] to your embedding.
2

Hi there, the output of positional_encoding() is of shape [1,maximum_position_encoding, self.embedding_dim], thus :seq_len is needed to ensure match of size.