Transformer Architecture: Fully connected Dimension Exercise 4

Course Q&A Deep Learning Specialization Sequence Models
1

Hi Mentor,

In the encoder layer, exercise 4, why input x having shape (Batchsize,input_seq_len,fully_connected_dimns) ?

Because input x should have embedding dimension right ?

x – Tensor of shape (batch_size, input_seq_len, fully_connected_dim)

2

Hi @Anbu,
It’s always better if you mention Week and Assignment numbers in your title, so that the mentors and the fellow learners can easily understand and learn from your query. You can rename your title using the little pencil icon next to your title.

Regards,
Elemento

3

Okay sir,

This is trasformer architecture programming assignment. Exercise 4

4

Hi @Anbu,
You don’t need to refer to me as “Sir”. I am also a fellow learner, just like you :smile:. As for your query, I have requested the other mentors to take a look at it once, as I am not very well-versed with Transformers.

Regards,
Elemento

5

Hi Anbu,

Yes, this should be (batchsize, input_seq_len, embedding_dim). Thanks for noticing!