Hi Sir, In the Assignment notebook shape of the attention output mentioned as (batchsize, input_seq_length,fully_connected_dim) but in the tensorflow documentation. below is description mentioned for attention output shape. what is E here ? The result of the computation, of shape (B, T, E) , where…

I don’t see batchsize, input_seq_length, or fully_conected_dim used in the notebook. Can you provide a screen capture images where you see those values being used?

It will also help if you mention which week and assignment you’re working on.

The assignment is course 5 week 4 Transformer architecture. In the below link, at the bottom shape of attention output mentioned tf.keras.layers.MultiHeadAttention | TensorFlow Core v2.8.0 MultiHeadAttention layer.

We dont know what is E means ? Because usually It should value depth as the last dimension .

Thanks for specifying which assignment you are asking about. I was looking in the Week 3 Assignment 1 notebook.

[image] Anbu: We dont know what is E means ? Since the assignment does not set the output_shape argument, it defaults to “None”. So ‘E’ should be the “query shape last dimension”.

Query Input means ? Also attention output shape last dimension should be value_depth as per lecuture, concepts and exercise 3 scaled dot product attention. can you please clarify why E ? instead of value_depth dimension

I do not know. You would have to ask the authors of the Keras documentation why they used ‘E’ for this.

Also, I think you will be interested in the discussion at this thread: [image] Understanding multi-headed attention - C5W4A1 Sequence Models This is a question about understanding the inputs /outputs of the mha layer calls in the decoder blocks that form the core of the tr…

Attention Output shape

Course Q&A Deep Learning Specialization Sequence Models

TMosh May 11, 2022, 1:47am 10

Also, I think you will be interested in the discussion at this thread:

Topic		Replies	Views
Query Input Last Dimension Sequence Models coursera-platform	10	608	May 12, 2022
C5W4 Assignment: Multi-head attention output dimension Sequence Models week-module-4 , coursera-platform	2	292	January 18, 2024
Q about keras doc of tf.keras.layers.MultiHeadAttention Sequence Models coursera-platform	6	565	July 18, 2021
Transformer: dimensions of encoder output and decoder Q matrix Sequence Models coursera-platform	1	610	April 21, 2022
W4 Assignment-Exercise6; why shape after second Add&Norm Layer is (batch_size, n_target, full_connected_dim) not (batch_size, n_target, d_model)? Sequence Models coursera-platform	2	449	October 12, 2023

Attention Output shape

Related topics