C5 W4 Exercise 5 - Encoder "Wrong values case 3"

Course Q&A Deep Learning Specialization Sequence Models
1

HI,

I have been spending few days figuring out the problem.

Looking at the test case,
training=False
mask = [[[[1., 1., 1.]]], [[[1., 1., 0.]]]]

training parameter is already provided to the dropout layer. Is it something to do with mask? Pls kindly provide guidance

Error msg detail from the unit test
~/work/W4A1/public_tests.py in Encoder_test(target)
142 [[-0.01885331, 0.8866553 , -1.624897 , 0.75709504],
143 [ 0.4165045 , 0.27912217, -1.6719477 , 0.97632086],
→ 144 [ 0.71298015, -0.7565592 , -1.1861688 , 1.2297478 ]]]), “Wrong values case 3”
145
146 print(“\033[92mAll tests passed”)

2

My first guess is that there could be a problem in how you are using the training parameter in either the Encoder() class or the EncoderLayer() class - especially with regard to the dropout_ffn() function.

4

the unit test of EncoderLayer_test(EncoderLayer) has passed. It seems to me it has verified the setting of training parameter for dropout_ffn(). Thank you.

5

Can someone shed some light for me? All unit tests in the assignment have passed except this particular part in this exercise. It may just be a simple mistake that I am just in deadloop. Thanks in advance

6

Maybe this guide help.

7

Hello Kenny,

If possible can you share the screenshot of the complete error you got.

Regards
DP

8

Hello Deepti,

Below is the compolete error

AssertionError Traceback (most recent call last)
in
1 # UNIT TEST
----> 2 Encoder_test(Encoder)

~/work/W4A1/public_tests.py in Encoder_test(target)
142 [[-0.01885331, 0.8866553 , -1.624897 , 0.75709504],
143 [ 0.4165045 , 0.27912217, -1.6719477 , 0.97632086],
→ 144 [ 0.71298015, -0.7565592 , -1.1861688 , 1.2297478 ]]]), “Wrong values case 3”
145
146 print(“\033[92mAll tests passed”)

AssertionError: Wrong values case 3

9

Hello Kenny,

There is error in the encoder grader cell. Assertion is a programming concept that helps us to test an expression. If the expression returns true, the control of the program is moved to the next line. But if the expression returns false, an AssertionError is generated by Python.

If unable to find, send the encoder codes via personal DM. Click on the name and then message.

Note in your case the values are not matching for the last row based on grader codes.

Regards
DP

10

Hello Kenny,

you code for below statement is incomplete/incorrect, your encoding size (d) also need to initialised with np.newaxis but not same as positions.

initialize a matrix angle_rads of all the angles

Regards
DP

12

@kenny_chan, please do not share your code on the forum. That’s not allowed by the Code of Conduct.

13

Yes, that statement is correct.

14

I noticed in the previous grader cell angle code used by you is different but you have got the same result. Use np.power rather than mathematical equation.

15

Sorry. I have deleted the msg. I didn’t mean to. I thought i was still direct message to Deepti.

16

It still has the same problem. I’ll DM you my latest notebook separately

17

Kenny,

Kindly share screenshot of the error now you are getting.

Regards
DP

18

Hi Deepti,

It is same error. It is in the notebook which I DM you. Otherwise, I can send you the screenshot (basically it is the same) when I am home

19

Kenny,

The updated notebook where you used np.power is correct but requires correction with how it is implemented.

Please check you DM

20

Hello Kenny,

There are errors in other grader cells too, causing the current error you are getting. I will go one by one.

  1. ERROR IN UNQ_C3 (UNIQUE CELL IDENTIFIER, DO NOT EDIT)
    GRADED FUNCTION scaled_dot_product_attention
    def scaled_dot_product_attention(q, k, v, mask):

a. matmul_qk (incorrect code do not transpose k, hint refer the link given below to do the correction)
b. scale matmul_qk (incorrect code for both dk and scaled attention_logits) —incorrect code, refer the link given below
c. softmax is normalized on the last axis (seq_len_k) so that the scores
# add up to 1. ( this code is written correctly but is missing axis)

kindly refer the below page for doing the corrections
https://www.tensorflow.org/api_docs/python/tf/linalg/matmul

  1. ERROR IN UNQ_C4 (UNIQUE CELL IDENTIFIER, DO NOT EDIT)
    GRADED FUNCTION EncoderLayer
    class EncoderLayer(tf.keras.layers.Layer):

a. INCORRECT CODE (CLEARLY MENTIONS NOT TO USE TRAINING BUT YOU INCLUDED TRAINING INTO YOUR CODE)
calculate self-attention using mha(~1 line).
Dropout is added by Keras automatically if the dropout parameter is non-zero during training
b. REMOVE THIS PART OF THE CODE [0], training=training from skip_x_attention code
c. Kindly use training=training only where it is mentioned to use. the first fan_output doesn’t require you to pass training=training but the next step requires you to use.
d. Again training=training is not required for encoder_layer_output.

  1. ERROR IN UNQ_C5 (UNIQUE CELL IDENTIFIER, DO NOT EDIT)
    GRADED FUNCTION
    class Encoder(tf.keras.layers.Layer):

a. for the below code use tf.cast rather than tf.constant(this is mentioned before the grader cell)
Scale embedding by multiplying it by the square root of the embedding dimension

Do these corrections.

Let me know once your issue is resolved.

Regards
DP

1 Like
21

I have tried to correct the codes as you suggested but problem still persists. Frankly, I don’t quite get what’s wrong with the code for both dk and scaled attention_logits. But I corrected it to use tensorflow syntax. Again, all test have passed except the persistent wrong values case 3. Pls kindly further advise. I DM you my revised notebook. Thank you

22

I didn’t get any updated notebook Kenny.