C5 W4 class Encoder

Course Q&A Deep Learning Specialization Sequence Models
1

Hi, I am getting the error:

[[0 0 1 1]]
---------------------------------------------------------------------------
AssertionError                            Traceback (most recent call last)
<ipython-input-46-68aa9b8f5815> in <module>
      1 # UNIT TEST
----> 2 Encoder_test(Encoder)

~/work/W4A1/public_tests.py in Encoder_test(target)
    124                         [[-0.4612937 ,  1.0697356 , -1.4127715 ,  0.8043293 ],
    125                          [ 0.27027237,  0.28793618, -1.6370889 ,  1.0788803 ],
--> 126                          [ 1.2370994 , -1.0687275 , -0.8945037 ,  0.7261319 ]]]), "Wrong values case 1"
    127 
    128     encoderq_output = encoderq(x, True, np.array([[[[1., 1., 1.]]], [[[1., 1., 0.]]]]))

AssertionError: Wrong values case 1

when I run this in UNQ C5

# START CODE HERE
        # Pass input through the Embedding layer
        x = self.embedding(x)  # (batch_size, input_seq_len, embedding_dim)
        # Scale embedding by multiplying it by the square root of the embedding dimension
        x *= tf.math.sqrt(tf.cast(self.embedding_dim, tf.float32))
        # Add the position encoding to embedding
        x += self.pos_encoding[:, :seq_len, :]
        # Pass the encoded embedding through a dropout layer
        x = self.dropout(x, training=training)
        # Pass the output through the stack of encoding layers 
        for i in range(self.num_layers):
            x = self.enc_layers[i](x, training, mask)
        # END CODE HERE

P.S.- I am sorry if this seems like a silly mistake but I am having difficulty in this assignment and after reading many threads I am following this Transformer model for language understanding

2

I don’t see any problem with that code. Maybe the issue is in the EncoderLayer.

Be very careful with that TensorFlow tutorial. This assignment does not necessarily follow their tutorial.

3

That has all test cases passed.
The only error I am getting before this is in the get_angles part, I thought i’ll come back to it later and I guess I forgot about it
The code for it:

# START CODE HERE
    # Get i from dimension span k
    i = k//2
    # Calculate the angles using pos, i and d
    angles = pos / np.power(10000, (2 * (i // 2)) / np.float32(d))
    # print(i)
    # END CODE HERE

the error:

---------------------------------------------------------------------------
AssertionError                            Traceback (most recent call last)
<ipython-input-47-2c76efaf3fad> in <module>
      1 from public_tests import *
      2 
----> 3 get_angles_test(get_angles)
      4 
      5 # Example

~/work/W4A1/public_tests.py in get_angles_test(target)
     18     assert np.all(even_cols == odd_cols), "Submatrices of odd and even columns must be equal"
     19     limit = (position - 1) / np.power(10000,14.0/16.0)
---> 20     assert np.isclose(result[position - 1, d_model -1], limit ), f"Last value must be {limit}"
     21 
     22     print("\033[92mAll tests passed")

AssertionError: Last value must be 0.0009486832980505137
4

Why are you using “// 2” twice?

5

Omg I clearly missed that, thanks so much the UNQ C5 works fine now.
Thanks again :slight_smile: