W4 Exercise 7: Decoder - AssertionError: Wrong values in outd

In the Week 4 Transformer excercise, I am stuck in Exercise 7:

AssertionError                            Traceback (most recent call last)
<ipython-input-30-dd4b65c051b4> in <module>
      1 # UNIT TEST
----> 2 Decoder_test(Decoder, create_look_ahead_mask, create_padding_mask)

~/work/W4A1/public_tests.py in Decoder_test(target, create_look_ahead_mask, create_padding_mask)
    221     assert tf.is_tensor(outd), "Wrong type for outd. It must be a dict"
    222     assert np.allclose(tf.shape(outd), tf.shape(encoderq_output)), f"Wrong shape. We expected { tf.shape(encoderq_output)}"
--> 223     assert np.allclose(outd[1, 1], [-0.2715261, -0.5606001, -0.861783, 1.69390933]), "Wrong values in outd"
    224 
    225     keys = list(att_weights.keys())

AssertionError: Wrong values in outd

This seems to be a point where quite a few people were already stuck. I tried to go through a lot of the posts but I could not resolve the issue.

  • All units tests of Exercises 1-6 pass
  • In Exercise 6 and 7, I am sure that I use the argument training=training in the dropout layers

Any other hints on how to find the problem? Thanks a lot for your help.

In the Decoder test case, the variable “outd” refers to the first value returned by the Decoder class.

That is, the ‘x’ variable.

Areas to check:
That you are using self.embedding(…).
That you are using tf.math.sqrt(…), and that you’re using the correct tf.cast() arguments.
That you are using the correct slicing in the call to self.pos_encoding(…).
That you are using training=training in the self.dropout(…).
There are a lot of arguments for the self.dec_layers(…). Be sure you have the correct masks in the correct order.

That you are using self.embedding(…).

yes, I have self.embedding(x)

That you are using tf.math.sqrt(…), and that you’re using the correct tf.cast() arguments.

I think so: x *= tf.math.sqrt(tf.cast(self.embedding_dim, tf.float32))

That you are using the correct slicing in the call to self.pos_encoding(…).

From the instructions, I understood pos_encoding[:, seq_len, :]

That you are using training=training in the self.dropout(…).

Confirmed. I used training=training in all lines with self.dropout in Ex 6 and 7. In Ex6, I also used it in the calls of self.mha1 and self.mha2.

There are a lot of arguments for the self.dec_layers(…). Be sure you have the correct masks in the correct order.

Yes, I double-checked against the signature of the call method implemented in Exercise 6: call(self, x, enc_output, training, look_ahead_mask, padding_mask) and wrote self.dec_layers[i](x, enc_output, training, look_ahead_mask, padding_mask) in Ex.7

I forgot to say thanks for the quick reply!

Uff, I forgot a colon in [:, seq_len, :]

[:, :seq_len, :] fixed it…

1 Like