C3W2: Exercise 2,5 and error

Course Q&A Natural Language Processing NLP with Sequence Models
1

Week #2

Assignment 2 - Named Entity Recognition (NER)

Exercise 2:
image
I use the argument padding = ‘post’ I don’t know what else could be.

Exercise 5:

Create the mask, i.e., the values that will be ignored

I really don’t get it how to create the mask with the info I have.

Submit error:
Every time I try to submit my progress to see the error message so I can have more idea how to fix it I see this:
“There was a problem compiling the code from your notebook. Details:
expected ‘:’ (, line 228)” and I get 0.

2

Hi @Juan_David_Vargas_Al

This error message is a bit misleading since it “Expected” and “Got” do not match because of -1 and 0, but not because -1 are not “post”. So make sure you use pad value of -1 (and also how your label_ids are constructed).

As for Exercise 5 and the mask creation - you need to check if the labels (the y_true) do not equal -1. One way of doing it could be (y_true != -1) which would return a boolean tensor which later you would need to cast to tf.float32) and that would be your mask.

Cheers

3

Exercise 5:
image
I am getting now this error but not idea why could be

4

This unit test indicates that you did not complete the function masked_accuracy correctly.

Your function failed on 5 tests. For example, take the first check:

# true labels (y_true)
[array([[ 0,  2, -1, -1,  2]]),

# model's predictions (y_pred)
 array([[[0.12812445, 0.99904052, 0.23608898],
         [0.39658073, 0.38791074, 0.66974604],
         [0.93553907, 0.84631092, 0.31327352],
         [0.52454816, 0.44345289, 0.22957721],
         [0.53441391, 0.91396202, 0.45720481]]])]

The expected accuracy is 0.33333:

  • two predictions should not be counted because they are padding tokens (-1), so you’re left with 3 ([0,2,2]) on rows 1, 2, 5.
  • row 1 predicts index 1 (0.999) but the real label is 0, so wrong
  • row 2 predicts index 2 (0.669) and the real label is 2, so correct
  • row 5 predicts index 1 (0.913) but the real label is 2, so wrong

So the expected accuracy should be 1 / 3 = 0.3333. Your result is 1 for this case and in some other cases you even got 2 or 3 (which is impossible).

Note, to reproduce the numbers, so you can test your implementation:

np.random.seed(1)
# (y_true, y_pred)
(np.random.randint(-1, 3, size = (1,5)), np.random.rand(1,5,3))

Cheers

1 Like
5

@arvyzukai and @Juan_David_Vargas_Al ,
I had a huge difficult to implement the masked_accuracy function and I did it when I found @arvyzukai explanation (his example is better than the assignment).

Thanks a lot guys.

2 Likes
6

I agree! Thank you so much, @arvyzukai !

7

Hi @arvyukai, I still have the same problem after having done the same as you explained but I still got 6 tests passed and 3 tests failed. I am wondering if I understood correctly on computing accuracy rate as follows:

masked_acc = tf.reduce_sum(matches_true_pred)/tf.reduce_sum(y_true * mask)

Thanks

See errors:

Coursera C3W2 assignment