I can't get past the W1 assignment

Course Q&A Natural Language Processing NLP with Sequence Models
1

Hello everyone.

I have reached Exercise 06, and results seem to drift from the expected results of quite a bit despite passing all unit tests within the notebook.

One thing that struck my eye was the training loop results:

1° training loop:

Step    310: Ran 10 train steps in 5.57 secs
Step    310: train CrossEntropyLoss |  5.54518604
Step    310: eval  CrossEntropyLoss |  5.54518461
Step    310: eval          Accuracy |  4.00000000

which is very weird. Even when I go through the training the unit test keeps on telling me all tests are passed.

Also, when I try to turn probabilites into category predictions, I get very weird numbers

|Neg log prob [-0.6935647 -0.6927298]|Pos log prob [-0.6935647 -0.6927298]| is positive? [False False]| actual 1|
|---|---|---|---|
|Neg log prob [-0.6941879 -0.6921075]|Pos log prob [-0.6941879 -0.6921075]| is positive? [False False]| actual 1|
|Neg log prob [-0.6915502  -0.69474685]|Pos log prob [-0.6915502  -0.69474685]| is positive? [False False]| actual 1|

Can someone help me understand what is going on?

Thanks

2

Hi @Andrea_D_Agostino,

Can you share your lab ID with me ? In the assignment, when you click the top right “Help” button, a panel will open and your lab ID will be shown at the bottom.

I shall take a look.

When you reply back, kindly tag me in the post so that I’m notified.

Thanks,
Mubsi

3

Hello @Mubsi ,

i have sent you my lab ID through a message here on the platform. Let me know if you need anything else.

Regards,

4

Hi @Andrea_D_Agostino,

Yes, I received it. Check and let me know if your notebook is running fine now.

Cheers,
Mubsi

5

Yes, it does! Thank you @Mubsi. What was wrong? I’d like to know if any mistakes were present and understand them.

6

It same as my notebook on training_model(). Unittest output on my 100 step after loop

Step    100: Ran 10 train steps in 0.94 secs
Step    100: train CrossEntropyLoss |  5.54543543
Step    100: eval  CrossEntropyLoss |  5.54539061
Step    100: eval          Accuracy |  4.00000000

But all tests passed. :sweat_smile:

7

@Mubsi I am facing the same issue: accuracy don’t converge to 1 and stays the same (8 for some reason). I refreshed the lab and rebooted the server (from Help menu), but it did not affect results.

What should I do?

8

It seems that I have found a problem. Please let me know if my finding is correct.

I have debugged this locally (e.g. running code on my local machine instead of Coursera Labs).
I have reproduced described behavior locally which means that either I made some coding error or somehow misconfigured the model.

I went through the code and it does not seem to be coding error.
I have checked the model and I can see that weights of Dense layer are changing between runs.
However, neither loss nor accuracy is changing.

The only thing that left to check was our train and eval tasks.

I digged through TRAX documentation and found that lab code uses deprecated features of trax (tl.CrossEntropyLoss() and tl.Accuracy()). I have swapped them with tl.WeightedCategoryCrossEntropy() and tl.WeightedCategoryAccuracy() accordingly and voila:

Step    100: Ran 10 train steps in 1.01 secs
Step    100: train WeightedCategoryCrossEntropy |  0.03101570
Step    100: eval  WeightedCategoryCrossEntropy |  0.01737435
Step    100: eval      WeightedCategoryAccuracy |  1.00000000

@Mubsi please could you confirm if that’s the root cause of the observed problem or not?

P.S. The lab has hard dependency on NLTK 4.5 otherwise unit tests starts failing: process_tweet is producing different results and in turn vocabulary is different.

9

Hi @kos,

Can you share your lab ID with me ? In the assignment, when you click the top right “Help” button, a panel will open and your lab ID will be shown at the bottom.

I shall take a look.

When you reply back, kindly tag me in the post so that I’m notified.

Thanks,
Mubsi

10

@Mubsi my lab id is zvgfitme.

11

After introduction of WeightedCategoryAccuracy, unit test started to fail (unexpected loss function).
So, @Mubsi, your guidance here will be highly appreciated.

12

Hi @kos,

it is getting pretty late where I am from. I shall look at this tomorrow.

Best,
Mubsi

13

Hi @kos,

You changed the original code in the Section 4.1, from metrics=[tl.CrossEntropyLoss(), tl.Accuracy()], to metrics=[tl.WeightedCategoryCrossEntropy(), tl.WeightedCategoryAccuracy()]. This was causing the unit tests to fail afterwords.

There was clear warning on top of that code cell, # PLEASE, DO NOT MODIFY OR DELETE THIS CELL.

Please refrain from editing the original code, as it cause the unit tests and the autograder to fail. Once you have submitted your assignment, and gotten the desired grade, feel free to experiment with the assignment.

Cheers,
Mubsi

14

I have root caused the problem (accuracy is not converging to one) to the way Mean layer is created: no need for keepdims, just axis=1.

15

Hello @Mubsi I’m facing the same issue as the other folks here (i.e. no convergence in accuracy, accuracy stays at 4., and getting weird log probs, even though I’m passing all the tests). My lab ID = teknlhiw
Thank you!

16

Hi @renatem,

I have fixed your mistakes in C3_W1_Assignment_v2.ipynb. The training parts now run. I noticed you added extra print statements in the notebook, make sure to remove those and change the assignment name to its original before submitting it.

Cheers,
Mubsi

17

Hello @Mubsi I am having the same problem. The accuracy stays at 0.5 but I also passed all the tests. My lab ID = fsaqydyb

Thank you very much in advance!

18

Hi @ywc,

I made small changes in your Ex 1, 2 & 3. It seems to be working now.

Cheers,
Mubsi

19

@Mubsi I am having the same issue, can you take a look please? my labID is ogeiahlx

20

did you find an answer to this issue ?