C3W3 Assignment - IOU doesn't go beyond 0

I have been training the model with different hyperparameters but the IOU and dice score do not go above 0, although the training accuracy and validation accuracy increase while training. Please help.

What kind of accuracies do you get?, this should not be happening because you are not filling up any code for the class wise metrics.

These are my training results:

Epoch 1/10
125/125 [==============================] - 16s 50ms/step - loss: 0.6785 - accuracy: 0.2812 - val_loss: 0.6647 - val_accuracy: 0.7211
Epoch 2/10
125/125 [==============================] - 5s 43ms/step - loss: 0.6514 - accuracy: 0.6234 - val_loss: 0.6379 - val_accuracy: 0.8617
Epoch 3/10
125/125 [==============================] - 6s 48ms/step - loss: 0.6255 - accuracy: 0.8203 - val_loss: 0.6127 - val_accuracy: 0.9330
Epoch 4/10
125/125 [==============================] - 5s 43ms/step - loss: 0.6008 - accuracy: 0.9007 - val_loss: 0.5887 - val_accuracy: 0.9407
Epoch 5/10
125/125 [==============================] - 5s 43ms/step - loss: 0.5771 - accuracy: 0.9308 - val_loss: 0.5656 - val_accuracy: 0.9411
Epoch 6/10
125/125 [==============================] - 6s 49ms/step - loss: 0.5543 - accuracy: 0.9388 - val_loss: 0.5432 - val_accuracy: 0.9416
Epoch 7/10
125/125 [==============================] - 5s 43ms/step - loss: 0.5322 - accuracy: 0.9411 - val_loss: 0.5216 - val_accuracy: 0.9422
Epoch 8/10
125/125 [==============================] - 6s 49ms/step - loss: 0.5109 - accuracy: 0.9420 - val_loss: 0.5005 - val_accuracy: 0.9426
Epoch 9/10
125/125 [==============================] - 6s 49ms/step - loss: 0.4901 - accuracy: 0.9423 - val_loss: 0.4800 - val_accuracy: 0.9427
Epoch 10/10
125/125 [==============================] - 6s 46ms/step - loss: 0.4699 - accuracy: 0.9424 - val_loss: 0.4600 - val_accuracy: 0.9428

This is the output:

Digit 0: IOU: 0.0 Dice Score: 0.0
Digit 1: IOU: 0.0 Dice Score: 0.0
Digit 2: IOU: 0.0 Dice Score: 0.0
Digit 3: IOU: 0.0 Dice Score: 0.0
Digit 4: IOU: 0.0 Dice Score: 0.0
Digit 5: IOU: 0.0 Dice Score: 0.0
Digit 6: IOU: 0.0001399971998600067 Dice Score: 0.00027995520677497873
Digit 7: IOU: 0.0 Dice Score: 0.0
Digit 8: IOU: 0.0 Dice Score: 0.0
Digit 9: IOU: 0.000322164947934657 Dice Score: 0.000644122382215584

Grade is 0.0046216214779466365
You failed. Please check your model and re-train

That accuracy cant be right or you have changed something in the code by mistake which you are not supposed to have done.

That accuracy means that you should have in most of the cases a dice score and iou because those depend on the predictions.

I suggest you to go back again at the notebook from the beginning or even better off to reset the notebook if in case you cant pick up where its going wrong. You can save your work with another name for sure.

Did you ever resolve this issue? I’m having the same problem except my IOU and Dice are identically 0 the entire time. I suspect a typo, but still searching. I’ve attached some screenshots below:



Hi Reginald, I don’t remember but I don’t think I did. Can you please post it here if you get it resolved?

I’m having a number of different issues. When I went back and trained for 70 epochs instead of 10 (where the accuracy seemed to plateau) my code worked and I passed the IOU tests…but when I submitted I got 0/100 because the grader said there were issues with my submission (no specific failure given). Reading other threads, I tried using TF and Keras 2.8…but then with the exact same code, my IOU test failed! The other issue is you quickly run out of GPU time on Colab so I’ve had to dip into my GPU time on Kaggle, which is frustrating.

Assuming all the models are built properly and put together correctly, I would think about the hyperparameters chosen, the grader might also expect certain hyperparameters as well.

Hello Lakshmi,

can you share your model training code in private message inbox or your notebook (click my name and click on message to send the code). It clearly mentions there is a problem in your model, so some parameters must be misinterpreted or missed.

Regards
DP

Hello Reginald

Even your grading tells the same result about model. Try going back to previous cell and check where you can make a change to train your model with desired result. if not able to find, inbox your notebook to private message.

Regards
DP

Reginald I noticed one more thing in your screenshot, you have used epoch of 10, but if you notice in the expected output, epoch used is 70. try re-train model with 70 epochs, if it works.

Thank you for the prompt feedback. Ultimately, I have resolved the issue thankfully. I do think one issue was the number of epochs. Since I was running out of Colab GPU time, I trained for 10 since the accuracy was still around 95% and we only needed IOU of 60 to pass. I needed the full 70 which without a GPU takes hours.

Also, the strides seem to play an important role. I played around a bit the stride value from what was used in the videos that are passed to the conv_block (trying 2 and 3). I think these two factors helped resolve the issue. Thank you to everyone!

Happy to help, actually just before this cell it is mentioned to use any number of epoch, so even I did the same at beginning but even I got the same result. But then I noticed expected output epoch and made the change which worked for me.

Keep learning!!!
Regards
DP

1 Like

I have exactly the same problem, I have not been able to solve it, I would appreciate if you could share more information on how you solved the problem

For me, the issue ended up being the hyperparameter values. I needed 70 epochs as was used in the example even though with just 10 epochs the accuracy was 95+%. I hadn’t been using 70 because I ran out of GPU time on Colab free version. I ran it on Kaggle’s GPUs, just have to be careful to match the versions of Tensorflow/Keras, etc. So make sure to print(tf.__version__) in the colab and use that on Kaggle if you need.