Hi, I am trying to play around with lovasz loss function during my model’s training. I got confused with the results. They show a certain value of loss but no value for the accuracy. Even the visualization of the predicted image didn’t show anything.
If anyone could help me what could be the problem.
Thank you
1 Like
Can you provide more information?
I have never heard of the Lovasz loss function until you mentioned it here. So I checked both TF/Keras and PyTorch and it is not one of the provided functions in either of those platforms. I did the obvious google and found this “papers with code” link. That link includes a github site with code for both TF and PyTorch apparently. Or at least that’s what one sees at the top level of the repo.
So is that the code you are using or did you write your own?
1 Like
@TMosh and @paulinpaloalto, Yes, I tried the code from the link. I changed the activation function from ReLU to softmax to suit my needs, as I understood from the link. However, the model result is not stable, when trying to visualize it, the prediction result does not appear at all.
In addition, I also tried Tloss. Unfortunately, Tloss did not work as expected. I tried adapting the code from the segmentation remedy on GitHub, but the result was a Train loss of -19564.1875 and 0% accuracy. This model doesn’t seem to work properly, as the loss should be close to 0. My goal is to see the effect of loss functions for medical image segmentation, particularly with imbalanced dataset ( foreground vs background)
1 Like
According tou your answer when you change the activation relu to sigmoid that maen if you have such value in your layer like 1-10 it always change into 1 or 0 because that how the activation work after that calculation that all the values will bring to your optimizer maybe that why your accuracy closer to 0%
1 Like
I think we need to have a discussion here at a higher level to start with. We can’t see your data, we can’t see your code and you’re talking about using code that is not part of any of the classes here from DeepLearning.AI at least that I’ve ever run across. It’s also not part of any standard platform that we might be familiar with. But it sounds like you’re expecting us to be able to debug your code for you. The best we can hope for with what we know at this point is to give general advice that you can apply to solving whatever the issues are that you have.
How much experience do you have with deep learning in general? Is this the first time you are trying to implement semantic segmentation? That topic is covered in DLS Course 4 Week 3 here, but they use the U-Net architecture for semantic segmentation. Are you familiar with that assignment or with U-Net in general?
What is your base model here, before you started experimenting with Tloss and Lovasz? Did it work before you started the experiments? What kind of accuracy were you getting before?
My apologies,
I am new to Deep Learning and have little experience with U-net using the ACDC 2017 dataset. A month ago I just took an AI for Medical Diagnosis class, and I was curious about the loss function. https://github.com/JunMa11/SegLossOdyssey. That’s why I tried to do trial and error (plug and run) with it.
As for T-loss, I used the code from here: https://github.com/Digital-Dermatology/t-loss
Translated with DeepL.com (free version)
Thanks for the additional background on where you are in your learning process. It is great that you are exploring these topics, but it’s going to take more work to get meaningful results. If you want to experiment with any of the different loss functions that you have found, the suggestion is to start with a model that you have already defined and trained with the standard loss function for semantic segmentation, which they would presumably have showed you in your AI for Medical Diagnosis class. Then use that starting point as the basis for the experiments, so that you understand both the dataset and what your initial results are. Then the question is to try other loss functions and see what differences in the behavior of training and the performance of the resulting model you encounter.
Which learning platform are you using: TensorFlow or PyTorch or some other? Note that you are using loss functions that are not one of the standard ones provided by the platform, so the question is whether the developers who created those additional functions understand the requirements for loss functions in TF or Torch. E.g. in the TF case, they must provide various methods that are used during the “fit()” operations to do things like compute gradients as well as accuracy in addition to the loss values. I think TF can handle it using automatic differentiation even if the new function does not provide the relevant gradient APIs. I’ve never tried to build my own loss function in TF, so I don’t claim to understand everything that is required. But if you want to play this game, then you’ll need to dig into things a bit deeper to understand how to use the new APIs that you are accessing.
Dear Paul,
Indeed I still have a lot to learn about this.
Thank you very much for your insight and feedback.
Later on, I will come back with other questions.
1 Like
That sounds good. The references you provided above are really interesting and completely new to me. They go much deeper on the question of getting good performance from image segmentation than is covered here in DLS Course 4. The U-Net implementation that Prof Ng shows us just uses the standard softmax per pixel loss function. DLS C4 was published in September of 2017 and it looks from the dates on a lot of those papers that the state of the art was evolving rapidly at that point and beyond. The point about the natural imbalance between foreground objects and background in image segmentation sounds pretty important and it looks like the researchers have found better solutions to deal with that and get better results. It’s a good point in general that things do not “hold still” for very long in this space: there’s always new research and discoveries being made.
It would be interesting to hear about what you learn as you explore this space!