I was wondering if it makes any sense to modify the threshold probability used for obtaining the predictions (cat vs non-cat), ie., instead of using 0.5 as threshold, to use larger than 0.9 for positives and smaller for negatives or maybe, using >0.9 por positives and <0.1 por negatives. I did a couple of tests, but the results were unclear, although the training was successful, the accuracy was lower, for training as well as test data sets, I don’t know if it affects overfitting or affects in any other way the classification process. Thank you.
Hi @djpassadore,
You usually do the thresholding in the testing phase when you want to get your accuracy. It is true that the thresholding at 0.5 might not be the best. That is very task dependent. In some cases like Medical domain, we want the number of false negatives to be very low.
You can shift the threshold and see the changes in behavior of the metrics. The resulting graph is called the ROC curve.
However, you seem to have two thresholds. Now in that case, what classification would you give to an image which falls in between those two thresholds? Thus we should have only one threshold but the value that threshold might have is upto the user and the task he wants to perform.
Hope this helps!