On week 4, we saw how using a saliency map, we can visualize where our model is “looking at”. But we didn’t see how to use that information after we get it. In other words, once we see the most important areas of our input images, what should we do to improve the performance of our model? I have googled the subject, but I mostly get papers on researchers using the saliency maps in addition (mixing them) or before a model. Any ideas? Thanks in advance.
Let’s say your model is picking up a random feature (not robust) along with some other features to classify images of a particular class in your dataset. This may result in lower accuracy on the validation and test sets as this random feature may not be present in these sets.
You can use the results of saliency maps to suppress this random feature through custom data augmentation process. You can also modify the network architecture to improve the results as done by the ZFNet authors.
3 Likes
Thank you SainiAnkit!