Anomaly Detection: How to improve?

[Christian_Simonis]

Hi there

thanks for your question!

@Carina_Kothe

• Is the cause to the large amount of false negatives the big variety in defect sizes? Or is the problem that even in the ‘good’ pictures, there is still a lot of variety?

Without taking a look at the actual pictures, it’s hard to judge remotely. But if you have many false negatives, you rather want your model to learn „more“, how an anomaly looks like, meaning:

• a more complex model could potentially make sense to learn „anomalies“. You might wanna try: less drop out or less regularization. See also: Training set error?
• the threshold can be tuned to improve false negatives. In this case this would represent a trade-off w/ the false positives in general which would worsen in return…

@Carina_Kothe

• Anomaly detection is only trained with ‘good’ images. Intuitively I’d say training only with ‘very good’ images will not help either, because then many of the ‘good’ images will be labeled falsely as anomalous. => Do I have a chance in improving the algorithm or is a data set with too much variety in even ‘good’ images not suitable in anomaly detection?

there are several strategies, e.g. you try to get better, more and „more realistic“ normal labels. You could also think of applying some artificial noise (e.g white noise) to increase the robustness in total.

@Carina_Kothe

• Can data augmentation help with anomaly detection? After all, for the training only the good images are being used.

There are cases where tilting collides with your business problems (e.g. think of tilting a 9 or 6 too much if you to number classification). But if tilting or flipping pictures is OK from your domain point of view, it might be worth a try to help the model to generalise and learn what the defects are actually about. See also here some other techniques for augmentation:

• https://neptune.ai/blog/data-augmentation-in-python/amp
• Data Augmentation Techniques in CNN using Tensorflow | by Prasad Pai | YML Innovation Lab | Medium

This part of your question was not clear for me:

@Carina_Kothe

When looking at the anomaly image of the inference result, I can see that even the small defects are being detected as an anomaly. However, the whole image has usually about the same anomaly score as the threshold or is slightly below it, thus resulting in many false negatives.

What does „being detected as anomaly“ actually mean here? After all, I understood that your problem is rather too many false negatives (FNs) and the algorithm did not identify the anomalies correctly as such… how about the false positives (FPs)?
I am asking because I would be interested in how you chose your threshold.

Also: Did you check an ROC analysis w/ AUC?

File:ROC curves.svg

I would highly recommend to take a look at it. It might also help you in dimensioning your threshold well, considering FPs and FNs.

Note: a good friend of mine did his PhD in defect detection and reconstruction. Feel free to take a look at these repos:

• GitHub - AbdullahKaratas/Reconstruction-with-PCA: Reconstruction using PCA
• GitHub - AbdullahKaratas/Generate-Cutting-Edge-and-CNN-Reconstruction: Generate Cutting Edges And CNN Reconstruction

Best
Christian