Hi, I just finished 3rd week of this course and as a trial to practice what I learned, I am working on a project to detect lesions in dental X-ray images.
but I am facing a problem now which is classes distribution imbalance, I am working on five classes that are distributed due to the nature of the problem as follows:
Abscess → 3635 boxes
Caries → 12764 boxes
Impaction → 1936 boxes
Pockets → 10046boxes
Remaining Roots → 1681 boxes
these all classes are found in a dataset of 2593 dental X-ray images.
So I am now wondering if this imbalance will affect my model’s performance in detecting different classes or not, and if the answer is yes, what techniques are recommended to overcome this problem.
Actually, I thought about removing some of the major classes annotations (let’s take caries as an example) as a trial to balance the distribution, but this will get us to another scenario, now the dataset will have images with “caries” annotated and some other images with “caires” not annotated, am I confusing the model now? as I am telling it now to deal with “caries” as a problem in some image and as a background in some other images.
Short answers: Yes. Augmentation.
You probably want to think about incorporating augmentation anyway, because you want as many detector locations as possible exposed to a diversity of types. That is, if you always train on images with a cavity only right in the center of the image, the other grid cell locations will never be trained to predict that class. Also, to generalize well, you are likely to need to account for somewhat different views. Augmenting with spatial repositioning is likely to help with generalization, and while you’re doing it, you can plus up the underrepresented classes.
Be forewarned that augmentation with algorithms that use bounding boxes introduces additional complexity since you have to keep the bounding box coordinates in synch with changes to the image.
It would be interesting to compare accuracy with and without balancing. Let us know how it goes.
I think @ai_curious is replying to you, i was going to refer it to him. He is an expert in YoLo 
Xie xie xie xie @gent.spah
Perfect, but won’t augmentation also increase the number of overrepresented classes too??
This will take me to another question, Is the goal is to balance the proportion of each class in the whole dataset? or do we only care about having a sufficient number of the most underrepresented class and not paying any attention to the imbalance problem?
Please note that one dental X-ray image could have up to 5-10 “caries” but only 1 “impaction”.
Another point that I want to refer to; does removing some of the “caries” annotations help the model or it will confuse it more as I am exposing it to some images with “caries” defines as a problem to be detected and some other images with “caries” defined as a background.
Thank you so much
I wondered about that. Luckily my personal dental xrays have always had minimal interesting features so I’m not an expert on this data 
I definitely recommend against removing some labels from images that have them. Doing so tells the feature extractor ‘sometimes I care about this, sometimes I don’t’ but not providing a way for it to discern which to apply at a given time.
If all of your images with underrepresented classes also contain over represented classes, clearly you don’t solve the problem by applying the same augmentation to the entire image. Sometimes you can augment not at the entire image level, but only at the bounding box level. But again, since I am not an expert on dental images, I don’t know how feasible or helpful that would be.
That might suggest a different approach, which involves weighting. Not my particular area of experience or expertise, but I believe there are two approaches. One involves weighting in the loss function, and one involves weighting the initial bias. Here is a link to some official TensorFlow doc that discusses the latter in a structured data situation:
Is that Hebrew script it’s showing? No idea why 
I’ll look for a reference for the weighted loss function use case.
thanks a lot for your support
I think this is not so much a YOLO-specific question, but more of a general class imbalance question. The images have multiple classes present, some each of the over represented and under represented, so the vanilla augmentation approaches don’t help. Thought and suggestions?
Ideas here seem mainstream and worth a look:
I will keep doing my research and let you know if I reach to anything
thank you
This paper is from 2017 and doesn’t use YOLO (though one of the authors was also a coauthor of YOLO v1). It introduces the approach of weighting imbalanced classes within the loss function and might have some ideas you can steal. HTH
I am really grateful for your support.
thank you so much
this is how ChatGPT answered 
Addressing class imbalance in YOLO object detection when dealing with varying numbers of boxes per class in an image requires careful consideration. Here are some techniques you can try:
-
Weighted Loss: Assign different weights to the loss function based on the number of boxes per class in each image. For example, you can weight the loss contribution of each box by the inverse of the number of boxes for its corresponding class in the image. This gives more importance to the boxes of the minority classes and helps balance the impact of different class instances during training.
-
IoU Thresholding: Set a higher Intersection over Union (IoU) threshold for positive detections of the majority class. This means that even if an image contains multiple boxes for the majority class, only the box with the highest IoU with the ground truth will be considered as a positive detection. This way, the model focuses on the most accurate and representative detection for that class, reducing the dominance of multiple detections in the training process.
-
Class-Specific Sampling: During mini-batch creation, ensure that each mini-batch contains a balanced representation of different classes. You can achieve this by dynamically adjusting the sampling probability of each class based on its number of boxes in the image. This ensures that each class has a fair chance of being sampled in the mini-batch, regardless of the number of boxes present.
-
Data Augmentation: Apply data augmentation techniques specifically targeted at balancing the number of boxes per class. For example, you can introduce augmentation strategies that duplicate or replicate boxes of minority classes to increase their representation. This helps create a more balanced dataset and allows the model to learn from a variety of examples.
-
Progressive Sampling: Implement a progressive sampling strategy where you start training with a subset of the data that balances the class distribution. As the training progresses, gradually introduce more samples with imbalanced class distributions to further fine-tune the model’s ability to handle such cases.
It’s important to experiment with these techniques, either individually or in combination, and monitor the model’s performance on the minority and majority classes. Evaluating the model’s accuracy, precision, recall, and F1 scores for each class can help determine the effectiveness of the chosen approach and guide further adjustments if necessary.
+1 what they/them said 
Hello, can I know how did you do caries annotation, because you mention at one place you mentioned it as a background and another places as objective part of your image which is why the model couldn’t classify it in separate classes. actually as caries only as one class again requires numerous notation of x-ray imaging.
Hi, actually I have a friend who is a dental radiology specialist. he helped me with the data annotations.
if you feel trusting, you can share with me the annotation, I am a dentist. the reason why I felt your model is confusing might be because even if take considering only x-ray features, any diagnosis say it might be caries, abscess or any other lesions, will always have a combination of features to give a provisional diagnosis.
Also I feel training a dental radiographic imaging to come to a diagnosis would still be not accurate as in dentistry even caries has many classifies and then sub-classifier or combination of two. So to make it a binary classifier will always lead you to run in wide jungle searching for the fruit under the pile of muddled lake.
first the input for these diagnosis need to be appropriate for the model to be able to understand, so it can read or understand any variation, that is where @ai_curious mentioned about data augmentation. Data augmentation is actually explained in tensor flow course, which I would highly suggest anyone to do as it clear more and more about model training, data training.
Hope it helps.
Regards
DP
Okay, if you are interested in this project please inbox me. I think we can work on it together.