I’m currently trying to build a classifier to detect diseases from chest x-rays using the chexpert dataset. I’m in the data pre-processing stage and taking a first look at my labels, I noticed some classes have negative values.
According to the dataset documentation, this negative value means a condition was reported with uncertainty or ambiguity (a condition may or may not be present in the image) . For example (as gotten from the docs), here’s are two scenarios where the data was labelled as negative:
“The cardiac size cannot be evaluated.”
“The cardiac contours are stable.”
In both cases, one or more columns would have -1 as the label. I’m wondering what to do for images with such labels. I think it might be drop such rows but my consideration is when they contribute significantly to the size of the dataset.
I’d like to know the opinions of the community on how best to solve this negative labels problem.
Thank you.
If these labels are uncertain its better not to use them, instead with the few labels you have dont use straight accuracy but precision, recall and F1 score or even other metrics for imbalanced datasets. (my opinion).
1 Like
As said by @gent.spah such label should not be included. I would say you could create noise with these images for a robust model, it just an idea 
The cardiac contours are stable would come under normal or no disease so you can include that.
The cardiac size cannot be evaluated could be reason again due to other co-factors like chest injury, improper image, or chest conditions or cardiac conditions like cardiomegaly which can again be due to symptoms of multiple issue in a patient.
If the cardiac size cannot be evaluated is mentioned, look for other factors on why it was unable to evaluate and then you can label it as per your classification
Cardiac size is nothing but cardiothoracic ratio (CTR) which is the relationship between the size of the heart and the transverse dimension of the chest measured on a chest PA radiograph , is a commonly used parameter in the assessment of cardiomegaly.
Hope it helps!!!
Regards
DP
Thanks @gent.spah. I had the plan to drop them altogether as I could not justify converting the labels to a positive or negative class.
Thanks @Deepti_Prasad.
I would have to ignore all images altogether. This is because I don’t have access to the raw text labels (the examples I stated are just example labels provided by the labellers) but moving forward, I plan to actually collect real-world radiography data and the tips you’ve given will definitely help me label this data more effectively.
you didn’t have the raw data? then how were you classifying them? just with the labels provided by a third-party? good decision Timothy to start from scratch.
happy to help!!!
keep learning!!!
Regards
DP
Thanks @Deepti_Prasad
I’m currently working on a proof-of-concept using the chexpert dataset. The authors of this dataset did not provide the raw radiology reports AFAIK so I have to use the labels provided to build a model.
My end goal is to use this proof-of-concept to gather more data from clinics and hospitals in my locality and then work on generating the labels myself.
From the explanation, it seems like an exploratory analysis, so I hope you are going to use stratifiedKfold for your dataset based on the your explanation.
Also what I can see you are doing a more of a binary classification of labelling your data and then testing, which according to me in chest x-ray dataset would not fit in, if you want to explore the diagnosis part of modelling, but binary classification would surely work if your labelling negative for no disease or normalcy, positive for disease related any of anatomical landmark of interest(here as in chest region).
Most of the exploratory analysis where in you have predictive variable such as presence of fluid which can again be either categorical (presence or absence of fluid) or ordinal (based on stages of different type of disease, eg stages of pneumonia) making you detect a categorical response variable like here would either pleural effusion or pneumonia, so one needs to do ANOVA analysis.
I hope I didn’t confuse you much 
Wish you all the best of efforts and outcome.
Regards
DP