Pneumonia detection using deeplearning

Course Q&A AI For Medicine AI for Medical Diagnosis
1

After completing the first assignment. I have tried to implement pneumonia detection using transfer learning with Keras and TensorFlow. And Im getting the following results :

                                         Training Accuracy:             98.90%
                                          Validation Accuracy:        97.69%
                                          Test Accuracy:                   90.38%

I can notice that there is approximately 8% difference between the Training and test set accuracy, Which means there is a high variance if I’m not wrong.

Is my current model performance good or there is a way I can improve the performance on the test set? since the end goal is to deploy the model.

For more details on the model building process:

                     Usage of pretained Model:   ResnNet50 

                     Method1:            Feature Extraction with data augmentation  

                     Method2:           Fine-tuning the pre-trained model ( Best performance)
2

That is correct. Such differences indicate a high variance, meaning you are doing a poor job making the model interpretable.

I suggest you use an ensemble of some sort, increase training size or use augmentation.
Let me know

1 Like
3

Regardless of what exact percentages you produce, accuracy by itself is not a sufficient quality metric, especially in healthcare. You should think about a more robust set of measures. This article might provoke fruitful thought: On evaluation metrics for medical applications of artificial intelligence

You might also reconsider the statement

Your end goal should be more along the lines of improving the health and quality of life of your patient population. Deploying a ML model might be one way to affect that, but deploying a model shouldn’t be the end goal.

2 Likes