Test Accuracy higher than training for brain images

Is anyone face the issue of test accuracy higher than training for brain images? I have used the spm12 tool for the alignment and normalization of neuroimages. if someone has faced this issue please share his/her opinion and suggestions. It would be really appreciated.

Thank you.

Regards,

Abdul Rehman

Hi @Abdul_Rehman3

Experiencing test accuracy higher than training accuracy when working with brain images using tools like SPM12 may indicate potential overfitting, where the model learns to fit the training data too closely, impairing its ability to generalize to new, unseen data. To address this, ensure accurate and consistent data preprocessing, consider data augmentation to increase training data diversity, use regularization techniques, balance model complexity, perform cross-validation, explore different model architectures, and be mindful of feature selection. Experimentation and iterative refinement based on observed results are key to improving model performance and avoiding overfitting.

Regards:
Muhammad John Abbas

How the spm12 tool can cause the potential of overfitting. It is used to standardize the images and align them.

I am doing the following augmentation during training of model:
train_transform = T.Compose(
[
T.RandomResizedCrop(size=img_size),
T.RandomHorizontalFlip(p=0.5), # with 0.5 probability
T.RandomApply([color_jitter], p=0.8),
T.RandomApply([blur], p=0.5),
T.RandomGrayscale(p=0.2),
# imagenet stats
T.ToTensor(),
T.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])
]
)
what is your opinion on this?
And can you kindly elaborate more on what is meant by cross-validation and feature selection in DNN? It would give me a good insight of your opinion and then I can make my comment on this.

thank you.

Hi @Abdul_Rehman3 ,

You have only applied the augmentations to the training images. The images in the training would be separated from the test ones and a mismatch is created in between your training and test data. That might be the reason of accuracy mismatch.
You cannot horizontally flip the brain images the tumor location is very important. Its better to work without augmentation when you are working with brain images if you want large dataset, go to kaggle there is datasets of BraTS. This dataset is huge of size around a total of 13 GB. You don’t need to apply augmentation everywhere. It is used where the dataset is small.

Thank you. For any further assistance please ask. Appreciating your learning skills.

Hi @harshder03 Thank you for your suggestion. Actually, I am not working on tumors. My work is related to AD where neurodegeneration information matters so that’s why random flipping is applied.

Hi @Abdul_Rehman3 , Sorry for the misunderstanding. Actually, I got confused because you specifically tagged Week1 AI for Medical Diagnosis. you are working with Alzheimer’s detection in Neurodegeneration. What are the train and test accuracies you were getting. I guess it won’t be above 95. As there might be some mismatch between train and test. Can you describe in detail which data you are using?