How can there be all classes in 1 MRI Scan?

Hi. I am sorry if the question is naive.

In the Explore MRI Data & Labels Lab, we are looking at 1 MRI scan. How come that scan has all 4 classes present? How is it possible that edema, non-enhancing tumor and enhancing tumor would be close together in the same layer of a scan?

E.g. If we run

from collections import Counter
Counter(label_array[:,:,75].reshape(1,-1)[0])

we get all 4 classes.

Similary, if we inspect

label_array[160,:,75], we see that classes 1,2,3 are clustered together.

I am probably missing something in my understanding. I would appreciate some help.

Hi @Shahbaz_Masood!

I think your question is a very interesting one :slight_smile: . Recall that we are interested in studying image segmentation for this week (in contrast, for example, to week 1’s content). You can think of image segmentation problems as pixel-wise (or voxel-wise if you use 3D images) classification. So for your labels, you have an array of the same size as the image having the corresponding class label for each component. What we want in a medical segmentation algorithm to do, is to be able to extract features from the image to be able to predict the class of each part of the image.

Look at section 2.2 Visualize the Labels for a Specific Layer. You will see that here you’ll have a “mask” per label that lets you identify which parts of layer 50 of the MRI correspond to each of the classes studied (where the presence of each class is shown in white in the corresponding panel).

On the other hand, when you run this code:

from collections import Counter
Counter(label_array[:,:,75].reshape(1,-1)[0])

What you are getting is the number of pixels pertaining to each of the classes in that specific layer of the MRI. So when you ran it for layer 75 what you get is:

Counter({0.0: 55286, 1.0: 1248, 3.0: 531, 2.0: 535})

This means that for layer 75 of that MRI, 55286 are of class Normal , 1248 of class Edema, 531 of class Non-enhancing tumor, and 535 of class Enhancing tumor.

It is very important to mention that labeling medical images for segmentation problems should be done by specialists and the same image may even be annotated by more than one specialist.

I hope it is clearer for you now :smiley: .

Thank you @andres920310 for responding in detail.

A follow-up question would be: is it common for MRI scans to actually have multiple diagnoses (i.e. edema, non-enhancing tumor, and enhancing tumor) in the same MRI scan? Or it is an artifact of the majority voting of the specialists?

Hi @Shahbaz_Masood! Nice question too. It might depend on the task we are interested in. Both in classification and semantic segmentation, you can have multiple labels per observation. The presence of multiple labels depends directly on the task being studied, and it will apply to different medical images. For example, in lung nodule segmentation (for which CT scans are used over MRIs), you will usually find two classes that are typically coded as 1: Nodule and 0: Non-nodule. However, these annotations can be further refined, for example, having something like 0: Normal (Background and lung), 1: Nodule, 2: Non-nodule lesions. Or even you can have tasks in which you are also interested to segment organs for which you will have annotations that differentiate lungs from the background.

For instance, look at the LNDb database on lung cancer diagnosis. You can read the details of this database here. Similarly, every dataset would have a document describing the way the annotations are built and coded.

I hope this helps :smiley:

Thank you @andres920310. This makes sense.