Writing custom data generator

Hi all

I am trying to work on the whole dataset available for brain tumour, available at the website mentioned in the assignment (Task01_BrainTumour)

I would have do the step of creating subvolumes within the data generator, right (because I want to create new subvolumes every epoch, I presume)? How can I create subvolumes within the custom data generator, and under which function of the override will it go into?

Just to be clear, every epoch, for each original scan image, I will create 1 subvolume and pass that instead, for each training example. So eventually for a larger number of epochs, I will potentially cover every scan, right?

@nakamura Could you please take a look at it?

Hi @Jairaj_Mathur,

I suppose, to create subvolumes in the custom data generator, it would be one of the choices to modify the “VolumeDataGenerator” class in util.py.
The file “util.py” exists in the Lab Files of Coursera workspace.

Regarding another question, when creating a subvolume from a single original image, I suppose there is arbitrariness in the area to be chosen.
From that point of view, it might be unclear whether the method you suggest covers all scans. Of course, it does NOT consider the huge epoch-number case.
Besides, another concern is that it may affect learning If there are many images that do NOT include the target object.

@nakamura Thank you for you reply, what would be a better strategy than randomly sampling subvolumes? I thought this is what is being done in the programming exercise as well?

Hi @Jairaj_Mathur

I am sorry for the late reply.
I suppose the section on “Data Preprocessing using Patches” in the assignment may help you.
The background data is much in the dataset.
Therefore, it may be a good point to consider the rate of the background data when generating a subvolume of data.

Best regard,

1 Like