Dear Mentor,
Could you please guide me how to inspect the element in this situation?
In Section 2.2 Preprocess Your Data,
“dataset” passes 1060 pairs of (image_filenames, masks_filenames) to process_path function
Thank you.
1 Like
Have a look on this page:
Hello @JJaassoonn,
That code you shared does not yet go through the dataset, instead, it only defined the mappings. Modify the following (especially to change dataset to processed_image_ds) and run it to see the effect of tf.print
Cheers,
Raymond
Dear Mr Raymond,
I found that there are some weird behaviours when i applied the following codes.
import tensorflow as tf
image_list = ['./data/CameraRGB/002128.png', './data/CameraRGB/008579.png']
mask_list = ['./data/CameraMask/002128.png', './data/CameraMask/008579.png']
image_filenames = tf.constant(image_list)
masks_filenames = tf.constant(mask_list)
dataset = tf.data.Dataset.from_tensor_slices((image_filenames, masks_filenames))
def process_path(image_path, mask_path):
tf.print("here")
tf.print(image_path)
tf.print(mask_path)
tf.print("here")
return image_path,mask_path
case = 1
#case = 2
if(case == 1):
dataset = dataset.take(2)
print("dataset length: "+str(len(dataset)))
output=dataset.map(process_path)
print("output length: "+str(len(output)))
elif(case == 2):
dataset = dataset.take(2)
print("dataset length: "+str(len(dataset)))
output=dataset.map(process_path)
print("output length: "+str(len(output)))
for i,j in output:
tf.print("after")
tf.print(i)
tf.print(j)
tf.print("after")
Case 1 Output:
dataset length: 2
output length: 2
Case 2 Output:
dataset length: 2
output length: 2
here
./data/CameraRGB/002128.png
./data/CameraMask/002128.png
here
after
./data/CameraRGB/002128.png
./data/CameraMask/002128.png
after
here
./data/CameraRGB/008579.png
./data/CameraMask/008579.png
here
after
./data/CameraRGB/008579.png
./data/CameraMask/008579.png
after
These 2 codes provide 2 different results. The Case 2 is even strange, “output length: 2” skipped over the “dataset.map() function” and produces an output before it.
Why an additional for loop can make so much differences?
I expected to get this sequence of result, but it was totally different from the result above.
dataset length: 2
here
./data/CameraRGB/002128.png
./data/CameraRGB/008579.png
./data/CameraMask/002128.png
./data/CameraMask/008579.png
here
output length: 2
after
./data/CameraRGB/002128.png
./data/CameraMask/002128.png
after
after
./data/CameraRGB/008579.png
./data/CameraMask/008579.png
after
Could you please guide me on this issue?
Thank you.
Hello @JJaassoonn
How did you define that dataset?
And how did you define this dataset?
The assignment defines the first dataset like this:
To make sure we are on the same page, please update your first post by including the above definition line plus any other relevant modification to your Case 1 & Case 2 respectively (consider case 1 & 2 as two distinct cases independent of each other please.).
Thanks,
Raymond
Dear Mr Raymond,
The dataset is defined in
- Week 3 Assignment : Image_segmentation_Unet_v2
- Section 2.1 : Split Your Dataset into Unmasked and Masked Images
image_filenames = tf.constant(image_list)
masks_filenames = tf.constant(mask_list)
dataset = tf.data.Dataset.from_tensor_slices((image_filenames, masks_filenames))
This dataset was used on Case 1 and Case 2 respectively (consider Case 1 & 2 as two distinct cases independent of each other but both using the same dataset)
Thank you.
Would you mind modify your first post to include everything? It’s a good practice to provide full code for others to just run it and reproduce your result.
Dear Mr Raymond,
Sorry for the inconvenience caused. I have updated my first post above to include both cases.
Thank you.
Thank you @JJaassoonn.
To begin with, one of my case 2 results is this:
dataset length: 2
output length: 2
here
./data/CameraRGB/002128.png
./data/CameraMask/002128.png
here
here
after
./data/CameraRGB/008579.png
./data/CameraMask/008579.png
./data/CameraRGB/002128.png
here
./data/CameraMask/002128.png
after
after
./data/CameraRGB/008579.png
./data/CameraMask/008579.png
after
There is an important observations: I said “one of” because it changes everytime I run it. Try to run it for multiple times. What would you hypothsize given such observation that may explain your observation about the changing printing sequences?
Cheers,
Raymond
PS: The first four and the last four lines seem to be always unchanged. Right?
Dear Mr Raymond,
One thing is very weird is that if i execute the code straight away on the Week 3 Assignment workbook (on the coursera portal). The result is very stable (as shown on my first post above).
But then when i execute it on my local Jupyter Notebook workbook, i have gotten the result as same as yours, i.e. keep changing.
Oh, you can run it on Coursera? I cannot because it complains when running len(output). Did you make any changes to the Cousera environment?
No, i didn’t. Please ignore the "print("dataset length: “+str(len(dataset)))”.
I see.
Btw, for your result, seems two “here” are missing. Did you miss out anything when copying?
Check this:
empty
That is my intuition on the expected result, not the actual result.
I expected to get the result based on my understanding on coding.
Oh, my bad. I take that question back.
OK. I verify that I get your result running on Cousera and is unchanged.
So, @JJaassoonn, any hypothesis before I share mine?
I suspect Tensorflow is doing something on the background to make sure only one pair of example is being passed to the function at a time
Or not doing anything in parallel so that one comes after another.
Seems that on Cousera, it processes the first sample so it prints two “here”, and then prints the two “after” in the loop, and then the first loop completes, and then begin the second loop with processing the second sample which gives us another two “here”, and finally two “after”.
Then on our local environments, something seems to be parallelized .
My env uses TF2.13.0, while coursera uses TF2.9.1. Hmmm… would that be a behavior change due to versions?