Hey guys. So lately, i get bunch of errors, not because of training, i did the training part, but now i have to do the webcam test. It threw me some error, so i’m going to post my code below,
You may be better to read notebook very carefully.
Your unet output has 23 classes, which is a default setting in our exercise. In our assignment, there is a function, create_mask() to create a mask from this output. This will be sure to convert your 4D tensor into desirable format, most likely 480x680x1.
matplotlib supports your output as a color map directly, but, I suppose, in the case of cv2, you may need to convert by cv2.applyColorMap(). And, this takes numpy array as source, and data type needs to be one of cv2 mat type. So, you need more conversions, I think. (like tensor->numpy, int64->uint8)
By the way, it looks like you are using Jupyter notebook. If I remember correctly, cv2.imshow may not work correctly on Jupyter notebook.
Hey, thanks for your help! I can finally get it to work but though one thing. The camera finally pop up to show my webcam, but it shows this orange color. I think this is because of the colormap but i don’t know what colormap is for Semantic Segmentation. Do you have any idea what colormap to use for Semantic Segmentation?
Oh and also did you know where is the class? I meant like what color represent what object. Other than that thanks for your help again!
Picture:
I think this is because of the colormap but i don’t know what colormap is for Semantic Segmentation. Do you have any idea what colormap to use for Semantic Segmentation?
All depend on your design.
As you know, we set 23 classes in our assignment. So, output mask should have the value in the range of (0-22). You need to set an appropriate color map or palette. In the case of Matplotlib that we used for our assignment, default cmap is “viridis”. We can get it by
plt.rcParams[‘image.cmap’]
Of course, it can be changed easily.
There is no default color map for a semantic segmentation. (Our assignment is just borrowing a default setting of cmap in Matplotlib.) Depending to the number of classes, you need to choose the bast one to describe your targets.
How about for cv2 colormap? Because i think i can’t do for matplotlib.
I already provided all answers to you. It’s up to you.
If you want to see similar output as what we got in Matplotlib, then, it would be,
cv2.applyColorMap(mask[:,:,0]*10, cv2.COLORMAP_VIRIDIS)
Due to the differences in color distributions, I simply multiplies 10 to get a similar result. But, again, this is also up to you.
Hey, thanks for the response. I tried it, however, it resulted me in the picture below,
From a color distribution, your color indicates that your color index = 0. (If a color index is 255, then, it can be seen as “yellow”. Have you checked the output ?
I’m sorry, but what output? The output was just it predict how much ms per step. It also outputs a new window to track my webcam.
From an image that you posted, an observation is quite simple. You passed most likely all zero data (frame in your code) to cv2.
So, you need to ensure that you get the right output from u_net.predict, and also need to ensure that the data is processed correctly. That’s typical programming/debugging steps.
And, the right output depends on your network definitions, i.e, number of classes. Have you checked that “frame” has appropriate values in the range of 0 to “number of classes” ?
Based on the output, Most value i see are 13 and 9. I think that’s the problem.
I do not think it’s a problem since some classifications are done.
If you use a cv2 color map, the color balance (distributions) is different from Matplotlib.
See my input to you. I slightly scale up the color indexes for cv2 color map.
I believe you can see something if you do the same thing. 
I see. But is there any way i could look at the class? Like color blue represent car for example.
Color mapping is your choice. No one can say what your blue means.
For this particular exercise, I do not think there are text labels for output classes. As a ground truth, masks are provided. And, as you know, a mask include values from 0 to 22 corresponding to object types. But, again we do not know what 0 means, since there is no text label.
You can redefine by yourself, of course. It is a simple mapping between a value to text.
Looks like you posted the same question at here, and got a same answer already. Unfortunately, this exercise does not provide a text label…
You may be interested in this thread.
I listed up some data sources that provides image and annotation which should include label text. The number of classes is relatively large which may require additional time for training. And, you need to modify code slightly, but I suppose basic structure for the neural network should be similar.
Sure, i’ll check out the thread. Thanks for the help too!
