Where are ambiguous locations in left graph ? Is it at the intersection of two circles?
And are these 6 circles the 6 decision boundaries computed through gradient descent?
Whether the reasoning to "What would you expect a neural network model do? " is correct?
It would predict fairly good for huge chunk of dataset through 6 units of softmax outputs and would misclassify on some of the both training and cv data as well!
In left graph, The answer to underfitting → some other curve rather than a circle which would only cover sparse example?
In left graph, The answer to overfitting → complex curves trying to include some of the misclassified data points and exclude the ambiguous data points?
I am confused how the right graph’s decision boundary is drawn and what does it mean by model created knowing the source of data? Anyways we are creating the model through neural network architecture by the data from certain source?
Hello @Shashank_Garg, how are you?
(1) Yes, it is the intersection of any 2 circles.
(2) No. They are samples generated by
gen_blobs, right? can you verify that? since they are generated, and no algorithms are trained yet, so if you read the code carefully, those circles or decision boundaries are based on the expected truth out of the sample generation.
(3) Yes that you will need to have an output layer of 6 units and a softmax activation for it. Yes that it will misclassify only some samples. So I agree with you
(4 & 5) As I said in (2), the left graph is the truth, so I think your discussion in (4) & (5) should have been on the right graph. However, I agree with your answers that the shape of the boundary lines are different from those in the right graph. If it is overfitting, as you said, the boundary lines will be more complex to try to capture all data points that would have misclassified. For the “ambiguous” data points that you said, I believe you mean data points that are so closed or even overlapped. So I would say Yes again that a boundary line can only put overlapped data point to one side of it, so even a very complex boundary can’t differentiate them.
(6) Yes - “we are creating the model through neural network architecture by the data from certain source?” so I suggest we forget about “model created knowing the source of data?”. For the boundaries, a neural network actually produces boundary lines instead of circles. Those circles on the left graph were drawn for visualization purpose given the truth. Remember there are 6 neurons in the output layer? If you look at the neurons one-by-one, each neuron is like a logistic regression and you can draw a boundary line to describe what the logistic regression does. Right?
I am very fine sir and embracing the beauty of machine learning techniques. Just that the coding part and learning about libraries seem tough to me.
Yes Sir I actually misunderstood the message conveyed by the graphs and did not made an effort to read the code and I am sorry for that. Also this reflection helped me to ponder over the loopholes in my foundational study! Thank you Sir!
Thank you for sharing your thoughts in the first post of this thread. Please feel free to let us know if you have other questions