For logistic regression, we use a different cost funtion, ie loss function. In spite of its gradient descent graph is not a bowl shape like that of the squared error, it also has a smooth downward shape with global minimum point. But why there is also a graph on the right showing the log of the loss function?
Hello @flyunicorn,
We usually use log scale in visualization to magnify the lower range. For example, the left surface “looks” plateau in the middle, but with the log scale, it reveals that the seemingly plateau is actually a valley that is too shallow to see in the normal scale.
Btw, in our MLS, the difference between loss function and cost function is that, the former computes the loss of one sample while the latter sums the losses of multiple samples. Taking squared loss as an example:
Cheers,
Raymond
1 Like
Do not confuse the loss and the cost.
- The loss is a per-example value computed for a single example
- The cost is a value that is computed from all the losses, i.e. a value that is computed over all examples in the batch of measurements.
In the diagram above, the cost seems to be plotted on the right (in the space of w,b). It is indeed not “bowl” formed, as as would be the case for “sum of squared errors”.
The plot on the right apparently has been added to show there is no plateau around the 0 value.
1 Like