Is there anyone can let me know when we calculate the simple loss function when it will be negative infinity?
How to understand the below sentence in the video?
"And for some similarity functions, this simple loss function can even go to negative infinity. "
This means that you could construct such a simple loss function that could go to negative infinity. For example Euclidean distance is bound to [0, ∞).
In a video example for the loss we use cosine similarity which would not go to negative infinity. The loss function in the example is bound to [-2, 2] (diff = s(A, P) - s(A, N)) but if instead you used Euclidean distance for similarity s(A, N) you could easily imagine the value close to infinity. Another example could be just to construct loss function which has a denominator close to 0.