What exactly is the difference between Triplet Loss Function and Softmax? My understanding is that Softmax takes multiple images and establishes a network, but it can only process one at a time. Triplet Loss, on the other hand, introduces a new image (after Softmax has been applied) to see if it matches a previous image. Did I interpret this correctly? Feel free to contribute, and any feedback is greatly appreciated!
Softmax classifies objects or features based on probability distribution or class designation between a number of classes or features where as
triplet loss evluates data in groups of 3, Anchor (the baseline), a Positive (same class as the anchor), and a Negative (different class) . It measures the distances between these points to ensure that the anchor is closer to the positive example than it is to the negative example by a margin.
They are different algorithms designed to solve somewhat different problems. Softmax is designed to handle the case of “multiclass classification”: that is where you have a predefined set of object classes or types and you want to recognize which type is represented by a given input. For example, recognizing different types of animals or flowers. Is this picture of a cat, dog, kangaroo, aardvark or elephant? Softmax is trained by using the multiclass version of the Cross Entropy Loss function.
Triplet loss is designed to solve a more subtle problem: you have a collection of variable size of something like human faces and you want to distinguish between them. Meaning answer questions like “are these two pictures of the same person?” or “is this picture of someone in our employee database”. You don’t have a fixed number of different faces, so the softmax approach doesn’t work. The idea is to define a notion of “distance” between different faces. We approach this by training a so-called “embedding model” to analyze faces (or whatever the type of image is) and distill their characteristics as a vector of numbers. Training is driven by the cost function, which is where Triplet Loss comes in.
Of course Professor Ng is a much better explainer than I could hope to be and he covers all this in the lectures. If my comments above don’t shed enough light, then my suggestion would be to watch the relevant lectures again with what I said above in mind. Also note that they cover the purpose and definition of the Triplet Loss function in the assignment as well, so if you haven’t done that yet maybe you should “hold that thought” until you’ve been through the assignment.