I have a dataset of cats and dogs and trained a model with it. How do I add a thrid option that says not a cat or dog?
Are you already using softmax with 2 labels? If so, then you just add a third label for none and retrain. Of course you may have to relabel your dataset accordingly. If you are treating this as a binary classification with sigmoid as the activation and interpreting “not cat” as the equivalent of dog, then you have a bit more work to do, but it is just a matter of switching to softmax with the appropriate labels.
It can be challenging to label a data set for the “not dog or cat” condition, because there are an infinite number of things that are not dogs or cats. So how do you chose what goes in the training set?
It is a good point that any kind of labeling from scratch is very challenging. I was making the simplifying assumption that you were starting with a pre-existing dataset from Kaggle or the like that already has pictures of other things besides dogs and cats. Reducing pre-existing labels to just the ones you care about is a simpler matter, but creating any kind of labeled dataset from scratch is a significant task.