I have a curiosity: if I’m not wrong, Andrew defines Bayes error as the minimal theoretical discrepancy between our prediction and the actual label that anything or anyone can achieve, even in the future. Isn’t this inherently unknown? Maybe not necessarily wrong, but it looks to me that we just cannot possibly know that.
Or maybe this reduces to what the object of analysis is: maybe we are trying to interpret a sound, say a clang, that is supposed to be the word ‘hello’. Could it not be that the label is wrong? How can we know that the clang is in fact a ‘hello’ even if nothing, even in the future, can? Is there an acceptable signal to noise ratio? Maybe I’m missing something, or just babbling.