Hi folks, I’m revising my knowledge of hypothesis testing and watching the video about the definition of hypotheses. There’re some points in the video that cause me misunderstanding so I can’t continue further with the next lessons.
The first is like being shown in the attached image at 2:02 below:
I believe he means “then you can reject the alternative (instead of “null”) hypothesis”.
The second one is in the yellow text on the top right corner of the image. With my understanding above, it follows with “By failing to reject that the email IS spam, you are not accepting that it’s ham”. This is really hard to grasp the concept of hypotheses because previously, I think what you mean is “you can reject the alternative hypothesis”.
Can somebody explain to me and elaborate more on what the lecturer tries to convey in the video, please?
Thank you so much.
In hypothesis testing we always evaluate based on the null hypothesis, so we can either reject or fail to reject the null hypothesis, this happens because we cannot be 100% about the conclusions, we can only evaluate evidence and make our statement by testing how confident we are that the null hypothesis is true or false.
I agree that wording on hypothesis testing might be confusing, but it has a purpose.
The statement “By failing to reject that the email IS spam, you are not accepting that it’s ham” emphasizes that not having enough evidence to label an email as spam doesn’t automatically make it ham. It remains in a state of uncertainty.
Thank you @pastorsoto for giving me your thoughts on it. I totally agree with what you said about “In hypothesis testing we always evaluate based on the null hypothesis, so we can either reject or fail to reject the null hypothesis” because it unifies our understanding to only one thing, which is to reject or not the null hypothesis. With that said, it’d be more relevant if the yellow text said “By failing to reject that the email IS ham, you are not accepting that it’s spam”. What your take on that?
the first step where the null hypothesis is either accepted or rejected based on a probability value or conditional value in distribution on failing to reject the null hypothesis would consider the email to be spam.
You are only failing to reject the null hypothesis of email being ham and this failure or rejection of null hypothesis doesn’t mean that the email is spam.
Thanks @pastorsoto and @Deepti_Prasad. I get it but I still don’t know the logic of why the lecturer stated the conclusion in the yellow color, i.e. “By failing to reject that the email IS spam, you are not accepting that it’s ham” because the purpose of hypothesis testing is to reject or fail to reject the null hypothesis, i.e. the email is ham in this example. The conclusion in the blue text makes sense to me because we’re rejecting the null hypothesis. But the yellow text makes me a bit off the track of grasping this definition and hence makes us not on the same page. It should’ve been “By failing to reject the null hypothesis, you are not accepting the alternative hypothesis”.
Please feel free to correct my understanding if I’m wrong.
I totally get the confusion with the other statement as a student I used to get the same kind of confusion.
let me explain what the image is trying to explain.
The null hypothesis is the statement or claim being made (which we are trying to disprove) where as the alternative hypothesis is the hypothesis that we are trying to prove and which is accepted if we have sufficient evidence to reject the null hypothesis.
if you go by proper definition in statisitics
Null Hypothesis (H0) – This hypothesis states that there is no difference between groups or no relationship between variables. The null hypothesis is a presumption of status quo or no change. Hence the yellow color statement
Alternative Hypothesis (Ha) – This hypothesis should state what you expect the data to show, based on your research on the topic. This is your answer to your research question.
For example,
Null Hypothesis for a condition would state: “There is no difference in the salary of factory workers based on gender”.
where as Alternative Hypothesis would state: Male factory workers have a higher salary than female factory workers.
So the yellow statement for failing to reject that email is spam comes under null hypotheses, and as the hypothesis considers there is no relationship between the conditions, you cannot consider email to be ham, just because you fail to reject that email is spam!!!
Please feel free to ask if you are still confused!!
Thanks @Deepti_Prasad. I started to understand a bit more from your elaboration. I have a question, which is when the yellow statement referred to “By failiing to reject that email is spam (i.e. the alternative hypothesis)”, why do we need to reject the alternative hypothesis? Isn’t that the goal of hypothesis testing is to reject the null hypothesis (i.e. the email is ham) or fail to reject the null hypothesis?
here the hypothesis is considered either their is a relation between the two groups or variable(which is considered as alternative hypothesis),
or there is no relation between the two groups (Null hypothesis) so by considering a null hypothesis we already have established one condition that states if we fail to establish one condition(i.e. here email as spam) does not mean the other condition is considered true i.e. email is ham (this doesn’t mean you would take alternative hypothesis into consideration where we try to prove if email is not spam, then it is ham)
the reason for considering these two types of hypothesis separately is to consider all kind of probabilities could be hold true where a relation between variables or groups exist i.e alternative hypothese (the blue statement) and where relationship between variables or groups do not exist i.e. null hypothesis (yellow statement)
your confusion is coming because you are trying to merge both hypothesis together.
in probability distribution, we can either conditions i.e. either there is a relationship between variables or there is no relationship between variables. we cannot considered both conditions together in a hypothesis.
by stating the above statement you are trying to state there is a relationship between the two conditions stated and that would come under alternative hypothesis which would be incorrect in this case as here the hypothesis is considered to either reject or accept the null hypothesis where there would be no relation between two conditions i.e. if email is not spam, it would be ham would not hold true under the null hypothesis and that is what the yellow statement is trying to state.
Thanks for your time and efforts to elaborate on it @Deepti_Prasad. Coming back to this thread after I finished watching carefully the entire videos in this week for the second time, I still feel like we are not on the same page. Let me explain my point of view on that, what you were trying to explain to me seems like you just wanted to build up my intuition on the definition of hypotheses, which is what I highly believe that I understood. The problem being formulated here is given the null hypothesis H_0:\text{the email is ham}, we want to see what we can conclude about our belief in the alternative H_1:\text{the email is spam}. My wondering is about the yellow text in the attached image. I don’t say that it was a wrong conclusion. I think it was also a valid example. In fact, in later videos, the problem we are trying to address is “We reject or don’t reject the null hypothesis H_0?”. If we reject H_0, then we accept the H_1. If we don’t, there’s not enough evidence to say that we accept H_1. That remains in a state of uncertainty as @pastorsoto said. So by formulating the problem statement like that, the statement in yellow color should’ve been “By failing to reject that the email IS ham (i.e. don’t reject H_0), you are not accepting that it’s spam (i.e. don’t conclude anything about H_1)” to be more consistent. In general, I think failing to reject one of the hypotheses doesn’t mean we’re accepting the other hypothesis. We can only accept a hypothesis if we have enough evidence to reject the other one.
