HI I was looking at the video about randomized controlled trials in week 1, I have a doubt, I would really appreciate if someone can help me with it.
The doubt is as follows:
what is p value and why is it that the lower it is, more significant it is considered?
Hi Sukhjot,
Per my understanding, a p-value is the probability of observing a dataset this extreme (that is, the dataset you have), assuming it was produced by random chance alone and the treatment had no actual effect on the data-generating process.
So a lower p-value means there is a lower chance that the data was produced by random chance alone. This lower p-value then provides stronger evidence that the outcome was genuinely driven by the treatment (at least to a much larger extent), making the result more statistically significant.
I hope this does help.
to understand why lower p-value is considered more significant statistically, one needs to understand the normal distribution curve and standard deviation of mean.
A given sample data distribution follows the usual bell curve most of us know
Probability distribution follows 68 - 95- 99.7 rule
- 68% of data lies within 1 standard deviation of the mean.
- 95% of data lies within 2 standard deviations of the mean.
- 99.7% data lies within 3 standard deviations of the mean.
The true value or true representation of feature related to the data is presented in the middle part of the curve. This is also represented as the 95% confidence interval as it is closet to the mean (see the image below) representing true data points of sample.
Now when we consider the null hypothesis of experimental research, which assumes any x variable or a feature such as medicine A cures significant amount of sugar patient occured just by chance, at this time p-value will either be equal to 0.05(0.005 depending on sample size) or less than 0.05(0.005)
When the p-value is equals or less than 0.05, the data comes under the tail end outside the 95% confidence interval signifying the null hypothesis assumption of random occurance as true, otherwise when the p-value is greater than 0.05, the occurance is more closer to the true value or the data represented.
Remember any statistical study is never solely on just p-value or null hypothesis, more subsequent statistical testing is done based on the data, its feature, and usually is combined with
-
chi-square test used for categorical data to see if groups or counts differ from what one usually expects.
-
paired versus unpaired tests
- Paired tests compare the same group twice (like before and after a test).
- Unpaired tests compare two different groups (like group A and group B)
- Bayesian method where probability and statistics used to update the likelihood of a hypothesis as more evidence becomes available.
-
(P(H|E)) (Posterior): The updated probability that hypothesis (H) is true given the new evidence (E).
-
(P(E|H)) (Likelihood): The chance that evidence (E) happens if hypothesis (H) is true.
-
(P(H)) (Prior): Your initial guess or belief about (H) before seeing new data.
-
(P(E)) (Evidence): The total chance of observing the new evidence across all possible.
So it is not about lower p-value being more significant, but probability of data point being far away from true value or true representation of the data sample.
Regards
Dr. Deepti
Got it, you explained it very well Dr. Deepti. I am very grateful.
Thank you.
This did help me build an intuition.
My thanks Sir!
just to add some more insight on
How a statistical research experiment is approached 
[ Research Question / Hypothesis ]
↓
[ Experimental Design ] —> (Control Group vs. Treatment Group)
↓
[ Data Collection ] —> (Random Sampling / Measurement)
↓
[ Statistical Testing ] —> (Choose Test based on Data Type)
↓
±----±-------±-----+
↓ ↓
[ Parametric ] [ Non-Parametric ]
(Normal Data) (Skewed/Ordinal Data)
I. In case of normal data parametric tests are done
-
1.T-Test (Independent) which compares two group means.
for exam does a new blood pressure medication lower systolic blood pressure more than a placebo.
group 1(treatment group) 30 patients given new drugs with mean drop = 14.2 mmHg (Standard deviation = 3.1).
group 2 (control) 30 patients given a placebo. Mean drop = 4.8 mmHg (Standard deviation = 2.5).
Null Hypothesis there is no true difference in blood pressure reduction between the drug and the placebo.
Alternative Hypothesis there is a true difference in blood pressure reduction between the two groups.
Test interpretation
independent variable medication type (categorical: drug versus placebo).
dependent variable if there is change in systolic blood pressure in mmHg (continuous/metric).
assumptions checked data in both groups are normally distributed, variances are roughly equal and patient samples are completely independent.
result analysis calculating the test yields a t-value resulting in a p-value < 0.001. because p-value is less than the standard threshold of 0.05, you reject the null hypothesis. data confirm the new drug provides a statistically significant reduction in blood pressure compared to placebo. -
2.Paired T-Test which compares means from the same group twice. For example measuring the blood pressure of 20 patients before they take a new medication, and then measuring the blood pressure of those exact same 20 patients after finishing the medication.
-
3.ANOVA (Analysis of Variance) compares three or more group means.
Comparing student test scores taught using three different methods traditional, online and hybrid.
Null Hypothesis all teaching methods have the exact same average test score.
Alternative Hypothesis -at least one teaching method produces a significantly different average score.
Independent Variable teaching method (3 groups).
dependent Variable final test score
How ANOVA works
a) between group variance measuring how much the group averages (traditional versus online versus Hybrid) differ from the overall grand average.
b) within group variance measuring how much individual students vary around their own group’s average.
c) F-Statistic this divides the between-group variance by the within-group variance. A high F-value means the differences between the groups are too big to be fluke.
Test result of ANOVA
High p-value (insignificant) if p > 0.05, you accept the null hypothesis. The teaching methods do not perform differently.
*Low p-value" (significant) if p ≤ 0.05, you reject the null hypothesis. At least one method gives different results, so you run a post-hoc test (like Tukey’s test) to see which specific method stands out. -
4.Pearson Correlation measures the linear link between two numeric values. For example measuring the strength of the linear relationship between study hours and exam scores.
variable 1 is hours spent studying.
variable 2 what exam percentage student got
Test checka if an increase in study hours directly relates to a steady, predictable increase in test score.
II. Now comes to skewed or ordinal dara non-parametric tests are done.
-
1.Mann-Whitney U Test compares two independent groups. For example comparing the ranks of customer satisfaction scores between two store branches.
independent variable would be store location (north branch or south branch).
dependent variable customer satisfaction rating (Ranked 1 to 5 stars, skewed data). Test is checking if the north branch gets higher satisfaction rankings than the south branch. -
2.Wilcoxon Signed-Rank Test compares two related groups. For example comparing employee stress ranks before and after a wellness workshop.
independent variable is time (pre-workshop versus post-workshop).
dependent variable stress level rank (1 = Low, 10 = High).
Testing if the same 15 employees report lower stress rankings after attending a meditation workshop. -
3.Kruskal-Wallis Test compares three or more independent groups. For example compare pain relief ranks among three different brands of medication.
independent variable medication type (Drug A vs. Drug B vs. Drug C).
dependent variable* pain relief rank (1 = No relief, 5 = Complete relief).
Testing if patient pain relief rankings differ across three competing painkiller brands when the data is not normally distributed. -
4.Chi-Square Test this tests links between categoric variabless.
For example variable 1 is gender (categorical - male, female, Non-binary).
Variable 2 is movie genre (categorical: sci-Fi, comedy, drama).
Testing is counting the frequency of movie choices to see if a person’s preferred genre depends on their gender.
This is the overall all approach of statistical testing done in research.
Regards
Dr. Deepti
hi @DAResaid
yes you are right, I did have doubt that p(e|h) seems incorrect, thanks for the correction