Possible Error: Week 4\ML Application: A/B Testing

Possible Issue:
Slide and Audio/Transcription have different information which may cause confusion.

Video section starts at minute 7:40

In this slide the instructor notes that the observed statistic is -2.07, while the slide says otherwise (z=-1.336). The p-value seems to align with the slide and the audio/transcript though.

Which leads to another confusion: the instructor says that the p-value is smaller than the significance level(\alpha), when in fact it’s bigger (0.091 > 0.05). The decision on the slide seems correct though: fail to reject H_0 but the audio/transcript says otherwise (reject H_0).

Here is the transcript from that section:
You also consider a significance level of alpha equals 0.05. And as you just saw from the previous slides, if H0 is true, then the test statistic follows a Gaussian distribution with the data you have. This means that the observed statistic is negative 2.07. What is the p-value for the sample? Well, it will be the area under this Gaussian PDF which has a value of 0.091. Since this is smaller than the significance level, you can reject the null hypothesis and accept that as true, that the conversion rate of design B is bigger than the one for design A.

Is this truly an issue with the material, or is my understanding wrong? Please feel free to enlighten me :slight_smile:


@lucas.coutinho, can you cover this question?

Hi @markredito,

You are correct, the video audio is not alligned with the slides. We in fact had some calculations issues in this slide and corrected them, so the slide is the correct value, but the audio is not fixed. Thanks for addressing this issue, I will send the video to our video team to fix it as soon as possible.

Thanks @TMosh for tagging me!


1 Like

Thanks Lucas and TMosh! No worries! Just thinking of other students who may encounter it. Glad its getting fixed soon.

@markredito, the video is fixed! The video quality may be low for the next few hours due to some encoding on Coursera’s side, but the highest quality version will be live soon.


1 Like