Need help understanding Bayesian inference and MAP

[Kevin_Shey]

Can a mentor explain how I should interpret the result from this calculation (from the reading assignment Bayesian inference and MAP)

What does it mean when you have:

0.636 if theta = 0.5
0.364 if theta = 0.8

Why is the likelihood lower (0.364) when theta is 0.8 ?

Thank you.

[lucas.coutinho]

Hi @Kevin_Shey!

Thanks for your post and your question.

Firstly, I would note that in this case we are dealing with the actual probability and not the likelihood. The likelihood is usually the value without the denominator in the Bayes Theorem, because for maximizing/minimizing or, more generally, sorting purposes, it does not matter a constant positive term in the denominator.

In this case, it is still more likely for theta to be 0.5 because there is one chance in four of picking the coin with theta = 0.8. To make yourself more convinced, I have made a small python code to run this experiment.

import numpy as np
from scipy.stats import bernoulli

res = np.array([1,1,1,1,1,1,1,0,0,0], dtype = np.int64) ## Resulting array to compare. 7 1's and 3 0's

def throw_coins(theta = 0.5, sample_size = 10):
    return bernoulli.rvs(p = theta,size=sample_size)

p_05 = .5
p_08 = .8
match_05 = 0
match_08 = 0
for i in range(100000):
    coin_picked = bernoulli.rvs(p = 0.75, size = 1)[0] ## Picks the coin. 1 if 0.5 is picked, 0 if 0.8 is picked
    if coin_picked == 1: # If coin with theta = 0.5 is picked, throw it 10 times and check if it gives 7 1's and 3 0's 
        res_05 = throw_coins(p_05)
        if np.allclose(res,res_05):
            match_05 +=1
    else: # If coin with theta = 0.8 is picked, throw it 10 times and do the same checl
        res_08 = throw_coins(p_08)
        if np.allclose(res,res_08):
            match_08 += 1

print(f"Number of times the coin with theta = 0.5 outputs 7 1's and 3 0's: {match_05}")
print(f"Number of times the coin with theta = 0.8 outputs 7 1's and 3 0's: {match_08}")      

Number of times the coin with theta = 0.5 outputs 7 1's and 3 0's: 76
Number of times the coin with theta = 0.8 outputs 7 1's and 3 0's: 50

And from every entry with 7 1’s and 3 0’s (total of 76 + 50 = 126)

Proportion of times that the outcome came from the coin with theta = 0.5: 76/126 = 0.603

Proportion of times that the outcome came from the coin with theta = 0.8: 50/126 = 0.397

I’ve run the code above with 500000 iterations (it took some minutes) and the result was:

Number of times the coin with theta = 0.5 outputs 7 1's and 3 0's: 363
Number of times the coin with theta = 0.8 outputs 7 1's and 3 0's: 208

So the proportion is:

• theta = 0.5: 363/(363+208) = 0.636
• theta = 0.8: 208/(363+208) = 0.364

Which is precisely the value we’ve encountered theoretically.

I hope that helps.

Cheers,
Lucas

[Kevin_Shey]

Thank you for the detailed reply. I will be testing out your code.

Kevin