In Exercise 7 - (c_for_benefit_score) - I found the introductory explanations particularly confusing. In particular:
“concordant pair: the observed benefit and predicted benefit of pair 1 are both less than those for pair 2; or, the observed and predicted benefit of pair 1 are both greater than those for pair 2. Also, it should be a permissible pair of pairs.”
This is true if we do not deal with the Absolute Risk Reduction, but with the treatment effect (as in the video in the course). But here, the pairs are presented as characterized by their ARR. For a higher ARR, we expect a lower (more negative) observed effect.
But in c_for_benefit_score (# UNQ_C7), if I use the following line as a condition for concordance, I do not pass the c_for_benefit_score_test test:
The test c_for_benefit_score_test expects the expression to be positive (as in the video where we had the treatment effect on the death probability, not the ARR). To add some confusion, c_statistic_test does not work in my case if the expression is positive , but does if the expression is negative (in which case c_for_benefit_score_test fails). But that is probably a faulty implementation of mine for which support would be appreciated. tmp_cstat_test also works fine with the negative expression.
Welcome to the Discourse community. Thanks a lot for asking this question. In my reply I will provide the definition of Treatment Effect (TE) and Absolute Risk Reduction (ARR). I will also provide an additional hint to your problem. Thanks a lot for bringing this up.
ARR: Lets say we gave a particular treatment to 20% of the patients with a disease but our treatment could not save 12% of the patients. This means that 20%-12% = 8%. That 8% is the ARR. It means that we have saved 8% of the patients from a bad outcome.
TE: The causal effect of a given treatment or intervention (for example, the administering of a drug) on an outcome variable of interest. On an absolute scale treatment effect is not constant.
As a hint on checking for concordance:
You do not necessarily have to write a single statement to handle all different cases where you increase the concordant count. I see that you have understood that the value should be negative/positive and you are trying to manage that using a multiplication and a “< 0”.
Instead of managing all different cases in a single statement, just divide all cases with an “and” statement.
in python you can write
if (…condition1… and/or …condition2… and/or …condition3…):
and specifying your conditions for increasing concordance.
For understanding this syntax better, I would like to share a helpful link:
I think if you break down your single “multiplication condition < 0” into smaller conditions, you will understand what you are missing. You already show significant progress with your understanding of negatives and positives with your multiplication statement. I think attempting to divide all the cases in your multiplication statement can help you realize what you are missing.
If you have any additional questions, please feel free to ask it here as I will try to help you with the best of my ability.
Best,
Can
Regarding the definition you give of the ARR, I think it is not really precise since it does not measure the reduction in the risk of the event. We could imagine that patients with the disease with or without treatment have a risk of dying of 60% (12%/20% from your example). In that case, and by assumption, if we give a treatment to 20% of the population and 60% of this treated population dies (leading to the 8% people not dying), the treatment would have had no effect and the Absolute risk reduction would be 0%, not 8%.
Regarding the Treatment effect, I had in mind the relationship mentioned in the video " Average Treatment Effect", where the Average Treatment Effect is simply the negative of the Absolute Risk Reduction.
For the equation, I think it covers the cases to handle already, but I am grateful for any hint pointing to my mistake. In case we use the treatment effect (-ARR), we should have:
(obs_benefit difference > 0) AND (pred_benefit > 0) | concordant | product >0
(obs_benefit difference > 0) AND (pred_benefit < 0) | not concordant | product <0
(obs_benefit difference < 0) AND (pred_benefit > 0) | not concordant | product <0
(obs_benefit difference < 0) AND (pred_benefit < 0) | concordant |product >0