What about RCT's with more than 2 arms?

Course Q&A AI For Medicine AI For Medical Treatment
1

Hi,

I have a question that is a bit off topic maybe, but I was wondering in light of this course, what if an RCT had 3 treatment arms? Would you be able to asses a models prediction for these three, and secondary, how would you go about predicting for the next patient which treatment would benefit them best? is there a c-for-benefit like metric for these kind of situations?

Thanks for your answer.

Kind regards,

Hanneke

2

Hi @Hanneke,

There are two parts to your question, and I will try to answer them to the best of my knowledge:

1- RCT with 3 treatment arms:
To do such analysis, you would need to do comparative pair-wise analysis between the 3 arms. This is the limitation of RCT, but you can still achieve what you want. Once you have done this pairwise analysis, you can see which one is statistically better by using methods such as Anova,…

2- Predicting next patient’s treatment:
You would need to develop the usual predictive models. For that you would need the patient’s data, medical history, etc…

I hope that helps.

regards,
Samuel

3

Thanks for your reply Samuel,

If I understand you correctly you would advise to compute a c-index for treatment 1 vs treatment 2, treatment 1 vs treatment 3, and treatment 2 vs treatment 3, and afterwards to perform statistics on these outcomes? Interesting, I will think about that… If I would do pairwise analysis, I need to do some kind of correction to prevent type-I errors occuring I suppose. Do you have any ideas about a best approach?

In the end I am looking for the best way to assess which treatment out of let’s say 3 treatments would have the best outcome for a new patient coming into the clinic.

I have al lot of data for patients that have underwent these treatments, a lot of features, as wel as the outcome data (allbeit that it is open label data, and not RCT, so patients are not necesarily matched in age and gender for instance, and remission rates will be different between the treatment groups). I have been training DL as well as XGBoost models and used accuracy, f1 and balanced accuracy as metrics for instance. But now, in this course, learning about the cumulative hazard and risk scores as well as the c-index, I was wondering if I could somehow use these metrics to assess the quality of a classification/prediction model with remission as a target. To asses the ‘risk’ of remitting to a treatment for instance.

I am wondering if you have any ideas about that?
Thanks!

regards,

Hanneke

4

Hi @Hanneke,

For the pairwise analysis:

“If I would do pairwise analysis, I need to do some kind of correction to prevent type-I errors occuring I suppose”: Yes, you need to account for the type I errors, I believe, as you compare different pairs. You need to apply a correction method to adjust this.

As for the rest, This is going way beyond the course. However, two points to consider here:

  • Survival analysis would give you much more insights over XGBoost.

“patients are not necesarily matched in age and gender for instance, and remission rates will be different between the treatment groups”: Be wary of your model with no match in gender and age which will for sure introduce bias into the model.

Regards,
Samuel

5

Hi @Samuel_Chazy

Thanks for your tips! I really appreciate them and I will have another look at my options.

Kind regards,

Hanneke