Suppose i have a modest data set, say of 100 patients and i am planning to classify into 2 groups using 8 predictors (Mostly Magnetic resonance imaging data) should i always try logistic regression models first before trying neural networks . Is it wrong to try to fit a bigger neural network first on the testing data (before going for logistic regression models) to see if the variance is also low on CV data?

Hello @Manul_Das,

If you asked me why I would do logistic regression first, then one reason is that I want something based on a different approach to cross check my results from a neural network model, although I may also say it is a baseline and that my NN should do better than that.

So, I think there is nothing wrong to skip logistic regression, but if you think something is not right when you are training some neural networks, of course it depends on what problems you are facing, but it could be a good idea to see what logistic regression will tell us.

By the way, even if our neural network is overfitting, besides shrinking it back to logistic-regression-like (one layer only and one neuron only), we can also try regularizations.

I think you meant to say “training data”?

Cheers,

Raymond