Help for ML develop a deep learning model

Hello , I am currently working on new project which consist to develop a model which can predict price of claim insurance depend to informations about clients, contracts and claims datas. The trainins set had over 5400 examples of data . Iused a neural network model and I got overfitting with low MSE on training set and big MSE on CV and Test set. I’ve try different method, like feature engineering, regularization term tuning, adjust number of epochs…But I still got the same thing, low MSE on training set and big difference on Cv and Test set .

Please I need help, because, it’s an important project for me !

Did you try just adding regular old L2 regularization?

As an experiment, keep increasing the lambda value until you get a much higher training cost. Then check the validation cost and compare the two.

1 Like

Also I think, your dataset seems to be small and probably has much variability (meaning different data not much related to each other). You might need a bigger dataset which can introduce less variability among the data.

In the cases of small datasets I think Cross Fold Validation can help in improving performance too!

1 Like

I did a loop throught different values of lamba, and try to find with value of lamda I will get the balance between Training MSE, Test and Cv MSE.

For differents values of lamba, I got the balance with high errors ( Around 1.xxx) for training, Cv and Test MSE.

To be more precious, I’ve scaled the x train and y train, so error need to be close to 0 …

You’ll only get a cost near zero if the model is complex enough to give a good fit.

Your results suggest the model isn’t complex enough, or it isn’t training until convergence.

What optimizer are you using?

If that doesn’t help, then maybe try adding more hidden layer units, or more hidden layers.

1 Like

I am using Adam optimizer.
Normally, if data are scaled for the training part, a good alorithm should return a near 0 no?

In practical use, the minimum cost is never going to be zero.

And that’s not the goal - the goal is to find the weights that give the minimum cost - it does not have to be a small value.

What parameters are you setting?

I was training, neural network,
If I use a simple with low number layers in neural network, I got high error, in training and cv set.
If I am trying to go further away by adding layers no make the model fit well, I willk get overfitting, I was trying to apply features engineering, or L2 regularization, or try to adjust the nulber of epochs and batch size, but still got the overffiting issue.

Adding layers has nothing to do with the Adam optimizer settings.

increase training samples
reduce features
implement regularization as suggested by few
some implement historical based prediction