How to compare neural network models with different regularization parameters?

Let’s imagine that we are developing neural network model with Tensorflow for linear regression task. We want to try different regularization parameters in order to come up with a better model. But it means that our cost function changes too – it includes both predictions estimation (e.g. Mean squared error etc.) and regularization. So we can’t simply compare costs because they will have different structure.
How can we compare such models? Can we somehow log cost function without regularization term in Tensorflow model?
I use model.summary() in order to estimate cost on my data (train, cross validation, test)

Hello @m.kemarskyi.ip82,

We can define a metric function, and compare the models with it. For example, in model.fit(...), you can pass MeanSquaredError ( full list here ) as metrics :

image

so that TF will evaluate your model on those specified metrics using the training dataset and (if provided) the validation dataset.

Alternatively, but less preferably to me, instead of doing it during model training, you might pass both of the models’ predictions of the validation dataset to a metric function, and see which performs better.

Cheers,
Raymond

1 Like

Thanks! Very useful feature

You are welcome, @m.kemarskyi.ip82!