Bad hyper parameters vs bad model

Course Q&A Professional Certificate:PyTorch for Deep Learning PyTorch: Fundamentals
,
1

For the final assignment of C2W3, in the last section where we are to select hyperparameters to train our DistilBERT model, it suddenly became clear to me how tough this is, and why we were earlier trained to use Optuna for hyper parameter search given how tough it is.

So, for those more experienced, is it only an ‘art’ or does it get better with practice ?

And a grander question here: How do you know when no hyper parameters will work, or you simply have a bad model / error in the model.

I ask this in part because even with the suggested parameters I was still getting lousy loss and F1 scores (i.e. maybe there was something lousy/broken in my model that the grader just ‘missed’) ?

Any advice would be appreciated, that it is not in fact ‘conjuring Voodoo’, but actually a science.

2

It pretty much boils down to trying every trick in your toolbox to get improved performance.

If that doesn’t help, then question whether you’re using an appropriate sort of model.

This gets much easier with experience, because your bag-of-tools will grow.