Hello, community!
I have a question. When implementing custom loss functions that take hyperparameters, using a factory function or a class is just a matter of style or preference, or is there an objective criterion that should guide that decision?
Thanks in advance for your opinions.
Hi @fabricio
this finally depends on if you wanna reuse the loss function in other parts of your project, how relevant it is to fellow programmers in your team and how complex it is.
The loss function should be as much aligned with the business goal, that you are solving, as possible. E.g. if a simple model with medium accuracy if perfectly fine to solve the business problem, it makes sense to encourage a simpler model by regulating the loss function a bit more so that no overfitting occurs and the model is rather on the underfitting side.
Based on the complexity you can decide how to implement it. Side note: Experience shows in reality the model has to be just „good enough“ to solve the business problem.
Best regards
Christian
Hey Christian, thanks a lot for your reply. It makes sense.
After posting the question, I continued wondering where a Class implementation would be more suitable than a factory.
I think a Class would make sense only if the loss function should have some internal state that should be kept from training example to training example (maybe some autocorrelated time series data where the loss should change after a certain sequence is present?). I’ll end there since I still can’t think of a specific case.
Again, thanks a lot for your answer. It has been insightful.
Cheers!