W3 - RLHF Reward Model - loss of reward model

Course Q&A Generative AI with Large Language Models
1

Why is the loss log(σ(r_j - r_k))? shouldn’t it be -log(σ(r_j - r_k))?

Screenshot_20230930_162714
Screenshot_20230930_1627142340×1080 183 KB

1 Like
2

The choice between these two loss functions depends on the specific task and objective of your machine learning or deep learning model.
The former log(σ(r_j - r_k)))is more of a continuous measure of prediction quality,
while the latter -log(σ(r_j - r_k)) is typically used in binary classification problems where you want to penalize the model for making incorrect predictions.