Why use RL instead of supervised learning?

[Juan_Olano]

I would like to refer you to my previous answer to your previous question as I think the answer is similar:

The magic reward model?

One possible answer to your question: Yes, it could be better to have humans do the rewards, instead of a “simple reward model”. However, the cost and logistics to have the number of humans needed to be able to do this on a very LLM is not practical. For this reason, the Rewards Model is a very good proxy to solve this problem. The Rewards Model provides a very good response and it can be created, re-used, and it can scale, at reasonable costs. Like someone said a long time ago “Perfect is en…

The Constitutional model is one more step in the process of making LLMs better, and in this case, by making sure that the model is aligned with Human Interests and Needs. Still, we face the same problem of scale. So the solution has to be one that is within practical boundaries. I guess over time we will develop better components to achieve these goals. We are just in the infancy of this technology.