RLHF

The Perils of Optimizing Learned Reward Functions: Low Training Error Does Not Guarantee Low Regret