• RLHF — Reinforcement learning from human feedback — training a model to prefer answers humans rate as better, which turns a raw text predictor into a usable assistant.

DPO and IPO can over-optimise likelihood and lose the quality they were tuned for. Understanding where that happens tells you when to stop a preference-tuning run instead of trusting the objective to keep improving.

• articleLanguage Models Don T Always Say What They Think Unfaithful Explanations In Chain Of Thought Prompting 2023 05 07
• articleMteb Massive Text Embedding Benchmark 2023 03 19
• articleConsent In Crisis The Rapid Decline Of The Ai Data Commons 2024 07 19
• articleNo Need For Explanations Llms Can Implicitly Learn From Mistakes In Context 2025 05 21

• articleDirect Preference Optimization: Your Language Model is Secretly a Reward ModelRafael Rafailov et al.
• articleDirect Preference Optimization Beyond Chatbotshuggingface.co
• articleBack To Basics Revisiting Reinforce Style Optimization For Learning From Human Feedback In Llms 2024 02 23Cohere

Checking sign-in…

Loading comments…