What is Reinforcement Learning from Human Feedback?

Reinforcement Learning from Human Feedback

Definition A training method where human raters compare model outputs and their preferences are used to train a reward model, which then guides further RL training. RLHF aligns model behavior with human values and instructions far more effectively than pure supervised learning alone.

In Depth

The process has three stages: supervised fine-tuning on demonstrations, training a reward model on human preference rankings, and optimizing the LLM against the reward model using PPO or similar RL algorithms. RLHF was central to creating InstructGPT and subsequently ChatGPT, Claude, and other assistant models. Its main limitation is that quality depends heavily on the consistency and representativeness of human raters.

Browse more terms

AI Agent AI Alignment Agentic AI Artificial General Intelligence Artificial Superintelligence Attention Mechanism AutoML Benchmark Bias (AI) Chain-of-Thought Computer Vision Deepfake Diffusion Model Edge AI Embedding Federated Learning Few-Shot Learning Fine-Tuning Function Calling GPT