In Depth
DPO directly optimizes the language model on pairs of preferred vs. rejected outputs, eliminating the reward model training step. This makes alignment cheaper and faster while maintaining quality comparable to full RLHF.
DPO (Direct Preference Optimization)
Definition
A training technique that aligns AI models to human preferences without needing a separate reward model, simplifying the RLHF process.
Browse more terms
AI Agent
AI Alignment
Agentic AI
Artificial General Intelligence
Artificial Superintelligence
Attention Mechanism
AutoML
Autoregressive Model
Benchmark
Bias (AI)
CLIP
Chain-of-Thought
Computer Vision
Constitutional AI
Context Window
DPO (Direct Preference Optimization)
Deepfake
Diffusion Model
Document AI
Edge AI