What is LoRA (Low-Rank Adaptation)?

LoRA (Low-Rank Adaptation)

Definition A parameter-efficient fine-tuning method that trains only a small number of additional parameters instead of the entire model.

In Depth

LoRA freezes the original model weights and adds small trainable matrices to each layer. This reduces fine-tuning cost by 10-100x and allows multiple fine-tuned versions to share the same base model. QLoRA combines LoRA with quantization for even greater efficiency.

Browse more terms

AI Agent AI Alignment Agentic AI Artificial General Intelligence Artificial Superintelligence Attention Mechanism AutoML Autoregressive Model Benchmark Bias (AI) CLIP Chain-of-Thought Computer Vision Constitutional AI Context Window DPO (Direct Preference Optimization) Deepfake Diffusion Model Document AI Edge AI