Low-Rank Adaptation (LoRA)

Low-Rank Adaptation (LoRA) is an efficient fine-tuning technique that allows adapting large-scale pre-trained models with minimal parameter updates. By injecting trainable low-rank matrices into the model’s layers, it significantly reduces the memory and compute requirements for task-specific adaptation, making it a cornerstone for customizing large language and image models.