Different versions of low-rank adaptation have equivalent performance after controlling for learning rate

Summary

Different low-rank adaptation implementations have equivalent performance after controlling for learning rate differences (1).

Figures

Ref (1)

1.

Lee Y-A, Ko C-Y, Chen P-Y, Yeh M-Y. Learning Rate Matters: Vanilla LoRA May Suffice for LLM Fine-tuning. 2026; Available from: https://arxiv.org/abs/2602.04998