Summary
Fine-tuning base models (such as transformers) on test data of interest can lead to improved prediction of protein structure and variant effect (1). This was shown with ESMFold using standard masked language model loss, leading to improvements in pLDDT and zero-shot variant effect prediction. Low-rank adaptation is used to overcome the cost of backpropagation across such a large network.
Figures
Ref (1)
1.
Bushuiev A, Bushuiev R, Pimenova O, Zadorozhny N, Samusevich R, Manaskova E, et al. One protein is all you need. 2024; Available from: https://arxiv.org/abs/2411.02109