Adding structural adaptors to language models leads to improvements in thermostability prediction compared to structure-based NNs alone

Summary

Adding structural adaptor layers to protein language models leads to improvements in thermostability prediction compared to using structure-based neural networks alone (1). This was achieved by adding a cross-attention layer with ProteinMPNN embeddings to each layer of ESM2-650M and fine-tuning on the mega-scale thermostability dataset (2).

Figures

Ref (1)

See also

• Language models can be infused with structure via low-rank adapter layers

1.

Li Z, Luo Y. Generalizable and scalable protein stability prediction with rewired protein generative models. Nature Communications. 2025;17(1). Available from: https://doi.org/10.1038/s41467-025-67609-4

2.

Tsuboyama K, Dauparas J, Chen J, Laine E, Mohseni Behbahani Y, Weinstein JJ, et al. Mega-scale experimental analysis of protein folding stability in biology and design. Nature. 2023;620(7973):434–44. Available from: https://doi.org/10.1038/s41586-023-06328-6