Summary
Attention matrices of PLMs can capture whether a protein sequence is interpreted as a soluble protein or membrane protein (1). In membrane proteins, more attention is given to solvent-exposed hydrophobic residues.
See also
1.
Wenzel M, Grüner E, Strodthoff N. Insights into the inner workings of transformer models for protein function prediction. Bioinformatics. 2024;40(3). Available from: https://doi.org/10.1093/bioinformatics/btae031