Linear probes trained on SAE features from intermediate steps of protein structure prediction neural networks can be used for property prediction

Summary

Linear probes trained on SAE features from intermediate steps of protein structure prediction neural networks can be used for property prediction. Linear probes trained on SAE features from intermediate structure-prediction denoising steps can predict design-relevant properties better than standard pLDDT baselines in the reported setting (1).

Figures

Ref (1)

See also

• Sparse autoencoder
• Protein language models learn structure-level features, including disorder, in later layers
• Features for antibody property prediction derived from MD simulations outperform those from language models and static structures

1.

Kim S-J, Vonessen C. bish-bash-fold: what are protein structure prediction models learning? In: The 2026 Workshop on Generative and Agentic AI for Biology. 2026. Available from: https://openreview.net/forum?id=ogUnDI1Qqp