Unconditional design does not consistently improve with model scale

Summary

For unconditional sequence generation, scaling from 38M to 650M parameters did not consistently improve pLDDT or self-consistency perplexity of designs (1). This was shown with EvoDiff and is metric- and task-specific: larger protein language models can be more steerable under activation steering even when raw generation metrics plateau (2).

Figures

Ref (1)

See also

• Protein property prediction using PLMs does not benefit from scale except when predicting inferring features of either structural or sparsely populated sequence families

1.

Alamdari S, Thakkar N, van den Berg R, Lu AX, Fusi N, Amini AP, et al. Protein generation with evolutionary diffusion: sequence is all you need. In: NeurIPS 2023 Generative AI and Biology Workshop. 2023. Available from: https://mlanthology.org/neuripsw/2023/alamdari2023neuripsw-protein/

2.

Huang L-K, Zhu R, He B, Yao J. Steering Protein Language Models. In: International Conference on Machine Learning. PMLR; 2025. p. 26247–60. Available from: https://proceedings.mlr.press/v267/huang25ba.html