Summary
PLMs and PLM-based structure predictors generalize to some de novo designed proteins (1). Synthetic proteins designed using ESM were successfully expressed in the wet lab and adopted a structure consisted with predicted contacts. Likewise, ESMFold correctly predicted the structures of other de novo designed proteins (such as those designed by (2); see (3)).
Details
They use Gibbs sampling to generate new protein backbones by jointly considering and (see below).
Figures

Ref (1)
See also
1.
Verkuil R, Kabeli O, Du Y, Wicky BIM, Milles LF, Dauparas J, et al. Language models generalize beyond natural proteins. openRxiv; 2022. Available from: https://doi.org/10.1101/2022.12.21.521521
2.
Praetorius F, Leung PJY, Tessmer MH, Broerman A, Demakis C, Dishman AF, et al. Design of stimulus-responsive two-state hinge proteins. Science. 2023;381(6659):754–60. Available from: https://doi.org/10.1126/science.adg7731
3.
del Alamo D, Jeliazkov JR, Truan D, Karpiak JD. Conformational sampling and interpolation using language-based protein folding neural networks. openRxiv; 2023. Available from: https://doi.org/10.1101/2023.12.16.571997