Objectives and optimization

Training losses, auxiliary objectives, regularization, optimization schedules, and gradient estimators. Optimizing inputs through frozen models belongs under inference/guidance.

Quartz 4

Explorer

Objectives and optimization

Fine-tuning PLMs on rank is more effective than fine-tuning on property (regression)

Straight-through estimator

Using evolutionary information for label smoothing during inverse folding training leads to improved fitness prediction

Adding noise while training non-Ab inverse folding models improves self-consistency while worsening sequence recovery

BERT-based inverse folding models generate overly repetitive sequences

FAPE can be extended to ligands

Frame aligned point error

Including confidence improves DL-based docking

Including structure prediction confidence while training inverse folding improves sequence diversity but not sequence recovery

Inverse folding of CDRs benefits from span masking during training

Inverse folding sequence recovery can be improved by simultaneously learning side chain packing

Loss functions

MSA-based structure prediction models can be retrained as pathogenicity prediction models by upweighting BERT losses

RMSD is a poor training objective for structure prediction

Reinforcement learning outperforms fine-tuning in property-guided inverse folding

Fitness prediction using inverse folding can be improved with auxiliary B-factor and SASA losses

Learning rate decay

Residual connections smoothen the loss landscape of neural networks

Contrastive learning

Focal loss

Weight averaging