Training

Updating or adapting pretrained parameters: fine-tuning, adapters, forgetting, transfer, and combining task-specific weights. Test-time parameter updates still belong here.

• Sep 25, 2026

  Antibody-specific fine-tuning improves framework preservation and hotspot targeting in RFdiffusion

  • design/antibodies
  • training/fine-tuning

• Sep 07, 2026

  Fine-tuning PLMs on rank is more effective than fine-tuning on property (regression)

  • training/fine-tuning
  • training/objectives-and-optimization
  • prediction/variant-effects

• Aug 25, 2026

  Ab-Ag inverse folding methods benefit from pretraining

  • design/antibodies
  • training/fine-tuning
  • design/inverse-folding

• Aug 25, 2026

  Fine-tuning almost always improves property prediction

  • training/fine-tuning
  • inference/feature-extraction

• Aug 25, 2026

  Fine-tuning leads to small changes in parameter values in L2 space

  • training/fine-tuning

• Aug 25, 2026

  Low-rank adaptation causes models to learn less and forget less

  • training/fine-tuning

• Aug 25, 2026

  Masked LMs can be fine-tuned starting from autoregressive LMs, but not vice-versa

  • training/fine-tuning
  • model-design/architectures

• Aug 25, 2026

  Pretraining performance does not capture effectiveness on downstream tasks

  • training/pretraining-and-scaling
  • training/fine-tuning
  • evidence/generalization

• Aug 19, 2026

  Fine-tuned protein language models can struggle to generalize to positions absent from fine-tuning data

  • evidence/generalization
  • training/fine-tuning
  • prediction/variant-effects

• Aug 19, 2026

  Pretraining contributes nearly nothing to performance when fine-tuning protein language models in data-rich situations

  • training/fine-tuning
  • evidence/datasets

• Apr 21, 2026

  Ab finetuning improves naturalness of inverse folding designs

  • design/antibodies
  • training/fine-tuning
  • design/inverse-folding

• Apr 21, 2026

  Adding structural adaptors to language models leads to improvements in thermostability prediction compared to structure-based NNs alone

  • prediction/stability-expression
  • model-design/multimodal
  • training/fine-tuning

• Apr 21, 2026

  AlphaMissense

  • prediction/variant-effects
  • training/fine-tuning

• Apr 21, 2026

  Base PLMs must usually be fine-tuned to generate functionally active sequences

  • design/sequence-generation
  • training/fine-tuning

• Apr 21, 2026

  Contrastive learning improves segregation of functional clusters

  • model-analysis/representation-geometry
  • prediction/function
  • training/fine-tuning

• Apr 21, 2026

  Fine-tuning AlphaFold for MHC prediction by adding Evoformer layers improves RMSD and pLDDT

  • training/fine-tuning
  • prediction/structure

• Apr 21, 2026

  Fine-tuning base models on test cases can improve the performance of variant effect and structure prediction

  • training/fine-tuning
  • prediction/variant-effects
  • prediction/structure

• Apr 21, 2026

  Language models can be infused with structure via low-rank adapter layers

  • model-design/multimodal
  • training/fine-tuning
  • design/inverse-folding

• Apr 21, 2026

  MSA-based structure prediction models can be retrained as pathogenicity prediction models by upweighting BERT losses

  • prediction/variant-effects
  • training/fine-tuning
  • training/objectives-and-optimization

• Apr 21, 2026

  PLM embeddings fine-tuned using contrastive learning outperform other representations in drug-target interaction prediction

  • prediction/binding
  • training/fine-tuning
  • inference/feature-extraction

• Apr 21, 2026

  Reinforcement learning outperforms fine-tuning in property-guided inverse folding

  • design/inverse-folding
  • training/fine-tuning
  • training/objectives-and-optimization

• Apr 21, 2026

  Sequence homology composition can affect performance of fine-tuned protein language models for variant effect prediction

  • evidence/datasets
  • training/fine-tuning
  • prediction/variant-effects

• Apr 21, 2026

  Sequences with lower log-likelihoods are worse for zero-shot variant effect prediction using PLMs

  • prediction/variant-effects
  • prediction/confidence
  • training/fine-tuning

• Apr 21, 2026

  Unfreezing PLM during structure training improves prediction quality

  • training/fine-tuning
  • prediction/structure

• Apr 21, 2026

  Contrastive fine-tuning PLMs on inverse folding embeddings of experimental structures but not computational models improves downstream tasks

  • model-design/multimodal
  • training/fine-tuning
  • evidence/datasets

• Apr 21, 2026

  Different versions of low-rank adaptation have equivalent performance after controlling for learning rate

  • training/fine-tuning

• Apr 21, 2026

  Grokking

  • training/pretraining-and-scaling
  • training/fine-tuning

• Apr 21, 2026

  Low-rank Adaptation

  • training/fine-tuning

• Apr 21, 2026

  Low-rank adaptation changes the magnitude, but not directionality, of matrix operations relative to fine-tuning

  • training/fine-tuning

• Apr 21, 2026

  Low-rank adaptation with even fewer parameters can operate on truncated SVD matrices, but only with reinforcement learning

  • training/fine-tuning

• Apr 21, 2026

  Overtrained language models are more difficult to fine-tune

  • training/fine-tuning
  • training/pretraining-and-scaling

• Apr 21, 2026

  Weight-decomposed low-rank adaptation

  • training/fine-tuning

• Apr 20, 2026

  Contrastive learning

  • training/fine-tuning
  • inference/feature-extraction
  • training/objectives-and-optimization

• Apr 20, 2026

  Model merging

  • training/fine-tuning

• Apr 17, 2026

  Catastrophic forgetting

  • training/fine-tuning

Training losses, auxiliary objectives, regularization, optimization schedules, and gradient estimators. Optimizing inputs through frozen models belongs under inference/guidance.

• Sep 07, 2026

  Fine-tuning PLMs on rank is more effective than fine-tuning on property (regression)

  • training/fine-tuning
  • training/objectives-and-optimization
  • prediction/variant-effects

• Jul 19, 2026

  Straight-through estimator

  • training/objectives-and-optimization
  • model-design/multimodal
  • inference/guidance

• Apr 21, 2026

  Using evolutionary information for label smoothing during inverse folding training leads to improved fitness prediction

  • prediction/variant-effects
  • training/objectives-and-optimization
  • design/inverse-folding

• Apr 21, 2026

  Adding noise while training non-Ab inverse folding models improves self-consistency while worsening sequence recovery

  • design/inverse-folding
  • training/objectives-and-optimization
  • evidence/design-validation

• Apr 21, 2026

  BERT-based inverse folding models generate overly repetitive sequences

  • design/inverse-folding
  • training/objectives-and-optimization

• Apr 21, 2026

  FAPE can be extended to ligands

  • training/objectives-and-optimization
  • prediction/ligand-docking

• Apr 21, 2026

  Frame aligned point error

  • training/objectives-and-optimization
  • prediction/structure

• Apr 21, 2026

  Including confidence improves DL-based docking

  • prediction/ligand-docking
  • training/objectives-and-optimization

• Apr 21, 2026

  Including structure prediction confidence while training inverse folding improves sequence diversity but not sequence recovery

  • design/inverse-folding
  • training/objectives-and-optimization
  • evidence/design-validation

• Apr 21, 2026

  Inverse folding of CDRs benefits from span masking during training

  • design/antibodies
  • training/objectives-and-optimization
  • design/inverse-folding

• Apr 21, 2026

  Inverse folding sequence recovery can be improved by simultaneously learning side chain packing

  • design/inverse-folding
  • training/objectives-and-optimization

• Apr 21, 2026

  Loss functions

  • training/objectives-and-optimization

• Apr 21, 2026

  MSA-based structure prediction models can be retrained as pathogenicity prediction models by upweighting BERT losses

  • prediction/variant-effects
  • training/fine-tuning
  • training/objectives-and-optimization

• Apr 21, 2026

  RMSD is a poor training objective for structure prediction

  • training/objectives-and-optimization
  • prediction/structure

• Apr 21, 2026

  Reinforcement learning outperforms fine-tuning in property-guided inverse folding

  • design/inverse-folding
  • training/fine-tuning
  • training/objectives-and-optimization

• Apr 21, 2026

  Fitness prediction using inverse folding can be improved with auxiliary B-factor and SASA losses

  • prediction/variant-effects
  • training/objectives-and-optimization
  • design/inverse-folding

• Apr 21, 2026

  Learning rate decay

  • training/objectives-and-optimization

• Apr 21, 2026

  Residual connections smoothen the loss landscape of neural networks

  • model-design/architectures
  • training/objectives-and-optimization

• Apr 20, 2026

  Contrastive learning

  • training/fine-tuning
  • inference/feature-extraction
  • training/objectives-and-optimization

• Apr 20, 2026

  Focal loss

  • training/objectives-and-optimization

• Apr 17, 2026

  Weight averaging

  • training/objectives-and-optimization

Pretraining objectives, masking, model/data/compute scaling, and foundation-model learning dynamics. Dataset composition is covered by evidence/datasets when central.

• Aug 25, 2026

  Pretraining performance does not capture effectiveness on downstream tasks

  • training/pretraining-and-scaling
  • training/fine-tuning
  • evidence/generalization

• Aug 19, 2026

  Unconditional design does not consistently improve with model scale

  • design/sequence-generation
  • training/pretraining-and-scaling

• Jul 28, 2026

  MSA Transformer

  • training/pretraining-and-scaling
  • model-design/architectures

• Jul 28, 2026

  PLMs are biased by uneven distribution of sequence data in datasets such as UniRef and UniProt

  • evidence/datasets
  • training/pretraining-and-scaling

• Jul 28, 2026

  Sequence-only protein language models implicitly cluster proteins at fineness levels that increase with size

  • model-analysis/interpretability
  • training/pretraining-and-scaling

• Jul 28, 2026

  Training antibody language models on normalized mutation frequencies improves zero-shot expression prediction

  • prediction/stability-expression
  • training/pretraining-and-scaling
  • antibodies/maturation

• Jul 22, 2026

  Protein property prediction using PLMs does not benefit from scale except when predicting inferring features of either structural or sparsely populated sequence families

  • training/pretraining-and-scaling
  • prediction/variant-effects

• Jul 22, 2026

  Quanta hypothesis

  • training/pretraining-and-scaling
  • model-analysis/interpretability

• Jul 20, 2026

  Explicit germline encoding improves prediction of non-germline CDRH3 residues

  • training/pretraining-and-scaling
  • antibodies/repertoires

• Jul 17, 2026

  Antibody-antigen complex prediction scales with training data and compute

  • prediction/complexes
  • training/pretraining-and-scaling
  • evidence/datasets

• Apr 21, 2026

  AlphaFold3 learns intramolecular interactions faster than intermolecular interactions

  • training/pretraining-and-scaling
  • prediction/complexes

• Apr 21, 2026

  Alternate sequence clustering schemes outperform uniform sampling when training protein language models

  • evidence/datasets
  • training/pretraining-and-scaling

• Apr 21, 2026

  Alternative noise schedules improve training of multimodal PLMs

  • training/pretraining-and-scaling
  • model-design/multimodal

• Apr 21, 2026

  BERT models trained on sequence clusters outperform those trained on all data

  • evidence/datasets
  • training/pretraining-and-scaling

• Apr 21, 2026

  Distillation

  • evidence/datasets
  • training/pretraining-and-scaling

• Apr 21, 2026

  ESM

  • training/pretraining-and-scaling
  • inference/feature-extraction

• Apr 21, 2026

  IPA-based structure prediction neural networks train sequentially along three spatial axes

  • training/pretraining-and-scaling
  • prediction/structure

• Apr 21, 2026

  Including sequences from multiplexed ancestral sequence reconstruction improves PLM training

  • evidence/datasets
  • evolution/ancestry
  • training/pretraining-and-scaling

• Apr 21, 2026

  Larger PLMs are better at homolog detection

  • evolution/homology
  • training/pretraining-and-scaling

• Apr 21, 2026

  Larger PLMs are better at thermostability prediction

  • prediction/stability-expression
  • training/pretraining-and-scaling

• Apr 21, 2026

  Larger PLMs generate more novel sequences from more sparsely populated protein families

  • design/sequence-generation
  • training/pretraining-and-scaling

• Apr 21, 2026

  Masked PLMs are more sensitive to training imbalances than autoregressive PLMs

  • evidence/datasets
  • training/pretraining-and-scaling

• Apr 21, 2026

  Optimal scaling of antibody LMs improves prediction of masked CDRH3 residues but not framework residues

  • training/pretraining-and-scaling
  • antibodies/architecture

• Apr 21, 2026

  Paired antibody LMs outperform equivalent unpaired LMs on sequence recovery for various regions

  • training/pretraining-and-scaling
  • antibodies/architecture
  • evidence/datasets

• Apr 21, 2026

  ProGen

  • training/pretraining-and-scaling
  • design/sequence-generation

• Apr 21, 2026

  Protein language models have a lower bound of training loss regardless of size

  • training/pretraining-and-scaling

• Apr 21, 2026

  Protein language models make equally effective predictions when trained on individual proteins or protein families

  • training/pretraining-and-scaling
  • prediction/variant-effects

• Apr 21, 2026

  Protein models designed using inverse folding can be used to supplement training DBs for PLMs and structure prediction models

  • evidence/datasets
  • training/pretraining-and-scaling

• Apr 21, 2026

  Subnetworks within protein language models encode specific protein families

  • model-analysis/interpretability
  • training/pretraining-and-scaling

• Apr 21, 2026

  Training on newly updated databases doesn't guarantee better performance

  • evidence/datasets
  • training/pretraining-and-scaling

• Apr 21, 2026

  Tranception

  • prediction/variant-effects
  • training/pretraining-and-scaling

• Apr 21, 2026

  Correlation between sequence log-likelihood and variant effect prediction performance breaks down as PLMs get larger

  • prediction/variant-effects
  • prediction/confidence
  • training/pretraining-and-scaling

• Apr 21, 2026

  Grokking

  • training/pretraining-and-scaling
  • training/fine-tuning

• Apr 21, 2026

  Including optimal growth temperature during pre-training of PLMs improves prediction and design of thermostability

  • prediction/stability-expression
  • training/pretraining-and-scaling
  • design/sequence-generation

• Apr 21, 2026

  On fixed compute budgets, mixture-of-experts models outperform dense models

  • model-design/architectures
  • training/pretraining-and-scaling

• Apr 21, 2026

  Overtrained language models are more difficult to fine-tune

  • training/fine-tuning
  • training/pretraining-and-scaling

• Apr 21, 2026

  Scaling hypothesis

  • training/pretraining-and-scaling

• Apr 20, 2026

  Protein language models

  • training/pretraining-and-scaling
  • inference/feature-extraction

• Apr 17, 2026

  BERT

  • model-design/architectures
  • training/pretraining-and-scaling