Multi-sequence protein language models

Multi-sequence protein language models use a context of related sequences, either through multiple sequence alignments or through unaligned sequences pre-pended to the query.

Models

Alignment-based, which essentially treat the input as a 2D sequence

• MSA Transformer
• ProteinNPT
• Evoformer

Alignment-free, which pre-pends the homologous sequences to the query that is being subject to inference.

• ProFam1
• Dayhoff
• ProtMamba

Notes

• Variant effect prediction with homology-aware PLMs improves with ensembling of multiple prompts