Quartz 4

❯

❯

Attention (machine learning)

Attention (machine learning)

Created Apr 10, 2026Modified Apr 17, 2026

Attention is the central mechanism used by Transformer models during updating.

Modifications

Disentangled attention
Invariant point attention

Graph View

Backlinks

Adding structural adaptors to language models leads to improvements in thermostability prediction compared to structure-based NNs alone
Attention matrices from antibody LMs can be used for paratope prediction
Attention matrices of antibody language models do not correspond to contacts
Layer normalization
Membrane proteins are predicted by PLMs via solvent-exposed hydrophobic residues
Mixture-of-depths
PLM attention maps from specific heads can be used to predict allosteric networks
PLM attention matrices correspond to 3D contacts
Positional encoding
Transformer
Transformers attend to gap tokens to "opt out" of updates

Created with Quartz v4.5.2 © 2026

GitHub
Discord Community