Summary
The Protein backbone design method (Chroma) uses sub-quadratic scaling for calculating attention (1). In practice, this leads to attention between a residue and its 20 nearest neighbors plus 40 randomly selected residues obtained using an inverse cubic formula ( scaling). This led to favorable results compared to k-nearest neighbor.
Figures
Figure S4 from (1)
Figure S22 from (1)
1.
Ingraham JB, Baranov M, Costello Z, Barber KW, Wang W, Ismail A, et al. Illuminating protein space with a programmable generative model. Nature. 2023;623(7989):1070–8. Available from: https://doi.org/10.1038/s41586-023-06728-8