Tranception is an autoregressive protein language model that pulls information from Multiple sequence alignments at runtime.
Ref (1)
1.
Notin P, Dias M, Frazer J, Marchena-Hurtado J, Gomez AN, Marks D, et al. Tranception: Protein Fitness Prediction with Autoregressive Transformers and Inference-time Retrieval. In: International Conference on Machine Learning. PMLR; 2022. p. 16990–7017. Available from: https://proceedings.mlr.press/v162/notin22a.html