Architectures

Sep 07, 2026

Disentangled attention for learning joint sequence-structure representations

Aug 25, 2026

Masked LMs can be fine-tuned starting from autoregressive LMs, but not vice-versa

Jul 28, 2026

MSA Transformer

Jul 28, 2026

Query-conditioned outer product

Apr 21, 2026

AF3-generation methods can be substantially shrunk without sacrificing prediction quality, except on PPIs

Apr 21, 2026

Evoformer

Apr 21, 2026

Geometric Vector Perceptrons

Apr 21, 2026

IgFold

Apr 21, 2026

Increasing RosettaFold embedding dimension improves structure prediction

Apr 21, 2026

Invariant point attention

model-design/architectures

Apr 21, 2026

Multi-sequence protein language models

Apr 21, 2026

ProteinNPT

Apr 21, 2026

SimpleFold

Apr 21, 2026

Sparse autoencoder

Apr 21, 2026

Subquadratic scaling for protein backbone diffusion

Apr 21, 2026

Transformer

model-design/architectures

Apr 21, 2026

Triangular update can be repurposed for inverse folding

Apr 21, 2026

Triangular updates are dispensable for structure prediction

Apr 21, 2026

Invariant point attention is dispensable to structure prediction

Apr 21, 2026

Alpha carbon-only representations are sufficient for ML force fields

Apr 21, 2026

Encoder- and decoder-based models have different strengths and are equally useful on generative tasks

Apr 21, 2026

Layer normalization

model-design/architectures

Apr 21, 2026

Mixture-of-depths

model-design/architectures

Apr 21, 2026

On fixed compute budgets, mixture-of-experts models outperform dense models

Apr 21, 2026

Over-squashing

model-design/architectures

Apr 21, 2026

Positional encoding

model-design/architectures

Apr 21, 2026

Residual connections smoothen the loss landscape of neural networks

Apr 20, 2026

AlphaFold2

Apr 20, 2026

Disentangled attention

model-design/architectures

Apr 20, 2026

Residual connections

model-design/architectures

Apr 20, 2026

RoseTTAFold

Apr 17, 2026

Attention (machine learning)

model-design/architectures

Apr 17, 2026

BERT

Apr 17, 2026

Graph neural networks

model-design/architectures

Apr 17, 2026

Mixture-of-experts

model-design/architectures

Jun 04, 2024

Quartz 4

Explorer

Architectures

Disentangled attention for learning joint sequence-structure representations

Masked LMs can be fine-tuned starting from autoregressive LMs, but not vice-versa

MSA Transformer

Query-conditioned outer product

AF3-generation methods can be substantially shrunk without sacrificing prediction quality, except on PPIs

Evoformer

Geometric Vector Perceptrons

IgFold

Increasing RosettaFold embedding dimension improves structure prediction

Invariant point attention

Multi-sequence protein language models

ProteinNPT

SimpleFold

Sparse autoencoder

Subquadratic scaling for protein backbone diffusion

Transformer

Triangular update can be repurposed for inverse folding

Triangular updates are dispensable for structure prediction

Invariant point attention is dispensable to structure prediction

Alpha carbon-only representations are sufficient for ML force fields

Encoder- and decoder-based models have different strengths and are equally useful on generative tasks

Layer normalization

Mixture-of-depths

On fixed compute budgets, mixture-of-experts models outperform dense models

Over-squashing

Positional encoding

Residual connections smoothen the loss landscape of neural networks

AlphaFold2

Disentangled attention

Residual connections

RoseTTAFold

Attention (machine learning)

BERT

Graph neural networks

Mixture-of-experts

Transformers attend to gap tokens to "opt out" of updates