Summary
Concatenating features from ESM-1b and GEARNet was found to be less effective than either cross-attention, ESM-to-GearNet, or even just ESM-1b embeddings alone (1). Meanwhile, (2) found that concatenation was outperformed by the use of ResNet as an autoencoder bottleneck.
Figures
| Method | F_max | AUPR |
|---|---|---|
| ESM-1b | 0.864 | 0.889 |
| ESM-GearNet | ||
| - w/ parallel fusion | 0.733 | 0.759 |
| - w/ series fusion | 0.883 | 0.890 |
| - w/ cross fusion | 0.880 | 0.893 |
Ref (1)
1.
Zhang Z, Wang C, Xu M, Chenthamarakshan V, Lozano A, Das P, et al. A Systematic Study of Joint Representation Learning on Protein Sequences and Structures. 2023; Available from: https://arxiv.org/abs/2303.06275
2.
Detlefsen NS, Hauberg S, Boomsma W. Learning meaningful representations of protein sequences. Nature Communications. 2022;13(1). Available from: https://doi.org/10.1038/s41467-022-29443-w