Quartz 4

Home

❯

notes

❯

Residual connections

Residual connections

Created Jun 18, 2024Modified Apr 20, 2026

Summary

Residual connections are used in deep neural networks to avoid the Vanishing gradient problem where the gradient does not reach the earlier layers of the model during backpropagation.


Graph View

Backlinks

  • Residual connections smoothen the loss landscape of neural networks
  • Transformer

Created with Quartz v4.5.2 © 2026

  • GitHub
  • Discord Community