Focal loss

Summary

Focal loss is a type of loss function that downweighs well-predicted labels.

Details

Focal loss ($FL$) is defined by $FL(p_i)=-\alpha (1-p_i{)}^{\gamma }\log (p_i)$ where $\gamma$ is a user-picked modulating factor that decreases the gradient of easy examples as it increases, and $\alpha$ is a weight. A value of zero reverts to cross-entropy.

Figures

From https://medium.com/swlh/focal-loss-what-why-and-how-df6735f26616

See also

• PLMs are biased toward germline antibodies