GammaNet - Stable Feature-Space Decay in Linear RNNs
GammaNet - Allowing Kimi Delta Attention to decay along meaningful directions instead of meaningless coordinates.
GammaNet - Allowing Kimi Delta Attention to decay along meaningful directions instead of meaningless coordinates.
Bra-ket notation as a unifying language for linear maps, MLPs, and self-attention — making the key-value structure of deep learning explicit.