linear-attention

GammaNet - Stable Feature-Space Decay in Linear RNNs

GammaNet - Allowing Kimi Delta Attention to decay along meaningful directions instead of meaningless coordinates.