Visualizing the Loss Landscape of Neural Nets: Residual vs AttnResThe loss landscapes of 8, 12 and 16-layer nanochat models: Karpathy residuals next to attention residuals.November 19, 2026·3 min·499 words·Lucas Sun