Karpathy residualsloss at the trained weights: 3.066
Loading the 3D view…
Attention residuals (AttnRes)loss at the trained weights: 3.065
Loading the 3D view…
3.111.1 and aboveloss, nats per token

Drag a surface to rotate it. The center of each surface is the trained model, and each model has its own two random directions. Unticking "Smoothed mesh" shows the measured 51 × 51 grid.