Why does Muon work?
I wanted to understand why the Muon optimizer works, so I went back through SGD, momentum, Adam and Shampoo and built a simulation for each idea.
I wanted to understand why the Muon optimizer works, so I went back through SGD, momentum, Adam and Shampoo and built a simulation for each idea.