lion-pytorch

Why we included this project

Phil Wang's lion-pytorch is a clean, lightweight port of the Lion optimizer that Google Brain discovered through evolutionary search. You can drop it into an existing PyTorch training loop with a single import and test whether its sign-momentum update actually beats your current AdamW baseline. The README carries the practical tuning guidance from the paper, most importantly a learning rate that should run 3 to 10 times lower than AdamW's, paired with a correspondingly larger decoupled weight decay. It also collects honest field reports from researchers who tried it, including the finding that holding the learning rate constant made Lion underperform Adam, while dividing it by roughly ten produced better early results. That mix of a faithful reference implementation and real-world caveats helps practitioners decide whether the optimizer is worth adopting in their own projects.