learning-to-learn

Why we included this project

DeepMind published the 'learning to learn' paper in 2016, and this is the TensorFlow code that backed it. The idea is to stop picking an optimizer by hand and instead train a MetaOptimizer to produce one: train.py unrolls gradient descent and learns an update rule suited to a particular class of problems, then evaluate.py measures how the learned optimizer holds up against a baseline like Adam. The bundled problems start with a single-variable quadratic and scale up to MNIST and CIFAR classification, and because a new problem is just a function you hand to the MetaOptimizer, adding your own task is straightforward. It is the most direct place to start for anyone who wants to see how gradient-descent-by-gradient-descent actually works in code, and it stays readable because it is built on Sonnet rather than a pile of custom plumbing.