An overview of Neural Optimizer Search using an iterative process to discover new optimizers.