The most important hyperparameter in a neural network. "When training a neural network, if I’m allowed to tune only one hyperparameter, the one that I choose to tune without any hesitation is the learning rate."