Learning rate
| Created | |
|---|---|
| Tags | NN |
Learning rate is a hyper-parameter that controls how much we are adjusting the weights of our network with respect the loss gradient. [src]
A larger learning rate allows the model to learn faster, but we risk overshooting the lowest point.
A smaller learning rate may allow the model to learn a more optimal set of weights but takes longer time to train, and increases the risk of overfitting.