Exploding gradient
| Created | |
|---|---|
| Tags | NN |
The large error gradients accumulate and result in very large updates to neural network model weights during training. This may make the model unstable and unable to learn from the data.
A common solution is to change the error derivative before back propagating it
- scaling the vector norm
- clipping gradient values to a range