Available in: GBM
Use this option to reduce the learning rate by this factor after every tree. When used, then for N trees, you would start with
learn_rate and end with
learn_rate * learn_rate_annealing^N.
The following provides some reference factors. (Refer to Taylor series for more information.):
0.99^100 = 0.366
0.99^1000 = 4.3e-5
0.999^1000 = 0.368
0.999^10000 = 4.5e-5
With this option, then instead of
learn_rate=0.01, you can try (for example)
learn_rate=0.05 along with
learn_rate_annealing=0.99. The result should converge much faster with almost the same accuracy. Note, however, that this can also result in overfitting, so use caution when specifying this option.
The value range for this option is between 0 and 1. This option defaults to 1.0, which disables the learning rate annealing.