Learning rate finder

Contains the LearningRateFinder class to aid with choosing the largest possible learning rate.

class vanguard.optimise.finder.LearningRateFinder(controller)

Estimates the best learning rate for a controller/data combination.

Try an increasing geometric sequence of learning rates for a small number of iterations to find the best learning rate (i.e. the largest learning rate giving stable training).

Parameters:

controller (GPController)

__init__(controller)

Initialise self.

Parameters:

controller (GPController) – An instantiated vanguard GP controller whose learning rate shall be optimised.

property best_learning_rate: float

Return the current best (lowest-loss) learning rate.

find(start_lr=1e-05, end_lr=10, num_divisions=100, max_iterations=20)

Try the range of learning rates and record the loss obtained.

Parameters:

• start_lr (float) – The smallest learning rate to try.

• end_lr (float) – The largest learning rate to try.

• num_divisions (int) – The number of learning rates to try.

• max_iterations (int) – The top number of iterations of gradient descent to run for each learning rate.

Return type:

None

plot(**kwargs)

Plot the obtained loss-vs-lr curve.

Return type:

None