caffe solver

*base_lr 

*display : if 3 then interval iter = 3 * average_loss(default 1)

Display the loss averaged over the last average_loss iterations

* test_iter

# of validation set / validation batch size , The number of iterations between two testing phases.

* test_interval

how often you validate..... usually for large nets you set test_interval in the order of 5K

*max_iter

# of epochs * # of train data / # of train batch size

*iter_size

this allows to average gradients over several training mini batches

*stepsize

learning rate decay step

*lr_policy

fixed: always return base_lr

step : retrun base_lr * gamma^(floor(iter/stepsize)

exp : return base_lr * gamma^iter

inv : return base_lr * ( 1+gamma*iter)^-power

multistep : similar to step but it allows non uniform steps defined by stepvalue

poly : the effective learning rate follows a polynomial decay, to be zero by the max_iter.

return base_lr * ( 1 - iter/max_iter) ^ power

sigmoid : the effective learning rate follows a sigmoid decay

return base_lr * ( 1/(1 + exp(-gamma*(iter-stepsize))))

*solver_type

SGD, NESTEROV, ADAGRAD, RMSPROP, ADADELTA, ADAM

http://caffe.berkeleyvision.org/tutorial/solver.html