![]() |
Home | Main Page | Modules | Namespace List | Class Hierarchy | Alphabetical List | Data Structures | File List | Namespace Members | Data Fields | Globals | Related Pages |
#include <elxAdaptiveStochasticGradientDescent.h>
A gradient descent optimizer with an adaptive gain.
This class is a wrap around the AdaptiveStochasticGradientDescentOptimizer class. It takes care of setting parameters and printing progress information. For more information about the optimization method, please read the documentation of the AdaptiveStochasticGradientDescentOptimizer class.
This optimizer is very suitable to be used in combination with the Random image sampler, or with the RandomCoordinate image sampler, with the setting (NewSamplesEveryIteration "true"). Much effort has been spent on providing reasonable default values for all parameters, to simplify usage. In most registration problems, good results should be obtained without specifying any of the parameters described below (except the first of course, which defines the optimizer to use).
This optimization method is described in the following references:
[1] P. Cruz, "Almost sure convergence and asymptotical normality of a generalization of Kesten's stochastic approximation algorithm for multidimensional case." Technical Report, 2005. http://hdl.handle.net/2052/74
[2] S. Klein, J.P.W. Pluim, and M. Staring, M.A. Viergever, "Adaptive stochastic gradient descent optimization for image registration," International Journal of Computer Vision, vol. 81, no. 3, pp. 227-239, 2009. http://dx.doi.org/10.1007/s11263-008-0168-y
Acceleration in case of many transform parameters was proposed in the following paper:
[3] Y.Qiao, B.P.F. Lelieveldt, M.Staring "Fast automatic estimation of the optimization step size for nonrigid image registration," SPIE Medical Imaging: Image Processing,February, 2014. http://elastix.isi.uu.nl/marius/publications/2014_c_SPIEMI.php
The parameters used in this class are:
Optimizer: Select this optimizer as follows:
(Optimizer "AdaptiveStochasticGradientDescent")
MaximumNumberOfIterations: The maximum number of iterations in each resolution.
example: (MaximumNumberOfIterations 100 100 50)
Default/recommended value: 500. When you are in a hurry, you may go down to 250 for example. When you have plenty of time, and want to be absolutely sure of the best results, a setting of 2000 is reasonable. In general, 500 gives satisfactory results.
MaximumNumberOfSamplingAttempts: The maximum number of sampling attempts. Sometimes not enough corresponding samples can be drawn, upon which an exception is thrown. With this parameter it is possible to try to draw another set of samples.
example: (MaximumNumberOfSamplingAttempts 10 15 10)
Default value: 0, i.e. just fail immediately, for backward compatibility.
AutomaticParameterEstimation: When this parameter is set to "true", many other parameters are calculated automatically: SP_a, SP_alpha, SigmoidMax, SigmoidMin, and SigmoidScale. In the elastix.log file the actually chosen values for these parameters can be found.
example: (AutomaticParameterEstimation "true")
Default/recommended value: "true". The parameter can be specified for each resolution, or for all resolutions at once.
UseAdaptiveStepSizes: When this parameter is set to "true", the adaptive step size mechanism described in the documentation of itk::AdaptiveStochasticGradientDescentOptimizer is used. The parameter can be specified for each resolution, or for all resolutions at once.
example: (UseAdaptiveStepSizes "true")
Default/recommend value: "true", because it makes the registration more robust. In case of using a RandomCoordinate sampler, with (UseRandomSampleRegion "true"), the adaptive step size mechanism is turned off, no matter the user setting.
MaximumStepLength: Also called . This parameter can be considered as the maximum voxel displacement between two iterations. The larger this parameter, the more aggressive the optimization. The parameter can be specified for each resolution, or for all resolutions at once.
example: (MaximumStepLength 1.0)
Default: mean voxel spacing of fixed and moving image. This seems to work well in general. This parameter only has influence when AutomaticParameterEstimation is used.
SP_a: The gain at each iteration
is defined by
.
SP_a can be defined for each resolution.
example: (SP_a 3200.0 3200.0 1600.0)
The default value is 400.0. Tuning this variable for you specific problem is recommended. Alternatively set the AutomaticParameterEstimation to "true". In that case, you do not need to specify SP_a. SP_a has no influence when AutomaticParameterEstimation is used.
SP_A: The gain at each iteration
is defined by
.
SP_A can be defined for each resolution.
example: (SP_A 50.0 50.0 100.0)
The default/recommended value for this particular optimizer is 20.0.
SP_alpha: The gain at each iteration
is defined by
.
SP_alpha can be defined for each resolution.
example: (SP_alpha 0.602 0.602 0.602)
The default/recommended value for this particular optimizer is 1.0. Alternatively set the AutomaticParameterEstimation to "true". In that case, you do not need to specify SP_alpha. SP_alpha has no influence when AutomaticParameterEstimation is used.
SigmoidMax: The maximum of the sigmoid function ( ). Must be larger than 0. The parameter can be specified for each resolution, or for all resolutions at once.
example: (SigmoidMax 1.0)
Default/recommended value: 1.0. This parameter has no influence when AutomaticParameterEstimation is used. In that case, always a value 1.0 is used.
SigmoidMin: The minimum of the sigmoid function ( ). Must be smaller than 0. The parameter can be specified for each resolution, or for all resolutions at once.
example: (SigmoidMin -0.8)
Default value: -0.8. This parameter has no influence when AutomaticParameterEstimation is used. In that case, the value is automatically determined, depending on the images, metric etc.
SigmoidScale: The scale/width of the sigmoid function ( ). The parameter can be specified for each resolution, or for all resolutions at once.
example: (SigmoidScale 0.00001)
Default value: 1e-8. This parameter has no influence when AutomaticParameterEstimation is used. In that case, the value is automatically determined, depending on the images, metric etc.
SigmoidInitialTime: the initial time input for the sigmoid ( ). Must be larger than 0.0. The parameter can be specified for each resolution, or for all resolutions at once.
example: (SigmoidInitialTime 0.0 5.0 5.0)
Default value: 0.0. When increased, the optimization starts with smaller steps, leaving the possibility to increase the steps when necessary. If set to 0.0, the method starts with with the largest step allowed.
NumberOfGradientMeasurements: Number of gradients N to estimate the average square magnitudes of the exact gradient and the approximation error. The parameter can be specified for each resolution, or for all resolutions at once.
example: (NumberOfGradientMeasurements 10)
Default value: 0, which means that the value is automatically estimated. In principle, the more the better, but the slower. In practice N=10 is usually sufficient. But the automatic estimation achieved by N=0 also works good. The parameter has only influence when AutomaticParameterEstimation is used.
NumberOfJacobianMeasurements: The number of voxels M where the Jacobian is measured, which is used to estimate the covariance matrix. The parameter can be specified for each resolution, or for all resolutions at once.
example: (NumberOfJacobianMeasurements 5000 10000 20000)
Default value: M = max( 1000, nrofparams ), with nrofparams the number of transform parameters. This is a rather crude rule of thumb, which seems to work in practice. In principle, the more the better, but the slower. The parameter has only influence when AutomaticParameterEstimation is used.
NumberOfSamplesForExactGradient: The number of image samples used to compute the 'exact' gradient. The samples are chosen on a uniform grid. The parameter can be specified for each resolution, or for all resolutions at once.
example: (NumberOfSamplesForExactGradient 100000)
Default/recommended: 100000. This works in general. If the image is smaller, the number of samples is automatically reduced. In principle, the more the better, but the slower. The parameter has only influence when AutomaticParameterEstimation is used.
ASGDParameterEstimationMethod: The ASGD parameter estimation method used in this optimizer. The parameter can be specified for each resolution.
example: (ASGDParameterEstimationMethod "Original")
or (ASGDParameterEstimationMethod "DisplacementDistribution")
Default: Original.
MaximumDisplacementEstimationMethod: The suitable position selection method used only for displacement distribution estimation method. The parameter can be specified for each resolution.
example: (MaximumDisplacementEstimationMethod "2sigma")
or (MaximumDisplacementEstimationMethod "95percentile")
Default: 2sigma.
NoiseCompensation: Selects whether or not to use noise compensation. The parameter can be specified for each resolution, or for all resolutions at once.
example: (NoiseCompensation "true")
Default/recommended: true.
Definition at line 191 of file elxAdaptiveStochasticGradientDescent.h.
Data Structures | |
struct | SettingsType |
Public Member Functions | |
virtual void | AfterEachIteration (void) |
virtual void | AfterEachResolution (void) |
virtual void | AfterRegistration (void) |
virtual void | BeforeEachResolution (void) |
virtual void | BeforeRegistration (void) |
elxClassNameMacro ("AdaptiveStochasticGradientDescent") | |
virtual bool | GetAutomaticParameterEstimation () const |
virtual const char * | GetClassName () const |
virtual const SizeValueType & | GetMaximumNumberOfSamplingAttempts () |
virtual double | GetMaximumStepLength () const |
virtual void | MetricErrorResponse (itk::ExceptionObject &err) |
virtual void | ResumeOptimization (void) |
virtual void | SetAutomaticParameterEstimation (bool _arg) |
virtual void | SetMaximumNumberOfSamplingAttempts (SizeValueType _arg) |
virtual void | SetMaximumStepLength (double _arg) |
virtual void | StartOptimization (void) |
![]() | |
virtual double | GetSigmoidMax () const |
virtual double | GetSigmoidMin () const |
virtual double | GetSigmoidScale () const |
virtual bool | GetUseAdaptiveStepSizes () const |
virtual void | SetSigmoidMax (double _arg) |
virtual void | SetSigmoidMin (double _arg) |
virtual void | SetSigmoidScale (double _arg) |
virtual void | SetUseAdaptiveStepSizes (bool _arg) |
![]() | |
virtual void | AdvanceOneStep (void) |
virtual double | GetCurrentTime () const |
virtual double | GetInitialTime () const |
virtual double | GetParam_a () const |
virtual double | GetParam_A () const |
virtual double | GetParam_alpha () const |
virtual void | ResetCurrentTimeToInitialTime (void) |
virtual void | SetInitialTime (double _arg) |
virtual void | SetParam_a (double _arg) |
virtual void | SetParam_A (double _arg) |
virtual void | SetParam_alpha (double _arg) |
![]() | |
virtual unsigned int | GetCurrentIteration () const |
virtual const DerivativeType & | GetGradient () |
virtual const double & | GetLearningRate () |
virtual const unsigned long & | GetNumberOfIterations () |
virtual const StopConditionType & | GetStopCondition () |
virtual const double & | GetValue () |
virtual void | MetricErrorResponse (ExceptionObject &err) |
virtual void | SetLearningRate (double _arg) |
virtual void | SetNumberOfIterations (unsigned long _arg) |
void | SetNumberOfThreads (ThreadIdType numberOfThreads) |
virtual void | SetUseEigen (bool _arg) |
virtual void | SetUseMultiThread (bool _arg) |
virtual void | SetUseOpenMP (bool _arg) |
virtual void | StopOptimization (void) |
![]() | |
virtual const ParametersType & | GetCurrentPosition (void) const |
virtual bool | GetMaximize () const |
virtual const ScaledCostFunctionType * | GetScaledCostFunction () |
virtual const ParametersType & | GetScaledCurrentPosition () |
bool | GetUseScales (void) const |
virtual void | InitializeScales (void) |
virtual void | MaximizeOff () |
virtual void | MaximizeOn () |
virtual void | SetCostFunction (CostFunctionType *costFunction) |
virtual void | SetMaximize (bool _arg) |
virtual void | SetUseScales (bool arg) |
![]() | |
virtual void | AfterRegistrationBase (void) |
virtual void | BeforeEachResolutionBase () |
virtual ITKBaseType * | GetAsITKBaseType (void) |
virtual const ITKBaseType * | GetAsITKBaseType (void) const |
virtual void | SetCurrentPositionPublic (const ParametersType ¶m) |
virtual void | SetSinusScales (double amplitude, double frequency, unsigned long numberOfParameters) |
![]() | |
virtual ConfigurationType * | GetConfiguration (void) const |
virtual ElastixType * | GetElastix (void) const |
virtual RegistrationPointer | GetRegistration (void) const |
virtual void | SetConfiguration (ConfigurationType *_arg) |
virtual void | SetElastix (ElastixType *_arg) |
![]() | |
virtual void | AfterEachIterationBase (void) |
virtual void | AfterEachResolutionBase (void) |
virtual int | BeforeAll (void) |
virtual int | BeforeAllBase (void) |
virtual void | BeforeRegistrationBase (void) |
std::string | ConvertSecondsToDHMS (const double totalSeconds, const unsigned int precision) const |
virtual const char * | elxGetClassName (void) const |
virtual const char * | GetComponentLabel (void) const |
virtual void | SetComponentLabel (const char *label, unsigned int idx) |
Static Public Member Functions | |
static Pointer | New () |
![]() | |
static Pointer | New () |
![]() | |
static Pointer | New () |
![]() | |
static Pointer | New () |
![]() | |
static Pointer | New () |
Private Member Functions | |
AdaptiveStochasticGradientDescent (const Self &) | |
void | operator= (const Self &) |
|
protected |
Definition at line 336 of file elxAdaptiveStochasticGradientDescent.h.
|
protected |
Definition at line 335 of file elxAdaptiveStochasticGradientDescent.h.
|
protected |
Definition at line 303 of file elxAdaptiveStochasticGradientDescent.h.
|
protected |
Definition at line 297 of file elxAdaptiveStochasticGradientDescent.h.
typedef Superclass2::ConfigurationPointer elastix::AdaptiveStochasticGradientDescent< TElastix >::ConfigurationPointer |
Definition at line 226 of file elxAdaptiveStochasticGradientDescent.h.
typedef Superclass2::ConfigurationType elastix::AdaptiveStochasticGradientDescent< TElastix >::ConfigurationType |
Definition at line 225 of file elxAdaptiveStochasticGradientDescent.h.
typedef itk::SmartPointer< const Self > elastix::AdaptiveStochasticGradientDescent< TElastix >::ConstPointer |
Definition at line 202 of file elxAdaptiveStochasticGradientDescent.h.
|
protected |
Definition at line 331 of file elxAdaptiveStochasticGradientDescent.h.
typedef Superclass1::CostFunctionPointer elastix::AdaptiveStochasticGradientDescent< TElastix >::CostFunctionPointer |
Definition at line 219 of file elxAdaptiveStochasticGradientDescent.h.
typedef Superclass1::CostFunctionType elastix::AdaptiveStochasticGradientDescent< TElastix >::CostFunctionType |
Typedef's inherited from Superclass1.
Definition at line 218 of file elxAdaptiveStochasticGradientDescent.h.
typedef Superclass2::ElastixPointer elastix::AdaptiveStochasticGradientDescent< TElastix >::ElastixPointer |
Definition at line 224 of file elxAdaptiveStochasticGradientDescent.h.
typedef Superclass2::ElastixType elastix::AdaptiveStochasticGradientDescent< TElastix >::ElastixType |
Typedef's inherited from Superclass2.
Definition at line 223 of file elxAdaptiveStochasticGradientDescent.h.
|
protected |
Definition at line 291 of file elxAdaptiveStochasticGradientDescent.h.
|
protected |
Definition at line 292 of file elxAdaptiveStochasticGradientDescent.h.
|
protected |
Definition at line 290 of file elxAdaptiveStochasticGradientDescent.h.
|
protected |
Protected typedefs
Definition at line 282 of file elxAdaptiveStochasticGradientDescent.h.
|
protected |
Definition at line 316 of file elxAdaptiveStochasticGradientDescent.h.
|
protected |
Definition at line 315 of file elxAdaptiveStochasticGradientDescent.h.
|
protected |
Definition at line 314 of file elxAdaptiveStochasticGradientDescent.h.
|
protected |
Definition at line 312 of file elxAdaptiveStochasticGradientDescent.h.
|
protected |
Definition at line 310 of file elxAdaptiveStochasticGradientDescent.h.
|
protected |
Definition at line 308 of file elxAdaptiveStochasticGradientDescent.h.
|
protected |
Definition at line 319 of file elxAdaptiveStochasticGradientDescent.h.
|
protected |
Definition at line 318 of file elxAdaptiveStochasticGradientDescent.h.
|
protected |
Definition at line 307 of file elxAdaptiveStochasticGradientDescent.h.
|
protected |
Samplers:
Definition at line 306 of file elxAdaptiveStochasticGradientDescent.h.
typedef Superclass2::ITKBaseType elastix::AdaptiveStochasticGradientDescent< TElastix >::ITKBaseType |
Definition at line 229 of file elxAdaptiveStochasticGradientDescent.h.
|
protected |
Definition at line 293 of file elxAdaptiveStochasticGradientDescent.h.
|
protected |
Definition at line 295 of file elxAdaptiveStochasticGradientDescent.h.
|
protected |
Definition at line 298 of file elxAdaptiveStochasticGradientDescent.h.
|
protected |
Definition at line 288 of file elxAdaptiveStochasticGradientDescent.h.
|
protected |
Definition at line 338 of file elxAdaptiveStochasticGradientDescent.h.
typedef Superclass1::ParametersType elastix::AdaptiveStochasticGradientDescent< TElastix >::ParametersType |
Typedef for the ParametersType.
Definition at line 233 of file elxAdaptiveStochasticGradientDescent.h.
typedef itk::SmartPointer< Self > elastix::AdaptiveStochasticGradientDescent< TElastix >::Pointer |
Definition at line 201 of file elxAdaptiveStochasticGradientDescent.h.
|
protected |
Definition at line 325 of file elxAdaptiveStochasticGradientDescent.h.
|
protected |
Definition at line 324 of file elxAdaptiveStochasticGradientDescent.h.
|
protected |
Definition at line 323 of file elxAdaptiveStochasticGradientDescent.h.
|
protected |
Other protected typedefs
Definition at line 322 of file elxAdaptiveStochasticGradientDescent.h.
typedef Superclass2::RegistrationPointer elastix::AdaptiveStochasticGradientDescent< TElastix >::RegistrationPointer |
Definition at line 228 of file elxAdaptiveStochasticGradientDescent.h.
typedef Superclass2::RegistrationType elastix::AdaptiveStochasticGradientDescent< TElastix >::RegistrationType |
Definition at line 227 of file elxAdaptiveStochasticGradientDescent.h.
typedef AdaptiveStochasticGradientDescent elastix::AdaptiveStochasticGradientDescent< TElastix >::Self |
Standard ITK.
Definition at line 198 of file elxAdaptiveStochasticGradientDescent.h.
|
protected |
Definition at line 300 of file elxAdaptiveStochasticGradientDescent.h.
typedef itk::SizeValueType elastix::AdaptiveStochasticGradientDescent< TElastix >::SizeValueType |
Definition at line 230 of file elxAdaptiveStochasticGradientDescent.h.
typedef Superclass1::StopConditionType elastix::AdaptiveStochasticGradientDescent< TElastix >::StopConditionType |
Definition at line 220 of file elxAdaptiveStochasticGradientDescent.h.
typedef AdaptiveStochasticGradientDescentOptimizer elastix::AdaptiveStochasticGradientDescent< TElastix >::Superclass1 |
Definition at line 199 of file elxAdaptiveStochasticGradientDescent.h.
typedef OptimizerBase< TElastix > elastix::AdaptiveStochasticGradientDescent< TElastix >::Superclass2 |
Definition at line 200 of file elxAdaptiveStochasticGradientDescent.h.
|
protected |
Typedefs for support of sparse Jacobians and AdvancedTransforms.
Definition at line 328 of file elxAdaptiveStochasticGradientDescent.h.
|
protected |
Definition at line 294 of file elxAdaptiveStochasticGradientDescent.h.
|
protected |
|
inlineprotectedvirtual |
Definition at line 341 of file elxAdaptiveStochasticGradientDescent.h.
|
private |
|
protectedvirtual |
Helper function that adds a random perturbation delta to the input parameters, with delta ~ sigma * N(0,I). Used by SampleGradients.
|
virtual |
Reimplemented from elastix::BaseComponent.
|
virtual |
Reimplemented from elastix::BaseComponent.
|
virtual |
Reimplemented from elastix::BaseComponent.
|
protectedvirtual |
Select different method to estimate some reasonable values for the parameters SP_a, SP_alpha (=1), SigmoidMin, SigmoidMax (=1), and SigmoidScale.
|
protectedvirtual |
Original estimation method to get the reasonable values for the parameters SP_a, SP_alpha (=1), SigmoidMin, SigmoidMax (=1), and SigmoidScale.
|
protectedvirtual |
Estimates some reasonable values for the parameters using displacement distribution SP_a, SP_alpha (=1)
|
virtual |
Reimplemented from elastix::BaseComponent.
|
virtual |
Methods invoked by elastix, in which parameters can be set and progress information can be printed.
Reimplemented from elastix::BaseComponent.
|
protectedvirtual |
Check if the transform is an advanced transform. Called by Initialize.
elastix::AdaptiveStochasticGradientDescent< TElastix >::elxClassNameMacro | ( | "AdaptiveStochasticGradientDescent< TElastix >" | ) |
Name of this class. Use this name in the parameter file to select this specific optimizer. example: (Optimizer "AdaptiveStochasticGradientDescent")
|
virtual |
|
virtual |
Run-time type information (and related methods).
Reimplemented from itk::AdaptiveStochasticGradientDescentOptimizer.
|
virtual |
Get the MaximumNumberOfSamplingAttempts.
|
virtual |
|
protectedvirtual |
Helper function, which calls GetScaledValueAndDerivative and does some exception handling. Used by SampleGradients.
|
protected |
|
protected |
|
virtual |
Stop optimization and pass on exception.
|
static |
Method for creation through the object factory.
|
private |
|
protectedvirtual |
Print the contents of the settings vector to elxout.
|
virtual |
If automatic gain estimation is desired, then estimate SP_a, SP_alpha SigmoidScale, SigmoidMax, SigmoidMin. After that call Superclass' implementation.
Reimplemented from itk::GradientDescentOptimizer2.
|
protectedvirtual |
Measure some derivatives, exact and approximated. Returns the squared magnitude of the gradient and approximation error. Needed for the automatic parameter estimation. Gradients are measured at position mu_n, which are generated according to: mu_n - mu_0 ~ N(0, perturbationSigma^2 I ); gg = g^T g, etc.
|
virtual |
Set/Get whether automatic parameter estimation is desired. If true, make sure to set the maximum step length.
The following parameters are automatically determined: SP_a, SP_alpha (=1), SigmoidMin, SigmoidMax (=1), SigmoidScale. A usually suitable value for SP_A is 20, which is the default setting, if not specified by the user.
|
virtual |
Set the MaximumNumberOfSamplingAttempts.
|
virtual |
Set/Get maximum step length.
|
virtual |
Check if any scales are set, and set the UseScales flag on or off; after that call the superclass' implementation.
Reimplemented from itk::StandardGradientDescentOptimizer.
|
protected |
The transform stored as AdvancedTransform
Definition at line 352 of file elxAdaptiveStochasticGradientDescent.h.
|
private |
Definition at line 408 of file elxAdaptiveStochasticGradientDescent.h.
|
private |
Definition at line 415 of file elxAdaptiveStochasticGradientDescent.h.
|
private |
Definition at line 413 of file elxAdaptiveStochasticGradientDescent.h.
|
private |
Private variables for band size estimation of covariance matrix.
Definition at line 418 of file elxAdaptiveStochasticGradientDescent.h.
|
private |
Private variables for the sampling attempts.
Definition at line 412 of file elxAdaptiveStochasticGradientDescent.h.
|
private |
Definition at line 409 of file elxAdaptiveStochasticGradientDescent.h.
|
private |
Definition at line 419 of file elxAdaptiveStochasticGradientDescent.h.
|
protected |
Some options for automatic parameter estimation.
Definition at line 347 of file elxAdaptiveStochasticGradientDescent.h.
|
protected |
Definition at line 348 of file elxAdaptiveStochasticGradientDescent.h.
|
protected |
Definition at line 349 of file elxAdaptiveStochasticGradientDescent.h.
|
private |
Definition at line 423 of file elxAdaptiveStochasticGradientDescent.h.
|
private |
Definition at line 414 of file elxAdaptiveStochasticGradientDescent.h.
|
protected |
RandomGenerator for AddRandomPerturbation.
Definition at line 355 of file elxAdaptiveStochasticGradientDescent.h.
|
protected |
Variable to store the automatically determined settings for each resolution.
Definition at line 344 of file elxAdaptiveStochasticGradientDescent.h.
|
protected |
Definition at line 357 of file elxAdaptiveStochasticGradientDescent.h.
|
private |
The flag of using noise compensation.
Definition at line 422 of file elxAdaptiveStochasticGradientDescent.h.
Generated on 07-03-2016 for elastix by ![]() |
![]() |