Evidence for model selection of a regularization network/Gaussian process. More...
#include <shark/ObjectiveFunctions/NegativeGaussianProcessEvidence.h>
Public Types | |
typedef LabeledData< InputType, LabelType > | DatasetType |
typedef AbstractKernelFunction< InputType > | KernelType |
![]() | |
enum | Feature |
List of features that are supported by an implementation. More... | |
typedef RealVector | SearchPointType |
typedef double | ResultType |
typedef SearchPointType | FirstOrderDerivative |
typedef TypedFlags< Feature > | Features |
This statement declares the member m_features. See Core/Flags.h for details. More... | |
typedef TypedFeatureNotAvailableException< Feature > | FeatureNotAvailableException |
Public Member Functions | |
NegativeGaussianProcessEvidence (DatasetType const &dataset, KernelType *kernel, bool unconstrained=false) | |
std::string | name () const |
From INameable: return the class name. More... | |
std::size_t | numberOfVariables () const |
Accesses the number of variables. More... | |
double | eval (const RealVector ¶meters) const |
double | evalDerivative (const RealVector ¶meters, FirstOrderDerivative &derivative) const |
void | setThreshold (double d) |
set threshold value for truncating partial derivatives More... | |
void | setThresholds (RealVector &c) |
set threshold values for truncating partial derivatives More... | |
![]() | |
const Features & | features () const |
virtual void | updateFeatures () |
bool | hasValue () const |
returns whether this function can calculate it's function value More... | |
bool | hasFirstDerivative () const |
returns whether this function can calculate the first derivative More... | |
bool | hasSecondDerivative () const |
returns whether this function can calculate the second derivative More... | |
bool | canProposeStartingPoint () const |
returns whether this function can propose a starting point. More... | |
bool | isConstrained () const |
returns whether this function can return More... | |
bool | hasConstraintHandler () const |
returns whether this function can return More... | |
bool | canProvideClosestFeasible () const |
Returns whether this function can calculate thee closest feasible to an infeasible point. More... | |
bool | isThreadSafe () const |
Returns true, when the function can be usd in parallel threads. More... | |
AbstractObjectiveFunction () | |
Default ctor. More... | |
virtual | ~AbstractObjectiveFunction () |
Virtual destructor. More... | |
virtual void | init () |
virtual bool | hasScalableDimensionality () const |
virtual void | setNumberOfVariables (std::size_t numberOfVariables) |
Adjusts the number of variables if the function is scalable. More... | |
virtual std::size_t | numberOfObjectives () const |
virtual bool | hasScalableObjectives () const |
virtual void | setNumberOfObjectives (std::size_t numberOfObjectives) |
Adjusts the number of objectives if the function is scalable. More... | |
std::size_t | evaluationCounter () const |
Accesses the evaluation counter of the function. More... | |
AbstractConstraintHandler< SearchPointType > const & | getConstraintHandler () const |
Returns the constraint handler of the function if it has one. More... | |
virtual bool | isFeasible (const SearchPointType &input) const |
Tests whether a point in SearchSpace is feasible, e.g., whether the constraints are fulfilled. More... | |
virtual void | closestFeasible (SearchPointType &input) const |
If supported, the supplied point is repaired such that it satisfies all of the function's constraints. More... | |
virtual SearchPointType | proposeStartingPoint () const |
Proposes a starting point in the feasible search space of the function. More... | |
ResultType | operator() (const SearchPointType &input) const |
Evaluates the function. Useful together with STL-Algorithms like std::transform. More... | |
virtual ResultType | evalDerivative (const SearchPointType &input, SecondOrderDerivative &derivative) const |
Evaluates the objective function and calculates its gradient. More... | |
![]() | |
virtual | ~INameable () |
Additional Inherited Members | |
![]() | |
void | announceConstraintHandler (AbstractConstraintHandler< SearchPointType > const *handler) |
helper function which is called to announce the presence of an constraint handler. More... | |
![]() | |
Features | m_features |
std::size_t | m_evaluationCounter |
Evaluation counter, default value: 0. More... | |
AbstractConstraintHandler< SearchPointType > const * | m_constraintHandler |
Evidence for model selection of a regularization network/Gaussian process.
Let \(M\) denote the (kernel Gram) covariance matrix and \(t\) the corresponding label vector. For the evidence we have:
\[ E = 1/2 \cdot [ -\log(\det(M)) - t^T M^{-1} t - N \log(2 \pi)] \]
The evidence is also known as marginal (log)likelihood. For details, please see:
C.E. Rasmussen & C.K.I. Williams, Gaussian Processes for Machine Learning, section 5.4, MIT Press, 2006
C.M. Bishop, Pattern Recognition and Machine Learning, section 6.4.3, Springer, 2006
The regularization parameter can be encoded in different ways. The exponential encoding is the proper choice for unconstraint optimization. Be careful not to mix up different encodings between trainer and evidence.
Definition at line 68 of file NegativeGaussianProcessEvidence.h.
typedef LabeledData<InputType,LabelType> shark::NegativeGaussianProcessEvidence< InputType, OutputType, LabelType >::DatasetType |
Definition at line 71 of file NegativeGaussianProcessEvidence.h.
typedef AbstractKernelFunction<InputType> shark::NegativeGaussianProcessEvidence< InputType, OutputType, LabelType >::KernelType |
Definition at line 72 of file NegativeGaussianProcessEvidence.h.
|
inline |
dataset | training data for the Gaussian process |
kernel | pointer to external kernel function |
unconstrained | exponential encoding of regularization parameter for unconstraint optimization |
Definition at line 77 of file NegativeGaussianProcessEvidence.h.
References shark::AbstractObjectiveFunction< RealVector, double >::HAS_FIRST_DERIVATIVE, shark::AbstractKernelFunction< InputTypeT >::hasFirstParameterDerivative(), shark::AbstractObjectiveFunction< RealVector, double >::m_features, and shark::NegativeGaussianProcessEvidence< InputType, OutputType, LabelType >::setThreshold().
|
inlinevirtual |
Let \(M\) denote the (kernel Gram) covariance matrix and \(t\) the label vector. For the evidence we have:
\[ E= 1/2 \cdot [ -\log(\det(M)) - t^T M^{-1} t - N \log(2 \pi) ] \]
Reimplemented from shark::AbstractObjectiveFunction< RealVector, double >.
Definition at line 99 of file NegativeGaussianProcessEvidence.h.
References shark::calculateRegularizedKernelMatrix(), shark::blas::choleskyDecomposition(), shark::blas::column(), shark::Data< Type >::elements(), shark::blas::init(), shark::LabeledData< InputT, LabelT >::inputs(), shark::LabeledData< InputT, LabelT >::labels(), shark::AbstractObjectiveFunction< RealVector, double >::m_evaluationCounter, shark::blas::norm_sqr(), shark::LabeledData< InputT, LabelT >::numberOfElements(), shark::IParameterizable::numberOfParameters(), shark::IParameterizable::setParameterVector(), SHARK_ASSERT, and shark::blas::trace().
|
inlinevirtual |
Let \(M\) denote the regularized (kernel Gram) covariance matrix. For the evidence we have:
\[ E = 1/2 \cdot [ -\log(\det(M)) - t^T M^{-1} t - N \log(2 \pi) ] \]
For a kernel parameter \(p\) and \(C = \beta^{-1}\) we get the derivatives:
\[ dE/dC = 1/2 \cdot [ -tr(M^{-1}) + (M^{-1} t)^2 ] \]
\[ dE/dp = 1/2 \cdot [ -tr(M^{-1} dM/dp) + t^T (M^{-1} dM/dp M^{-1}) t ] \]
Reimplemented from shark::AbstractObjectiveFunction< RealVector, double >.
Definition at line 150 of file NegativeGaussianProcessEvidence.h.
References shark::calculateKernelMatrixParameterDerivative(), shark::calculateRegularizedKernelMatrix(), shark::blas::choleskyDecomposition(), shark::blas::column(), shark::Data< Type >::elements(), shark::blas::init(), shark::blas::inner_prod(), shark::LabeledData< InputT, LabelT >::inputs(), shark::LabeledData< InputT, LabelT >::labels(), shark::AbstractObjectiveFunction< RealVector, double >::m_evaluationCounter, shark::LabeledData< InputT, LabelT >::numberOfElements(), shark::IParameterizable::numberOfParameters(), shark::blas::outer_prod(), shark::blas::prod(), shark::IParameterizable::setParameterVector(), SHARK_ASSERT, and shark::blas::trace().
|
inlinevirtual |
From INameable: return the class name.
Reimplemented from shark::INameable.
Definition at line 90 of file NegativeGaussianProcessEvidence.h.
|
inlinevirtual |
Accesses the number of variables.
Implements shark::AbstractObjectiveFunction< RealVector, double >.
Definition at line 93 of file NegativeGaussianProcessEvidence.h.
References shark::IParameterizable::numberOfParameters().
|
inline |
set threshold value for truncating partial derivatives
Definition at line 232 of file NegativeGaussianProcessEvidence.h.
References shark::IParameterizable::numberOfParameters().
Referenced by shark::NegativeGaussianProcessEvidence< InputType, OutputType, LabelType >::NegativeGaussianProcessEvidence().
|
inline |
set threshold values for truncating partial derivatives
Definition at line 237 of file NegativeGaussianProcessEvidence.h.
References SHARK_ASSERT.