shark::AbstractModel< InputTypeT, OutputTypeT > Class Template Referenceabstract

Base class for all Models. More...

#include <shark/Models/AbstractModel.h>

+ Inheritance diagram for shark::AbstractModel< InputTypeT, OutputTypeT >:

Public Types

enum  Feature {
  HAS_FIRST_PARAMETER_DERIVATIVE = 1, HAS_SECOND_PARAMETER_DERIVATIVE = 2, HAS_FIRST_INPUT_DERIVATIVE = 4, HAS_SECOND_INPUT_DERIVATIVE = 8,
  IS_SEQUENTIAL = 16
}
 
typedef InputTypeT InputType
 Defines the input type of the model. More...
 
typedef OutputTypeT OutputType
 Defines the output type of the model. More...
 
typedef Batch< InputType >::type BatchInputType
 defines the batch type of the input type. More...
 
typedef Batch< OutputType >::type BatchOutputType
 defines the batch type of the output type More...
 
typedef TypedFlags< FeatureFeatures
 
typedef TypedFeatureNotAvailableException< FeatureFeatureNotAvailableException
 

Public Member Functions

 AbstractModel ()
 
virtual ~AbstractModel ()
 
const Featuresfeatures () const
 
virtual void updateFeatures ()
 
bool hasFirstParameterDerivative () const
 Returns true when the first parameter derivative is implemented. More...
 
bool hasSecondParameterDerivative () const
 Returns true when the second parameter derivative is implemented. More...
 
bool hasFirstInputDerivative () const
 Returns true when the first input derivative is implemented. More...
 
bool hasSecondInputDerivative () const
 Returns true when the second parameter derivative is implemented. More...
 
bool isSequential () const
 
virtual boost::shared_ptr< StatecreateState () const
 Creates an internal state of the model. More...
 
virtual void read (InArchive &archive)
 From ISerializable, reads a model from an archive. More...
 
virtual void write (OutArchive &archive) const
 writes a model to an archive More...
 
virtual void eval (BatchInputType const &patterns, BatchOutputType &outputs) const
 Standard interface for evaluating the response of the model to a batch of patterns. More...
 
virtual void eval (BatchInputType const &patterns, BatchOutputType &outputs, State &state) const =0
 Standard interface for evaluating the response of the model to a batch of patterns. More...
 
virtual void eval (InputType const &pattern, OutputType &output) const
 Standard interface for evaluating the response of the model to a single pattern. More...
 
Data< OutputTypeoperator() (Data< InputType > const &patterns) const
 Model evaluation as an operator for a whole dataset. This is a convenience function. More...
 
OutputType operator() (InputType const &pattern) const
 Model evaluation as an operator for a single pattern. This is a convenience function. More...
 
BatchOutputType operator() (BatchInputType const &patterns) const
 Model evaluation as an operator for a single pattern. This is a convenience function. More...
 
virtual void weightedParameterDerivative (BatchInputType const &pattern, BatchOutputType const &coefficients, State const &state, RealVector &derivative) const
 calculates the weighted sum of derivatives w.r.t the parameters. More...
 
virtual void weightedParameterDerivative (BatchInputType const &pattern, BatchOutputType const &coefficients, Batch< RealMatrix >::type const &errorHessian, State const &state, RealVector &derivative, RealMatrix &hessian) const
 calculates the weighted sum of derivatives w.r.t the parameters More...
 
virtual void weightedInputDerivative (BatchInputType const &pattern, BatchOutputType const &coefficients, State const &state, BatchInputType &derivative) const
 calculates the weighted sum of derivatives w.r.t the inputs More...
 
virtual void weightedInputDerivative (BatchInputType const &pattern, BatchOutputType const &coefficients, typename Batch< RealMatrix >::type const &errorHessian, State const &state, RealMatrix &derivative, Batch< RealMatrix >::type &hessian) const
 calculates the weighted sum of derivatives w.r.t the inputs More...
 
virtual void weightedDerivatives (BatchInputType const &patterns, BatchOutputType const &coefficients, State const &state, RealVector &parameterDerivative, BatchInputType &inputDerivative) const
 calculates weighted input and parameter derivative at the same time More...
 
- Public Member Functions inherited from shark::IParameterizable
virtual ~IParameterizable ()
 
virtual RealVector parameterVector () const
 Return the parameter vector. More...
 
virtual void setParameterVector (RealVector const &newParameters)
 Set the parameter vector. More...
 
virtual std::size_t numberOfParameters () const
 Return the number of parameters. More...
 
- Public Member Functions inherited from shark::INameable
virtual ~INameable ()
 
virtual std::string name () const
 returns the name of the object More...
 
- Public Member Functions inherited from shark::ISerializable
virtual ~ISerializable ()
 Virtual d'tor. More...
 
void load (InArchive &archive, unsigned int version)
 Versioned loading of components, calls read(...). More...
 
void save (OutArchive &archive, unsigned int version) const
 Versioned storing of components, calls write(...). More...
 
 BOOST_SERIALIZATION_SPLIT_MEMBER ()
 

Protected Attributes

Features m_features
 

Detailed Description

template<class InputTypeT, class OutputTypeT>
class shark::AbstractModel< InputTypeT, OutputTypeT >

Base class for all Models.

A model is one of the three fundaments of supervised learning: model, error measure and an optimization algorithm. It is a concept of a function which performs a mapping \( x \rightarrow f_w(x)\). In contrast to an error function it has two sets of parameters: The first is the current point to map \(x\), the others are the internal model parameters \(w\) which define the mapping. Often a model is used to find an optimal mapping for a problem, for example a function which best fits the points of a given dataset. Therefore, AbstractModel does not only offer the mapping itself, but also a set of special derivatives with respect to \( x \) and \( w \). Most of the time, only the derivative with respect to \( w \) is needed, but in some special problems, like finding optimal stimuli or stacking models, also the input derivative is needed.
Models are optimized for batch processing. This means, that instead of only one data point at a time, it can
evaluate a big set of inputs at the same time, using optimized routines for this task.
The derivatives are weighted, which means that the derivatives of every single output are added together weighted by coefficients (see weightedParameterDerivative). This is an optimization for the chain rule which is very efficient to calculate most of the time.
It is allowed to store intermediate values during eval and use them to speed up calculation of derivatives. Therefore it must be guaranteed that eval() is called before calculating derivatives. This is no restriction, since typical error measures need the mapping itself and not only the derivative.
Models have names and can be serialised

Definition at line 78 of file AbstractModel.h.

Member Typedef Documentation

§ BatchInputType

template<class InputTypeT, class OutputTypeT>
typedef Batch<InputType>::type shark::AbstractModel< InputTypeT, OutputTypeT >::BatchInputType

defines the batch type of the input type.

This could for example be std::vector<InputType> but for example for RealVector it could be RealMatrix

Definition at line 90 of file AbstractModel.h.

§ BatchOutputType

template<class InputTypeT, class OutputTypeT>
typedef Batch<OutputType>::type shark::AbstractModel< InputTypeT, OutputTypeT >::BatchOutputType

defines the batch type of the output type

Definition at line 92 of file AbstractModel.h.

§ FeatureNotAvailableException

template<class InputTypeT, class OutputTypeT>
typedef TypedFeatureNotAvailableException<Feature> shark::AbstractModel< InputTypeT, OutputTypeT >::FeatureNotAvailableException

Definition at line 106 of file AbstractModel.h.

§ Features

template<class InputTypeT, class OutputTypeT>
typedef TypedFlags<Feature> shark::AbstractModel< InputTypeT, OutputTypeT >::Features

Definition at line 106 of file AbstractModel.h.

§ InputType

template<class InputTypeT, class OutputTypeT>
typedef InputTypeT shark::AbstractModel< InputTypeT, OutputTypeT >::InputType

Defines the input type of the model.

Definition at line 82 of file AbstractModel.h.

§ OutputType

template<class InputTypeT, class OutputTypeT>
typedef OutputTypeT shark::AbstractModel< InputTypeT, OutputTypeT >::OutputType

Defines the output type of the model.

Definition at line 84 of file AbstractModel.h.

Member Enumeration Documentation

§ Feature

template<class InputTypeT, class OutputTypeT>
enum shark::AbstractModel::Feature
Enumerator
HAS_FIRST_PARAMETER_DERIVATIVE 
HAS_SECOND_PARAMETER_DERIVATIVE 
HAS_FIRST_INPUT_DERIVATIVE 
HAS_SECOND_INPUT_DERIVATIVE 
IS_SEQUENTIAL 

Definition at line 99 of file AbstractModel.h.

Constructor & Destructor Documentation

§ AbstractModel()

template<class InputTypeT, class OutputTypeT>
shark::AbstractModel< InputTypeT, OutputTypeT >::AbstractModel ( )
inline

Definition at line 95 of file AbstractModel.h.

§ ~AbstractModel()

template<class InputTypeT, class OutputTypeT>
virtual shark::AbstractModel< InputTypeT, OutputTypeT >::~AbstractModel ( )
inlinevirtual

Definition at line 97 of file AbstractModel.h.

Member Function Documentation

§ createState()

template<class InputTypeT, class OutputTypeT>
virtual boost::shared_ptr<State> shark::AbstractModel< InputTypeT, OutputTypeT >::createState ( ) const
inlinevirtual

Creates an internal state of the model.

The state is needed when the derivatives are to be calculated. Eval can store a state which is then reused to speed up the calculations of the derivatives. This also allows eval to be evaluated in parallel!

Reimplemented in shark::ConcatenatedModel< InputType, OutputType >, shark::FFNet< HiddenNeuron, OutputNeuron >, shark::ConvolutionalRBM< VisibleLayerT, HiddenLayerT, RngT >, shark::KernelExpansion< InputType >, shark::KernelExpansion< RealVector >, shark::RBM< VisibleLayerT, HiddenLayerT, RngT >, shark::LinearModel< InputType >, shark::LinearModel< VectorType >, shark::TiedAutoencoder< HiddenNeuron, OutputNeuron >, shark::Autoencoder< HiddenNeuron, OutputNeuron >, shark::OneVersusOneClassifier< InputType >, shark::ConvexCombination, shark::NBClassifier< InputType, OutputType >, shark::SoftNearestNeighborClassifier< InputType >, shark::ThresholdVectorConverter, shark::CMACMap, shark::NearestNeighborClassifier< InputType >, shark::CARTClassifier< LabelType >, shark::CARTClassifier< RealVector >, shark::NearestNeighborRegression< InputType >, shark::GaussianNoiseModel, shark::Normalizer< DataType >, shark::RBFLayer, shark::RNNet, shark::Softmax, shark::SigmoidModel, shark::ImpulseNoiseModel, shark::ThresholdConverter, shark::LinearNorm, and shark::MissingFeaturesKernelExpansion< InputType >.

Definition at line 134 of file AbstractModel.h.

Referenced by shark::AbstractModel< InputT, unsigned int >::eval(), and shark::NegativeLogLikelihood::evalDerivative().

§ eval() [1/3]

template<class InputTypeT, class OutputTypeT>
virtual void shark::AbstractModel< InputTypeT, OutputTypeT >::eval ( BatchInputType const &  patterns,
BatchOutputType outputs 
) const
inlinevirtual

§ eval() [2/3]

template<class InputTypeT, class OutputTypeT>
virtual void shark::AbstractModel< InputTypeT, OutputTypeT >::eval ( BatchInputType const &  patterns,
BatchOutputType outputs,
State state 
) const
pure virtual

§ eval() [3/3]

template<class InputTypeT, class OutputTypeT>
virtual void shark::AbstractModel< InputTypeT, OutputTypeT >::eval ( InputType const &  pattern,
OutputType output 
) const
inlinevirtual

Standard interface for evaluating the response of the model to a single pattern.

Parameters
patternthe input of the model
outputthe prediction or response of the model to the pattern

Reimplemented in shark::ArgMaxConverter< Model >, shark::ArgMaxConverter< LinearModel< VectorType > >, shark::ArgMaxConverter< KernelExpansion< InputType > >, shark::SoftClusteringModel< InputT >, and shark::HardClusteringModel< InputT >.

Definition at line 183 of file AbstractModel.h.

§ features()

template<class InputTypeT, class OutputTypeT>
const Features& shark::AbstractModel< InputTypeT, OutputTypeT >::features ( ) const
inline

Definition at line 106 of file AbstractModel.h.

§ hasFirstInputDerivative()

template<class InputTypeT, class OutputTypeT>
bool shark::AbstractModel< InputTypeT, OutputTypeT >::hasFirstInputDerivative ( ) const
inline

Returns true when the first input derivative is implemented.

Definition at line 117 of file AbstractModel.h.

Referenced by shark::AbstractModel< InputT, unsigned int >::createState().

§ hasFirstParameterDerivative()

template<class InputTypeT, class OutputTypeT>
bool shark::AbstractModel< InputTypeT, OutputTypeT >::hasFirstParameterDerivative ( ) const
inline

Returns true when the first parameter derivative is implemented.

Definition at line 109 of file AbstractModel.h.

Referenced by shark::AbstractModel< InputT, unsigned int >::createState(), and shark::NegativeLogLikelihood::NegativeLogLikelihood().

§ hasSecondInputDerivative()

template<class InputTypeT, class OutputTypeT>
bool shark::AbstractModel< InputTypeT, OutputTypeT >::hasSecondInputDerivative ( ) const
inline

Returns true when the second parameter derivative is implemented.

Definition at line 121 of file AbstractModel.h.

Referenced by shark::AbstractModel< InputT, unsigned int >::createState().

§ hasSecondParameterDerivative()

template<class InputTypeT, class OutputTypeT>
bool shark::AbstractModel< InputTypeT, OutputTypeT >::hasSecondParameterDerivative ( ) const
inline

Returns true when the second parameter derivative is implemented.

Definition at line 113 of file AbstractModel.h.

Referenced by shark::AbstractModel< InputT, unsigned int >::createState().

§ isSequential()

template<class InputTypeT, class OutputTypeT>
bool shark::AbstractModel< InputTypeT, OutputTypeT >::isSequential ( ) const
inline

Definition at line 124 of file AbstractModel.h.

§ operator()() [1/3]

template<class InputTypeT, class OutputTypeT>
Data<OutputType> shark::AbstractModel< InputTypeT, OutputTypeT >::operator() ( Data< InputType > const &  patterns) const
inline

Model evaluation as an operator for a whole dataset. This is a convenience function.

Parameters
patternsthe input of the model
Returns
the responses of the model

Definition at line 195 of file AbstractModel.h.

§ operator()() [2/3]

template<class InputTypeT, class OutputTypeT>
OutputType shark::AbstractModel< InputTypeT, OutputTypeT >::operator() ( InputType const &  pattern) const
inline

Model evaluation as an operator for a single pattern. This is a convenience function.

Parameters
patternthe input of the model
Returns
the response of the model

Definition at line 208 of file AbstractModel.h.

§ operator()() [3/3]

template<class InputTypeT, class OutputTypeT>
BatchOutputType shark::AbstractModel< InputTypeT, OutputTypeT >::operator() ( BatchInputType const &  patterns) const
inline

Model evaluation as an operator for a single pattern. This is a convenience function.

Parameters
patternsthe input of the model
Returns
the response of the model

Definition at line 218 of file AbstractModel.h.

§ read()

template<class InputTypeT, class OutputTypeT>
virtual void shark::AbstractModel< InputTypeT, OutputTypeT >::read ( InArchive archive)
inlinevirtual

§ updateFeatures()

template<class InputTypeT, class OutputTypeT>
virtual void shark::AbstractModel< InputTypeT, OutputTypeT >::updateFeatures ( )
inlinevirtual

Definition at line 106 of file AbstractModel.h.

§ weightedDerivatives()

template<class InputTypeT, class OutputTypeT>
virtual void shark::AbstractModel< InputTypeT, OutputTypeT >::weightedDerivatives ( BatchInputType const &  patterns,
BatchOutputType const &  coefficients,
State const &  state,
RealVector &  parameterDerivative,
BatchInputType inputDerivative 
) const
inlinevirtual

calculates weighted input and parameter derivative at the same time

Sometimes, both derivatives are needed at the same time. But sometimes, when calculating the weighted parameter derivative, the input derivative can be calculated for free. This is for example true for the feed-forward neural networks. However, there exists the obvious default implementation to just calculate the derivatives one after another.

Parameters
patternsthe patterns to evaluate
coefficientsthe coefficients which are used to calculate the weighted sum
stateintermediate results stored by eval to sped up calculations of the derivatives
parameterDerivativethe calculated parameter derivative as sum over all derivates of all patterns
inputDerivativethe calculated derivative for every pattern

Reimplemented in shark::ConcatenatedModel< InputType, OutputType >, shark::FFNet< HiddenNeuron, OutputNeuron >, shark::Autoencoder< HiddenNeuron, OutputNeuron >, and shark::TiedAutoencoder< HiddenNeuron, OutputNeuron >.

Definition at line 302 of file AbstractModel.h.

§ weightedInputDerivative() [1/2]

template<class InputTypeT, class OutputTypeT>
virtual void shark::AbstractModel< InputTypeT, OutputTypeT >::weightedInputDerivative ( BatchInputType const &  pattern,
BatchOutputType const &  coefficients,
State const &  state,
BatchInputType derivative 
) const
inlinevirtual

calculates the weighted sum of derivatives w.r.t the inputs

Parameters
patternthe patterns to evaluate
coefficientsthe coefficients which are used to calculate the weighted sum for every pattern
stateintermediate results stored by eval to sped up calculations of the derivatives
derivativethe calculated derivative for every pattern

Reimplemented in shark::LinearModel< InputType >, shark::LinearModel< VectorType >, shark::ConvexCombination, and shark::SigmoidModel.

Definition at line 264 of file AbstractModel.h.

Referenced by shark::AbstractModel< InputT, unsigned int >::weightedDerivatives().

§ weightedInputDerivative() [2/2]

template<class InputTypeT, class OutputTypeT>
virtual void shark::AbstractModel< InputTypeT, OutputTypeT >::weightedInputDerivative ( BatchInputType const &  pattern,
BatchOutputType const &  coefficients,
typename Batch< RealMatrix >::type const &  errorHessian,
State const &  state,
RealMatrix &  derivative,
Batch< RealMatrix >::type &  hessian 
) const
inlinevirtual

calculates the weighted sum of derivatives w.r.t the inputs

Parameters
patternthe pattern to evaluate
coefficientsthe coefficients which are used to calculate the weighted sum
errorHessianthe second derivative of the error function for every pattern
stateintermediate results stored by eval to sped up calculations of the derivatives
derivativethe calculated derivative for every pattern
hessianthe calculated hessian for every pattern

Definition at line 281 of file AbstractModel.h.

§ weightedParameterDerivative() [1/2]

template<class InputTypeT, class OutputTypeT>
virtual void shark::AbstractModel< InputTypeT, OutputTypeT >::weightedParameterDerivative ( BatchInputType const &  pattern,
BatchOutputType const &  coefficients,
State const &  state,
RealVector &  derivative 
) const
inlinevirtual

calculates the weighted sum of derivatives w.r.t the parameters.

Parameters
patternthe patterns to evaluate
coefficientsthe coefficients which are used to calculate the weighted sum for every pattern
stateintermediate results stored by eval to speed up calculations of the derivatives
derivativethe calculated derivative as sum over all derivates of all patterns

Reimplemented in shark::ConcatenatedModel< InputType, OutputType >, shark::RBFLayer, shark::Softmax, shark::SigmoidModel, and shark::LinearNorm.

Definition at line 230 of file AbstractModel.h.

Referenced by shark::NegativeLogLikelihood::evalDerivative(), and shark::AbstractModel< InputT, unsigned int >::weightedDerivatives().

§ weightedParameterDerivative() [2/2]

template<class InputTypeT, class OutputTypeT>
virtual void shark::AbstractModel< InputTypeT, OutputTypeT >::weightedParameterDerivative ( BatchInputType const &  pattern,
BatchOutputType const &  coefficients,
Batch< RealMatrix >::type const &  errorHessian,
State const &  state,
RealVector &  derivative,
RealMatrix &  hessian 
) const
inlinevirtual

calculates the weighted sum of derivatives w.r.t the parameters

Parameters
patternthe patterns to evaluate
coefficientsthe coefficients which are used to calculate the weighted sum for every pattern
errorHessianthe second derivative of the error function for every pattern
stateintermediate results stored by eval to speed up calculations of the derivatives
derivativethe calculated derivative as sum over all derivates of all patterns
hessianthe calculated hessian as sum over all derivates of all patterns

Definition at line 247 of file AbstractModel.h.

§ write()

template<class InputTypeT, class OutputTypeT>
virtual void shark::AbstractModel< InputTypeT, OutputTypeT >::write ( OutArchive archive) const
inlinevirtual

Member Data Documentation

§ m_features


The documentation for this class was generated from the following file: