shark::CrossEntropyIndependent Class Reference

Error measure for classification tasks of non exclusive attributes that can be used for model training. More...

#include <shark/ObjectiveFunctions/Loss/CrossEntropyIndependent.h>

+ Inheritance diagram for shark::CrossEntropyIndependent:

Public Member Functions

 CrossEntropyIndependent ()
 
std::string name () const
 From INameable: return the class name. More...
 
double eval (unsigned int const &target, RealVector const &prediction) const
 evaluate the loss for a target and a prediction More...
 
double evalDerivative (unsigned int const &target, RealVector const &prediction, RealVector &gradient) const
 evaluate the loss and its derivative for a target and a prediction More...
 
double evalDerivative (unsigned int const &target, RealVector const &prediction, RealVector &gradient, typename base_type::MatrixType &hessian) const
 evaluate the loss and its first and second derivative for a target and a prediction More...
 
- Public Member Functions inherited from shark::AbstractLoss< unsigned int, RealVector >
 AbstractLoss ()
 
virtual double eval (BatchLabelType const &target, BatchOutputType const &prediction) const=0
 evaluate the loss for a batch of targets and a prediction More...
 
double eval (Data< LabelType > const &targets, Data< OutputType > const &predictions) const
 
virtual double evalDerivative (BatchLabelType const &target, BatchOutputType const &prediction, BatchOutputType &gradient) const
 evaluate the loss and the derivative w.r.t. the prediction More...
 
double operator() (LabelType const &target, OutputType const &prediction) const
 evaluate the loss for a target and a prediction More...
 
double operator() (BatchLabelType const &target, BatchOutputType const &prediction) const
 
- Public Member Functions inherited from shark::AbstractCost< unsigned int, RealVector >
virtual ~AbstractCost ()
 
const Featuresfeatures () const
 
virtual void updateFeatures ()
 
bool hasFirstDerivative () const
 returns true when the first parameter derivative is implemented More...
 
bool isLossFunction () const
 returns true when the cost function is in fact a loss function More...
 
double operator() (Data< LabelType > const &targets, Data< OutputType > const &predictions) const
 
- Public Member Functions inherited from shark::INameable
virtual ~INameable ()
 

Additional Inherited Members

- Public Types inherited from shark::AbstractLoss< unsigned int, RealVector >
typedef RealVector OutputType
 
typedef unsigned int LabelType
 
typedef VectorMatrixTraits< OutputType >::DenseMatrixType MatrixType
 
typedef Batch< OutputType >::type BatchOutputType
 
typedef Batch< LabelType >::type BatchLabelType
 
- Public Types inherited from shark::AbstractCost< unsigned int, RealVector >
enum  Feature
 list of features a cost function can have More...
 
typedef RealVector OutputType
 
typedef unsigned int LabelType
 
typedef Batch< OutputType >::type BatchOutputType
 
typedef Batch< LabelType >::type BatchLabelType
 
typedef TypedFlags< FeatureFeatures
 
typedef TypedFeatureNotAvailableException< FeatureFeatureNotAvailableException
 
- Protected Attributes inherited from shark::AbstractCost< unsigned int, RealVector >
Features m_features
 

Detailed Description

Error measure for classification tasks of non exclusive attributes that can be used for model training.

If your model should return a vector whose components are connected to multiple mutually independent attributes and the k-th component of the output vector is representing the probability of the presence of the k-th attribute given any input vector, 'CrossEntropyIndependent' is the adequate error measure for model-training. For C>1, dimension of model's output and every output dimension represents a single attribute or class respectively, it follows the formular

\[ E = - \sum_{i=1}^N \sum_{k=1}^{C} \{tar^i_k \ln model_k(in^i) + (1-tar^i_k) \ln (1-model_k(in^i))\} \]

where i runs over all input patterns. This error functional can be derivated and so used for training. In case of only one single output dimension 'CrossEntropyIndependent' returns actually the true cross entropy for two classes, using the formalism

\[ E = - \sum_{i=1}^N \{tar^i \ln model(in^i) + (1-tar^i) \ln (1-model(in^i))\} \]

For theoretical reasons it is suggested to use for neural networks the logistic sigmoid activation function at the output neurons. However, actually every sigmoid activation could be applied to the output neurons as far as the image of this function is identical to the intervall [0,1]. In this implementation every target value to be chosen from {0,1} (binary encoding). For detailed information refer to (C.M. Bishop, Neural Networks for Pattern Recognition, Clarendon Press 1996, Chapter 6.8.)

Definition at line 73 of file CrossEntropyIndependent.h.

Constructor & Destructor Documentation

§ CrossEntropyIndependent()

Member Function Documentation

§ eval()

double shark::CrossEntropyIndependent::eval ( unsigned int const &  target,
RealVector const &  prediction 
) const
inlinevirtual

evaluate the loss for a target and a prediction

Parameters
targettarget value
predictionprediction, typically made by a model

Reimplemented from shark::AbstractLoss< unsigned int, RealVector >.

Definition at line 108 of file CrossEntropyIndependent.h.

§ evalDerivative() [1/2]

double shark::CrossEntropyIndependent::evalDerivative ( unsigned int const &  target,
RealVector const &  prediction,
RealVector &  gradient 
) const
inlinevirtual

evaluate the loss and its derivative for a target and a prediction

Parameters
targettarget value
predictionprediction, typically made by a model
gradientthe gradient of the loss function with respect to the prediction

Reimplemented from shark::AbstractLoss< unsigned int, RealVector >.

Definition at line 119 of file CrossEntropyIndependent.h.

References shark::sigmoid().

§ evalDerivative() [2/2]

double shark::CrossEntropyIndependent::evalDerivative ( unsigned int const &  target,
RealVector const &  prediction,
RealVector &  gradient,
typename base_type::MatrixType hessian 
) const
inlinevirtual

evaluate the loss and its first and second derivative for a target and a prediction

Parameters
targettarget value
predictionprediction, typically made by a model
gradientthe gradient of the loss function with respect to the prediction
hessianthe hessian of the loss function with respect to the prediction

Reimplemented from shark::AbstractLoss< unsigned int, RealVector >.

Definition at line 133 of file CrossEntropyIndependent.h.

References shark::blas::max(), and shark::sigmoid().

§ name()

std::string shark::CrossEntropyIndependent::name ( ) const
inlinevirtual

From INameable: return the class name.

Reimplemented from shark::INameable.

Definition at line 102 of file CrossEntropyIndependent.h.

References shark::AbstractLoss< unsigned int, RealVector >::eval().


The documentation for this class was generated from the following file: