Error measure for classification tasks of non exclusive attributes that can be used for model training. More...
#include <shark/ObjectiveFunctions/Loss/CrossEntropyIndependent.h>
Public Member Functions | |
CrossEntropyIndependent () | |
std::string | name () const |
From INameable: return the class name. More... | |
double | eval (unsigned int const &target, RealVector const &prediction) const |
evaluate the loss for a target and a prediction More... | |
double | evalDerivative (unsigned int const &target, RealVector const &prediction, RealVector &gradient) const |
evaluate the loss and its derivative for a target and a prediction More... | |
double | evalDerivative (unsigned int const &target, RealVector const &prediction, RealVector &gradient, typename base_type::MatrixType &hessian) const |
evaluate the loss and its first and second derivative for a target and a prediction More... | |
![]() | |
AbstractLoss () | |
virtual double | eval (BatchLabelType const &target, BatchOutputType const &prediction) const=0 |
evaluate the loss for a batch of targets and a prediction More... | |
double | eval (Data< LabelType > const &targets, Data< OutputType > const &predictions) const |
virtual double | evalDerivative (BatchLabelType const &target, BatchOutputType const &prediction, BatchOutputType &gradient) const |
evaluate the loss and the derivative w.r.t. the prediction More... | |
double | operator() (LabelType const &target, OutputType const &prediction) const |
evaluate the loss for a target and a prediction More... | |
double | operator() (BatchLabelType const &target, BatchOutputType const &prediction) const |
![]() | |
virtual | ~AbstractCost () |
const Features & | features () const |
virtual void | updateFeatures () |
bool | hasFirstDerivative () const |
returns true when the first parameter derivative is implemented More... | |
bool | isLossFunction () const |
returns true when the cost function is in fact a loss function More... | |
double | operator() (Data< LabelType > const &targets, Data< OutputType > const &predictions) const |
![]() | |
virtual | ~INameable () |
Additional Inherited Members | |
![]() | |
typedef RealVector | OutputType |
typedef unsigned int | LabelType |
typedef VectorMatrixTraits< OutputType >::DenseMatrixType | MatrixType |
typedef Batch< OutputType >::type | BatchOutputType |
typedef Batch< LabelType >::type | BatchLabelType |
![]() | |
enum | Feature |
list of features a cost function can have More... | |
typedef RealVector | OutputType |
typedef unsigned int | LabelType |
typedef Batch< OutputType >::type | BatchOutputType |
typedef Batch< LabelType >::type | BatchLabelType |
typedef TypedFlags< Feature > | Features |
typedef TypedFeatureNotAvailableException< Feature > | FeatureNotAvailableException |
![]() | |
Features | m_features |
Error measure for classification tasks of non exclusive attributes that can be used for model training.
If your model should return a vector whose components are connected to multiple mutually independent attributes and the k-th component of the output vector is representing the probability of the presence of the k-th attribute given any input vector, 'CrossEntropyIndependent' is the adequate error measure for model-training. For C>1, dimension of model's output and every output dimension represents a single attribute or class respectively, it follows the formular
\[ E = - \sum_{i=1}^N \sum_{k=1}^{C} \{tar^i_k \ln model_k(in^i) + (1-tar^i_k) \ln (1-model_k(in^i))\} \]
where i runs over all input patterns. This error functional can be derivated and so used for training. In case of only one single output dimension 'CrossEntropyIndependent' returns actually the true cross entropy for two classes, using the formalism
\[ E = - \sum_{i=1}^N \{tar^i \ln model(in^i) + (1-tar^i) \ln (1-model(in^i))\} \]
For theoretical reasons it is suggested to use for neural networks the logistic sigmoid activation function at the output neurons. However, actually every sigmoid activation could be applied to the output neurons as far as the image of this function is identical to the intervall [0,1]. In this implementation every target value to be chosen from {0,1} (binary encoding). For detailed information refer to (C.M. Bishop, Neural Networks for Pattern Recognition, Clarendon Press 1996, Chapter 6.8.)
Definition at line 73 of file CrossEntropyIndependent.h.
|
inline |
|
inlinevirtual |
evaluate the loss for a target and a prediction
target | target value |
prediction | prediction, typically made by a model |
Reimplemented from shark::AbstractLoss< unsigned int, RealVector >.
Definition at line 108 of file CrossEntropyIndependent.h.
|
inlinevirtual |
evaluate the loss and its derivative for a target and a prediction
target | target value |
prediction | prediction, typically made by a model |
gradient | the gradient of the loss function with respect to the prediction |
Reimplemented from shark::AbstractLoss< unsigned int, RealVector >.
Definition at line 119 of file CrossEntropyIndependent.h.
References shark::sigmoid().
|
inlinevirtual |
evaluate the loss and its first and second derivative for a target and a prediction
target | target value |
prediction | prediction, typically made by a model |
gradient | the gradient of the loss function with respect to the prediction |
hessian | the hessian of the loss function with respect to the prediction |
Reimplemented from shark::AbstractLoss< unsigned int, RealVector >.
Definition at line 133 of file CrossEntropyIndependent.h.
References shark::blas::max(), and shark::sigmoid().
|
inlinevirtual |
From INameable: return the class name.
Reimplemented from shark::INameable.
Definition at line 102 of file CrossEntropyIndependent.h.
References shark::AbstractLoss< unsigned int, RealVector >::eval().