Training of ADM-SVMs for multi-category classification. More...
#include <shark/Algorithms/Trainers/McSvmADMTrainer.h>
Public Types | |
typedef CacheType | QpFloatType |
typedef AbstractModel< InputType, RealVector > | ModelType |
typedef AbstractKernelFunction< InputType > | KernelType |
![]() | |
typedef AbstractKernelFunction< InputType > | KernelType |
![]() | |
typedef Model | ModelType |
typedef ModelType::InputType | InputType |
typedef LabelTypeT | LabelType |
typedef LabeledData< InputType, LabelType > | DatasetType |
Public Member Functions | |
McSvmADMTrainer (KernelType *kernel, double C, bool unconstrained=false) | |
std::string | name () const |
From INameable: return the class name. More... | |
void | train (KernelClassifier< InputType > &svm, const LabeledData< InputType, unsigned int > &dataset) |
![]() | |
AbstractSvmTrainer (KernelType *kernel, double C, bool offset, bool unconstrained=false) | |
AbstractSvmTrainer (KernelType *kernel, double negativeC, double positiveC, bool offset, bool unconstrained=false) | |
double | C () const |
Return the value of the regularization parameter C. More... | |
RealVector const & | regularizationParameters () const |
RealVector & | regularizationParameters () |
KernelType * | kernel () |
const KernelType * | kernel () const |
void | setKernel (KernelType *kernel) |
bool | isUnconstrained () const |
bool | trainOffset () const |
double | CacheSize () const |
void | setCacheSize (std::size_t size) |
RealVector | parameterVector () const |
get the hyper-parameter vector More... | |
void | setParameterVector (RealVector const &newParameters) |
set the vector of hyper-parameters More... | |
size_t | numberOfParameters () const |
return the number of hyper-parameters More... | |
![]() | |
virtual void | train (ModelType &model, DatasetType const &dataset)=0 |
Core of the Trainer interface. More... | |
![]() | |
virtual | ~INameable () |
![]() | |
virtual | ~ISerializable () |
Virtual d'tor. More... | |
virtual void | read (InArchive &archive) |
Read the component from the supplied archive. More... | |
virtual void | write (OutArchive &archive) const |
Write the component to the supplied archive. More... | |
void | load (InArchive &archive, unsigned int version) |
Versioned loading of components, calls read(...). More... | |
void | save (OutArchive &archive, unsigned int version) const |
Versioned storing of components, calls write(...). More... | |
BOOST_SERIALIZATION_SPLIT_MEMBER () | |
![]() | |
QpConfig (bool precomputedFlag=false, bool sparsifyFlag=true) | |
Constructor. More... | |
QpStoppingCondition & | stoppingCondition () |
Read/write access to the stopping condition. More... | |
QpStoppingCondition const & | stoppingCondition () const |
Read access to the stopping condition. More... | |
QpSolutionProperties & | solutionProperties () |
Access to the solution properties. More... | |
bool & | precomputeKernel () |
Flag for using a precomputed kernel matrix. More... | |
bool const & | precomputeKernel () const |
Flag for using a precomputed kernel matrix. More... | |
bool & | sparsify () |
Flag for sparsifying the model after training. More... | |
bool const & | sparsify () const |
Flag for sparsifying the model after training. More... | |
bool & | shrinking () |
Flag for shrinking in the decomposition solver. More... | |
bool const & | shrinking () const |
Flag for shrinking in the decomposition solver. More... | |
bool & | s2do () |
Flag for S2DO (instead of SMO) More... | |
bool const & | s2do () const |
Flag for S2DO (instead of SMO) More... | |
unsigned int & | verbosity () |
Verbosity level of the solver. More... | |
unsigned int const & | verbosity () const |
Verbosity level of the solver. More... | |
unsigned long long const & | accessCount () const |
Number of kernel accesses. More... | |
void | setMinAccuracy (double a) |
void | setMaxIterations (unsigned long long i) |
void | setTargetValue (double v) |
void | setMaxSeconds (double s) |
![]() | |
virtual | ~IParameterizable () |
Additional Inherited Members | |
![]() | |
KernelType * | m_kernel |
RealVector | m_regularizers |
Vector of regularization parameters. More... | |
bool | m_trainOffset |
bool | m_unconstrained |
Is log(C) stored internally as a parameter instead of C? If yes, then we get rid of the constraint C > 0 on the level of the parameter interface. More... | |
std::size_t | m_cacheSize |
Number of values in the kernel cache. The size of the cache in bytes is the size of one entry (4 for float, 8 for double) times this number. More... | |
![]() | |
QpStoppingCondition | m_stoppingcondition |
conditions for when to stop the QP solver More... | |
QpSolutionProperties | m_solutionproperties |
properties of the approximate solution found by the solver More... | |
bool | m_precomputedKernelMatrix |
should the solver use a precomputed kernel matrix? More... | |
bool | m_sparsify |
should the trainer sparsify the model after training? More... | |
bool | m_shrinking |
should shrinking be used? More... | |
bool | m_s2do |
should S2DO be used instead of SMO? More... | |
unsigned int | m_verbosity |
verbosity level (currently unused) More... | |
unsigned long long | m_accessCount |
kernel access count More... | |
Training of ADM-SVMs for multi-category classification.
The ADM-SVM is a special support vector machine variant for classification of more than two classes. Given are data tuples \( (x_i, y_i) \) with x-component denoting input and y-component denoting the label 1, ..., d (see the tutorial on label conventions; the implementation uses values 0 to d-1), a kernel function k(x, x') and a regularization constant C > 0. Let H denote the kernel induced reproducing kernel Hilbert space of k, and let \( \phi \) denote the corresponding feature map. Then the SVM classifier is the function
\[ h(x) = \arg \max (f_c(x)) \]
\[ f_c(x) = \langle w_c, \phi(x) \rangle + b_c \]
\[ f = (f_1, \dots, f_d) \]
with class-wise coefficients w_c and b_c given by the (primal) optimization problem
\[ \min \frac{1}{2} \sum_c \|w_c\|^2 + C \sum_i L(y_i, f(x_i)) \]
\[ \text{s.t. } \sum_c f_c = 0 \]
The special property of the so-called ADM machine is its loss function, which arises from the application of the discriminative maximum operator to absolute margin violations. Let \( h(m) = \max\{0, 1-m\} \) denote the hinge loss as a function of the margin m, then the ADM loss is given by
\[ L(y, f(x)) = \max_{c \not= y} h(-f_c(x)) \]
For more details refer to the technical report:
Fast Training of Multi-Class Support Vector Machines. Ü Dogan, T. Glasmachers, and C. Igel, Technical Report 2011/3, Department of Computer Science, University of Copenhagen, 2011.
Definition at line 97 of file McSvmADMTrainer.h.
typedef AbstractKernelFunction<InputType> shark::McSvmADMTrainer< InputType, CacheType >::KernelType |
Definition at line 104 of file McSvmADMTrainer.h.
typedef AbstractModel<InputType, RealVector> shark::McSvmADMTrainer< InputType, CacheType >::ModelType |
Definition at line 103 of file McSvmADMTrainer.h.
typedef CacheType shark::McSvmADMTrainer< InputType, CacheType >::QpFloatType |
Definition at line 101 of file McSvmADMTrainer.h.
|
inline |
Constructor
kernel | kernel function to use for training and prediction |
C | regularization parameter - always the 'true' value of C, even when unconstrained is set |
unconstrained | when a C-value is given via setParameter, should it be piped through the exp-function before using it in the solver? |
Definition at line 111 of file McSvmADMTrainer.h.
|
inlinevirtual |
From INameable: return the class name.
Reimplemented from shark::INameable.
Definition at line 116 of file McSvmADMTrainer.h.
|
inline |
Definition at line 119 of file McSvmADMTrainer.h.
References shark::QpSparseArray< QpFloatType >::add(), shark::AbstractSvmTrainer< InputType, unsigned int >::C(), shark::ArgMaxConverter< KernelExpansion< InputType > >::decisionFunction(), shark::LabeledData< InputT, LabelT >::element(), shark::LabeledData< InputT, LabelT >::inputs(), shark::LabeledData< InputT, LabelT >::labels(), shark::QpConfig::m_accessCount, shark::AbstractSvmTrainer< InputType, unsigned int >::m_cacheSize, shark::AbstractSvmTrainer< InputType, unsigned int >::m_kernel, shark::QpConfig::m_shrinking, shark::QpConfig::m_solutionproperties, shark::QpConfig::m_stoppingcondition, shark::AbstractSvmTrainer< InputType, unsigned int >::m_trainOffset, shark::numberOfClasses(), shark::LabeledData< InputT, LabelT >::numberOfElements(), shark::QpConfig::precomputeKernel(), shark::QpSparseArray< QpFloatType >::setDefaultValue(), shark::QpSolver< Problem, SelectionStrategy >::solve(), shark::BiasSolverSimplex< Matrix >::solve(), shark::QpConfig::sparsify(), and shark::blas::sum().