LASSO Regression. More...
#include <shark/Algorithms/Trainers/LassoRegression.h>
Public Types | |
typedef LinearModel< InputVectorType > | ModelType |
typedef LabeledData< InputVectorType, RealVector > | DataType |
![]() | |
typedef LinearModel< InputVectorType > | ModelType |
typedef ModelType::InputType | InputType |
typedef typename LinearModel< InputVectorType > ::OutputType | LabelType |
typedef LabeledData< InputType, LabelType > | DatasetType |
Public Member Functions | |
LassoRegression (double _lambda, double _accuracy=0.01) | |
Constructor. More... | |
std::string | name () const |
From INameable: return the class name. More... | |
double | lambda () const |
Return the current setting of the regularization parameter. More... | |
void | setLambda (double lambda) |
Set the regularization parameter. More... | |
double | accuracy () const |
Return the current setting of the accuracy (maximal gradient component of the optimization problem). More... | |
void | setAccuracy (double _accuracy) |
Set the accuracy (maximal gradient component of the optimization problem). More... | |
RealVector | parameterVector () const |
Get the regularization parameter lambda through the IParameterizable interface. More... | |
void | setParameterVector (const RealVector ¶m) |
Set the regularization parameter lambda through the IParameterizable interface. More... | |
size_t | numberOfParameters () const |
Return the number of parameters (one in this case). More... | |
void | train (ModelType &model, DataType const &dataset) |
Train a linear model with LASSO regression. More... | |
![]() | |
virtual void | train (ModelType &model, DatasetType const &dataset)=0 |
Core of the Trainer interface. More... | |
![]() | |
virtual | ~INameable () |
![]() | |
virtual | ~ISerializable () |
Virtual d'tor. More... | |
virtual void | read (InArchive &archive) |
Read the component from the supplied archive. More... | |
virtual void | write (OutArchive &archive) const |
Write the component to the supplied archive. More... | |
void | load (InArchive &archive, unsigned int version) |
Versioned loading of components, calls read(...). More... | |
void | save (OutArchive &archive, unsigned int version) const |
Versioned storing of components, calls write(...). More... | |
BOOST_SERIALIZATION_SPLIT_MEMBER () | |
![]() | |
virtual | ~IParameterizable () |
Protected Member Functions | |
void | trainInternal (RealVector &alpha, DataType const &dataset) |
Actual training procedure. More... | |
Protected Attributes | |
double | m_lambda |
regularization parameter More... | |
double | m_accuracy |
gradient accuracy More... | |
std::size_t | dim |
dimension; number of features More... | |
std::size_t | ell |
number of points More... | |
RealVector | label |
dense label vector, one entry per point More... | |
Batch< InputVectorType >::type | data |
matrix of sparse vectors, one row per feature More... | |
LASSO Regression.
LASSO Regression extracts a sparse vector of regression coefficients. The original method amounts to L1-constrained least squares regression, while this implementation uses an L1 penalty instead of a constraint (which is equivalent).
For data vectors \( x_i \) with real-valued labels \( y_i \) the trainer solves the problem \( \min_w \quad \frac{1}{2} \sum_i (w^T x_i - y_i)^2 + \lambda \|w\|_1 \). The target accuracy of the solution is measured in terms of the smallest component (L1 norm) of the gradient of the objective function.
The trainer has one template parameter, namely the type of the input vectors \( x_i \). These need to be vector valued, typically either RealVector of CompressedRealVector. The resulting weight vector w is represented by a LinearModel object. Currently model outputs and labels are restricted to a single dimension.
Definition at line 70 of file LassoRegression.h.
typedef LabeledData<InputVectorType, RealVector> shark::LassoRegression< InputVectorType >::DataType |
Definition at line 74 of file LassoRegression.h.
typedef LinearModel<InputVectorType> shark::LassoRegression< InputVectorType >::ModelType |
Definition at line 73 of file LassoRegression.h.
|
inline |
Constructor.
_lambda | value of the regularization parameter (see class description) |
_accuracy | stopping criterion for the iterative solver, maximal gradient component of the objective function (see class description) |
Definition at line 80 of file LassoRegression.h.
References shark::LassoRegression< InputVectorType >::m_accuracy, shark::LassoRegression< InputVectorType >::m_lambda, and RANGE_CHECK.
|
inline |
Return the current setting of the accuracy (maximal gradient component of the optimization problem).
Definition at line 107 of file LassoRegression.h.
References shark::LassoRegression< InputVectorType >::m_accuracy.
|
inline |
Return the current setting of the regularization parameter.
Definition at line 94 of file LassoRegression.h.
References shark::LassoRegression< InputVectorType >::m_lambda.
Referenced by shark::LassoRegression< InputVectorType >::setLambda(), and shark::LassoRegression< InputVectorType >::trainInternal().
|
inlinevirtual |
From INameable: return the class name.
Reimplemented from shark::INameable.
Definition at line 89 of file LassoRegression.h.
|
inlinevirtual |
Return the number of parameters (one in this case).
Reimplemented from shark::IParameterizable.
Definition at line 134 of file LassoRegression.h.
|
inlinevirtual |
Get the regularization parameter lambda through the IParameterizable interface.
Reimplemented from shark::IParameterizable.
Definition at line 120 of file LassoRegression.h.
References shark::LassoRegression< InputVectorType >::m_lambda.
|
inline |
Set the accuracy (maximal gradient component of the optimization problem).
Definition at line 113 of file LassoRegression.h.
References shark::LassoRegression< InputVectorType >::m_accuracy, and RANGE_CHECK.
|
inline |
Set the regularization parameter.
Definition at line 100 of file LassoRegression.h.
References shark::LassoRegression< InputVectorType >::lambda(), shark::LassoRegression< InputVectorType >::m_lambda, and RANGE_CHECK.
|
inlinevirtual |
Set the regularization parameter lambda through the IParameterizable interface.
Reimplemented from shark::IParameterizable.
Definition at line 126 of file LassoRegression.h.
References shark::LassoRegression< InputVectorType >::m_lambda, RANGE_CHECK, and SIZE_CHECK.
|
inline |
Train a linear model with LASSO regression.
Definition at line 140 of file LassoRegression.h.
References shark::LassoRegression< InputVectorType >::dim, shark::inputDimension(), shark::LinearModel< InputType >::outputSize(), shark::blas::row(), shark::LinearModel< InputType >::setStructure(), SIZE_CHECK, and shark::LassoRegression< InputVectorType >::trainInternal().
Referenced by main().
|
inlineprotected |
Actual training procedure.
Definition at line 156 of file LassoRegression.h.
References CHANGE_RATE, shark::blas::column(), shark::createBatch(), shark::LassoRegression< InputVectorType >::data, shark::blas::diag(), shark::LassoRegression< InputVectorType >::dim, shark::Data< Type >::elements(), shark::blas::inner_prod(), shark::LabeledData< InputT, LabelT >::inputs(), shark::LassoRegression< InputVectorType >::label, shark::LabeledData< InputT, LabelT >::labels(), shark::LassoRegression< InputVectorType >::lambda(), shark::LassoRegression< InputVectorType >::m_accuracy, shark::LassoRegression< InputVectorType >::m_lambda, shark::blas::max(), shark::blas::min(), shark::blas::noalias(), shark::blas::norm_sqr(), PREF_MAX, PREF_MIN, shark::blas::repeat(), shark::blas::row(), shark::swap(), shark::blas::trans(), and w.
Referenced by shark::LassoRegression< InputVectorType >::train().
|
protected |
matrix of sparse vectors, one row per feature
Definition at line 348 of file LassoRegression.h.
Referenced by shark::LassoRegression< InputVectorType >::trainInternal().
|
protected |
dimension; number of features
Definition at line 345 of file LassoRegression.h.
Referenced by shark::LassoRegression< InputVectorType >::train(), and shark::LassoRegression< InputVectorType >::trainInternal().
|
protected |
number of points
Definition at line 346 of file LassoRegression.h.
|
protected |
dense label vector, one entry per point
Definition at line 347 of file LassoRegression.h.
Referenced by shark::LassoRegression< InputVectorType >::trainInternal().
|
protected |
gradient accuracy
Definition at line 344 of file LassoRegression.h.
Referenced by shark::LassoRegression< InputVectorType >::accuracy(), shark::LassoRegression< InputVectorType >::LassoRegression(), shark::LassoRegression< InputVectorType >::setAccuracy(), and shark::LassoRegression< InputVectorType >::trainInternal().
|
protected |
regularization parameter
Definition at line 343 of file LassoRegression.h.
Referenced by shark::LassoRegression< InputVectorType >::lambda(), shark::LassoRegression< InputVectorType >::LassoRegression(), shark::LassoRegression< InputVectorType >::parameterVector(), shark::LassoRegression< InputVectorType >::setLambda(), shark::LassoRegression< InputVectorType >::setParameterVector(), and shark::LassoRegression< InputVectorType >::trainInternal().