shark::LassoRegression< InputVectorType > Class Template Reference

LASSO Regression. More...

#include <shark/Algorithms/Trainers/LassoRegression.h>

+ Inheritance diagram for shark::LassoRegression< InputVectorType >:

Public Types

typedef LinearModel< InputVectorType > ModelType
 
typedef LabeledData< InputVectorType, RealVector > DataType
 
- Public Types inherited from shark::AbstractTrainer< LinearModel< InputVectorType > >
typedef LinearModel< InputVectorType > ModelType
 
typedef ModelType::InputType InputType
 
typedef typename LinearModel< InputVectorType > ::OutputType LabelType
 
typedef LabeledData< InputType, LabelTypeDatasetType
 

Public Member Functions

 LassoRegression (double _lambda, double _accuracy=0.01)
 Constructor. More...
 
std::string name () const
 From INameable: return the class name. More...
 
double lambda () const
 Return the current setting of the regularization parameter. More...
 
void setLambda (double lambda)
 Set the regularization parameter. More...
 
double accuracy () const
 Return the current setting of the accuracy (maximal gradient component of the optimization problem). More...
 
void setAccuracy (double _accuracy)
 Set the accuracy (maximal gradient component of the optimization problem). More...
 
RealVector parameterVector () const
 Get the regularization parameter lambda through the IParameterizable interface. More...
 
void setParameterVector (const RealVector &param)
 Set the regularization parameter lambda through the IParameterizable interface. More...
 
size_t numberOfParameters () const
 Return the number of parameters (one in this case). More...
 
void train (ModelType &model, DataType const &dataset)
 Train a linear model with LASSO regression. More...
 
- Public Member Functions inherited from shark::AbstractTrainer< LinearModel< InputVectorType > >
virtual void train (ModelType &model, DatasetType const &dataset)=0
 Core of the Trainer interface. More...
 
- Public Member Functions inherited from shark::INameable
virtual ~INameable ()
 
- Public Member Functions inherited from shark::ISerializable
virtual ~ISerializable ()
 Virtual d'tor. More...
 
virtual void read (InArchive &archive)
 Read the component from the supplied archive. More...
 
virtual void write (OutArchive &archive) const
 Write the component to the supplied archive. More...
 
void load (InArchive &archive, unsigned int version)
 Versioned loading of components, calls read(...). More...
 
void save (OutArchive &archive, unsigned int version) const
 Versioned storing of components, calls write(...). More...
 
 BOOST_SERIALIZATION_SPLIT_MEMBER ()
 
- Public Member Functions inherited from shark::IParameterizable
virtual ~IParameterizable ()
 

Protected Member Functions

void trainInternal (RealVector &alpha, DataType const &dataset)
 Actual training procedure. More...
 

Protected Attributes

double m_lambda
 regularization parameter More...
 
double m_accuracy
 gradient accuracy More...
 
std::size_t dim
 dimension; number of features More...
 
std::size_t ell
 number of points More...
 
RealVector label
 dense label vector, one entry per point More...
 
Batch< InputVectorType >::type data
 matrix of sparse vectors, one row per feature More...
 

Detailed Description

template<class InputVectorType = RealVector>
class shark::LassoRegression< InputVectorType >

LASSO Regression.

LASSO Regression extracts a sparse vector of regression coefficients. The original method amounts to L1-constrained least squares regression, while this implementation uses an L1 penalty instead of a constraint (which is equivalent).

For data vectors \( x_i \) with real-valued labels \( y_i \) the trainer solves the problem \( \min_w \quad \frac{1}{2} \sum_i (w^T x_i - y_i)^2 + \lambda \|w\|_1 \). The target accuracy of the solution is measured in terms of the smallest component (L1 norm) of the gradient of the objective function.

The trainer has one template parameter, namely the type of the input vectors \( x_i \). These need to be vector valued, typically either RealVector of CompressedRealVector. The resulting weight vector w is represented by a LinearModel object. Currently model outputs and labels are restricted to a single dimension.

Definition at line 70 of file LassoRegression.h.

Member Typedef Documentation

§ DataType

template<class InputVectorType = RealVector>
typedef LabeledData<InputVectorType, RealVector> shark::LassoRegression< InputVectorType >::DataType

Definition at line 74 of file LassoRegression.h.

§ ModelType

template<class InputVectorType = RealVector>
typedef LinearModel<InputVectorType> shark::LassoRegression< InputVectorType >::ModelType

Definition at line 73 of file LassoRegression.h.

Constructor & Destructor Documentation

§ LassoRegression()

template<class InputVectorType = RealVector>
shark::LassoRegression< InputVectorType >::LassoRegression ( double  _lambda,
double  _accuracy = 0.01 
)
inline

Constructor.

Parameters
_lambdavalue of the regularization parameter (see class description)
_accuracystopping criterion for the iterative solver, maximal gradient component of the objective function (see class description)

Definition at line 80 of file LassoRegression.h.

References shark::LassoRegression< InputVectorType >::m_accuracy, shark::LassoRegression< InputVectorType >::m_lambda, and RANGE_CHECK.

Member Function Documentation

§ accuracy()

template<class InputVectorType = RealVector>
double shark::LassoRegression< InputVectorType >::accuracy ( ) const
inline

Return the current setting of the accuracy (maximal gradient component of the optimization problem).

Definition at line 107 of file LassoRegression.h.

References shark::LassoRegression< InputVectorType >::m_accuracy.

§ lambda()

template<class InputVectorType = RealVector>
double shark::LassoRegression< InputVectorType >::lambda ( ) const
inline

Return the current setting of the regularization parameter.

Definition at line 94 of file LassoRegression.h.

References shark::LassoRegression< InputVectorType >::m_lambda.

Referenced by shark::LassoRegression< InputVectorType >::setLambda(), and shark::LassoRegression< InputVectorType >::trainInternal().

§ name()

template<class InputVectorType = RealVector>
std::string shark::LassoRegression< InputVectorType >::name ( ) const
inlinevirtual

From INameable: return the class name.

Reimplemented from shark::INameable.

Definition at line 89 of file LassoRegression.h.

§ numberOfParameters()

template<class InputVectorType = RealVector>
size_t shark::LassoRegression< InputVectorType >::numberOfParameters ( ) const
inlinevirtual

Return the number of parameters (one in this case).

Reimplemented from shark::IParameterizable.

Definition at line 134 of file LassoRegression.h.

§ parameterVector()

template<class InputVectorType = RealVector>
RealVector shark::LassoRegression< InputVectorType >::parameterVector ( ) const
inlinevirtual

Get the regularization parameter lambda through the IParameterizable interface.

Reimplemented from shark::IParameterizable.

Definition at line 120 of file LassoRegression.h.

References shark::LassoRegression< InputVectorType >::m_lambda.

§ setAccuracy()

template<class InputVectorType = RealVector>
void shark::LassoRegression< InputVectorType >::setAccuracy ( double  _accuracy)
inline

Set the accuracy (maximal gradient component of the optimization problem).

Definition at line 113 of file LassoRegression.h.

References shark::LassoRegression< InputVectorType >::m_accuracy, and RANGE_CHECK.

§ setLambda()

template<class InputVectorType = RealVector>
void shark::LassoRegression< InputVectorType >::setLambda ( double  lambda)
inline

§ setParameterVector()

template<class InputVectorType = RealVector>
void shark::LassoRegression< InputVectorType >::setParameterVector ( const RealVector &  param)
inlinevirtual

Set the regularization parameter lambda through the IParameterizable interface.

Reimplemented from shark::IParameterizable.

Definition at line 126 of file LassoRegression.h.

References shark::LassoRegression< InputVectorType >::m_lambda, RANGE_CHECK, and SIZE_CHECK.

§ train()

template<class InputVectorType = RealVector>
void shark::LassoRegression< InputVectorType >::train ( ModelType model,
DataType const &  dataset 
)
inline

§ trainInternal()

Member Data Documentation

§ data

template<class InputVectorType = RealVector>
Batch<InputVectorType>::type shark::LassoRegression< InputVectorType >::data
protected

matrix of sparse vectors, one row per feature

Definition at line 348 of file LassoRegression.h.

Referenced by shark::LassoRegression< InputVectorType >::trainInternal().

§ dim

template<class InputVectorType = RealVector>
std::size_t shark::LassoRegression< InputVectorType >::dim
protected

§ ell

template<class InputVectorType = RealVector>
std::size_t shark::LassoRegression< InputVectorType >::ell
protected

number of points

Definition at line 346 of file LassoRegression.h.

§ label

template<class InputVectorType = RealVector>
RealVector shark::LassoRegression< InputVectorType >::label
protected

dense label vector, one entry per point

Definition at line 347 of file LassoRegression.h.

Referenced by shark::LassoRegression< InputVectorType >::trainInternal().

§ m_accuracy

§ m_lambda


The documentation for this class was generated from the following file: