1 /*! \page methods Methods
3 Shogun's ML functionality is currently split into feature
4 representations, feature preprocessors, kernels, kernel normalizers,
5 distances, classifier, clustering algorithms, distributions,
6 performance evaluation measures, regression methods, structured
7 output learners. The following gives a brief overview over all the
8 ML related Algorithms/Classes/Methods implemented within shogun.
10 \section featrep_sec Feature Representations
11 Shogun supports a wide range of feature representations. Among them are the so
12 called simple features (cf., CSimpleFeatures) that are standard 2-d Matrices,
13 strings (cf., CStringFeatures) that however in contrast to other meanings of
14 string are just a list of vectors of arbitrary length and sparse features
15 (cf., CSparseFeatures) to efficiently represent sparse matrices.
17 Each of these feature objects
19 \li Simple Features (CSimpleFeatures)
20 \li Strings (CStringFeatures)
21 \li Sparse Features (CSparseFeatures)
23 supports any of the standard types from bool to floats:
32 \li 32bit Unsigned Integer
33 \li 32bit Float matrix
34 \li 64bit Float matrix
35 \li 96bit Float matrix
37 Many other feature types available. Some of them are based on the three basic
38 feature types above, like CTOPFeatures (TOP Kernel features from CHMM),
39 CFKFeatures (Fisher Kernel features from CHMM) and CRealFileFeatures (vectors
40 fetched from a binary file). It should be noted that all
41 feature objects are derived from CFeatures
43 \li CAttributeFeatures - Features of attribute value pairs.
44 \li CCombinedDotFeatures - Features that allow stacking of dot features.
45 \li CCombinedFeatures - Features that allow stacking of arbitrary features.
46 \li CDotFeatures - Features that support a certain set of features (like multiplication with a scalar + addition to a dense vector). Examples are sparse and dense features.
47 \li CDummyFeatures - Features without content; Only number of vectors is known.
48 \li CExplicitSpecFeatures - Implement spectrum kernel feature space explicitly.
49 \li CImplicitWeightedSpecFeatures - DotFeatures that implicitly implement weighted spectrum kernel features.
50 \li CWDFeatures - DotFeatures that implicitly implement weighted degree kernel features.
52 In addition, labels are represented in CLabels and the alphabet of a string in
57 \section preproc_sec Preprocessors
58 The aforementioned features can be on-the-fly preprocessed to e.g. substract the mean or normalize vectors to norm 1 etc. The following pre-processors are implemented
59 \li CNormOne - Normalizes vectors to norm 1.
60 \li CLogPlusOne - add 1 and applies log().
61 \li CPCACut - Keeps eigenvectors with the highest eigenvalues.
62 \li CPruneVarSubMean - removes dimensions with little variance, substracting the mean.
63 \li CSortUlongString - Sorts vectors.
64 \li CSortWordString - Sorts vectors.
68 \section classifiers_sec Classifiers
70 A multitude of Classifiers are implemented in shogun. Among them are several
71 standard 2-class classifiers, 1-class classifiers and multi-class
72 classifiers. Several of them are linear classifiers and SVMs. Among the
73 fastest linear SVM-classifiers are CSGD, CSVMOcas and CLibLinear (capable of
74 dealing with millions of examples and features).
76 \subsection linclassi_sec Linear Classifiers
77 \li CPerceptron - standard online perceptron
78 \li CLDA - fishers linear discriminant
79 \li CLPM - linear programming machine (1-norm regularized SVM)
80 \li CLPBoost - linear programming machine using boosting on the features
81 \li CSVMPerf - a linear svm with l2-regularized bias
82 \li CLibLinear - a linear svm with l2-regularized bias
83 \li CSVMLin - a linear svm with l2-regularized bias
84 \li CSVMOcas - a linear svm with l2-regularized bias
85 \li CSubgradientSVM - SVM based on steepest subgradient descent
86 \li CSubgradientLPM - LPM based on steepest subgradient descent
89 \subsubsection svmclassi_sec Support Vector Machines
90 \li CSVMLight - A variant of SVMlight using pr_loqo as its internal solver.
91 \li CLibSVM - LibSVM modified to use shoguns kernel framework.
92 \li CMPDSVM - Minimal Primal Dual SVM
93 \li CGPBTSVM - Gradient Projection Technique SVM
94 \li CWDSVMOcas - CSVMOcas based SVM using explicitly spanned WD-Kernel feature space
95 \li CGMNPSVM - A true multiclass one vs. rest SVM
96 \li CGNPPSVM - SVM solver based on the generalized nearest point problem
97 \li CMCSVM - An experimental multiclass SVM
98 \li CLibSVMMultiClass - LibSVMs one vs. one multiclass SVM solver
99 \li CLibSVMOneClass - LibSVMs one-class SVM
102 \subsection distmachine_sec Distance Machines
103 \li k-Nearest Neighbor - Standard k-NN
108 \section regression_sec Regression
109 \subsection Support Vector Regression
110 \li CSVRLight - SVMLight based SVR
111 \li CLibSVR - LIBSVM based SVR
113 \subsection other_regress Others
114 \li CKRR - Kernel Ridge Regression
118 \section distrib_sec Distributions
119 \li CHMM - Hidden Markov Models
120 \li CHistogram - Histogram
121 \li CLinearHMM - Markov chains (embedded in ``Linear'' HMMs)
126 \section cluster_sec Clustering
127 \li CHierarchical - Agglomerative hierarchical single linkage clustering.
128 \li CKMeans - k-Means Clustering
133 \section mkl_sec Multiple Kernel Learning
134 \li CMKLRegression for q-norm MKL with Regression
135 \li CMKLOneClass for q-norm 1-class MKL
136 \li CMKLClassification for q-norm 2-class MKL
137 \li CGMNPMKL for 1-norm multi-class MKL
142 \section kernels_sec Kernels
143 \li CAUCKernel - To maximize AUC in SVM training (takes a kernel as input)
144 \li CChi2Kernel - Chi^2 Kernel
145 \li CCombinedKernel - Combined kernel to work with multiple kernels
146 \li CCommUlongStringKernel - Spectrum Kernel with spectrums of up to 64bit
147 \li CCommWordStringKernel - Spectrum kernel with spectrum of up to 16 bit
148 \li CConstKernel - A ``kernel'' returning a constant
149 \li CCustomKernel - A user supplied custom kernel
150 \li CDiagKernel - A kernel with nonzero elements only on the diagonal
151 \li CDistanceKernel - A transformation to transform distances into similarities
152 \li CFixedDegreeStringKernel - A string kernel
153 \li CGaussianKernel - The standard Gaussian kernel
154 \li CGaussianShiftKernel - Gaussian kernel with shift (inspired by the Weighted Degree shift kernel
155 \li CGaussianShortRealKernel - Gaussian Kernel on 32bit Floats
156 \li CHistogramWordStringKernel - A TOP kernel on Sequences
157 \li CLinearByteKernel - Linear Kernel on Bytes
158 \li CLinearKernel - Linear Kernel
159 \li CLinearStringKernel - Linear Kernel on Strings
160 \li CLinearWordKernel - Linear Kernel on Words
161 \li CLocalAlignmentStringKernel - The local alignment kernel
162 \li CLocalityImprovedStringKernel - The locality improved kernel
163 \li CMatchWordStringKernel - Another String kernel
164 \li COligoStringKernel - The oligo string kernel
165 \li CPolyKernel - the polynomial kernel
166 \li CPolyMatchStringKernel - polynomial kernel on strings
167 \li CPolyMatchWordStringKernel - polynomial kernel on strings
168 \li CPyramidChi2 - pyramid chi2 kernel (from image analysis)
169 \li CRegulatoryModulesStringKernel - regulatory modules string kernel
170 \li CSalzbergWordStringKernel - salzberg features based string kernel
171 \li CSigmoidKernel - Tanh sigmoidal kernel
172 \li CSimpleLocalityImprovedStringKernel - A variant of the locality improved kernel
173 \li CSparseGaussianKernel - Gaussian Kernel on sparse features
174 \li CSparseLinearKernel - Linear Kernel on sparse features
175 \li CSparsePolyKernel - Polynomial Kernel on sparse features
176 \li CTensorProductPairKernel - The Tensor Product Pair Kernel (TPPK)
177 \li CWeightedCommWordStringKernel - A weighted (or blended) spectrum kernel
178 \li CWeightedDegreePositionStringKernel - Weighted Degree kernel with shift
179 \li CWeightedDegreeStringKernel - Weighted Degree string kernel
184 \subsection kernel_normalizer Kernel Normalizers
185 Since several of the kernels pose numerical challenges to SVM optimizers,
186 kernels can be ``normalized'' for example to have ones on the diagonal.
188 \li CSqrtDiagKernelNormalizer - divide kernel by square root of product of diagonal
189 \li CAvgDiagKernelNormalizer - divide by average diagonal value
190 \li CFirstElementKernelNormalizer - divide by first kernel element k(0,0)
191 \li CIdentityKernelNormalizer - no normalization
192 \li CDiceKernelNormalizer - normalization inspired by the dice coefficient
193 \li CRidgeKernelNormalizer - adds a ridge on the kernel diagonal
194 \li CTanimotoKernelNormalizer - tanimoto coefficient inspired normalizer
195 \li CVarianceKernelNormalizer - normalize vectors in feature space to norm 1
200 \section dist_sec Distances
201 Distance Measures to measure the distance between objects. They can be used
202 in CDistanceMachine's like CKNN. The following distances are implemented:
204 \li CBrayCurtisDistance - Bray curtis distance
205 \li CCanberraMetric - Canberra metric
206 \li CChebyshewMetric - Chebyshew metric
207 \li CChiSquareDistance - Chi^2 distance
208 \li CCosineDistance - Cosine distance
209 \li CEuclidianDistance - Euclidian Distance
210 \li CGeodesicMetric - Geodesic metric
211 \li CHammingWordDistance - Hammin distance
212 \li CJensenMetric - Jensen metric
213 \li CManhattanMetric - Manhatten metric
214 \li CMinkowskiMetric - Minkowski metric
215 \li CTanimotoDistance - Tanimoto distance
220 \section eval_sec Evaluation
221 \subsection perf_sec Performance Measures
222 Performance measures assess the quality of a prediction and are implemented
223 in CPerformanceMeasures. They following measures are implemented:
224 \li Receiver Operating Curve (ROC)
225 \li Area under the ROC curve (auROC)
226 \li Area over the ROC curve (aoROC)
227 \li Precision Recall Curve (PRC)
228 \li Area under the PRC (auPRC)
229 \li Area over the PRC (aoPRC)
230 \li Detection Error Tradeoff (DET)
231 \li Area under the DET (auDET)
232 \li Area over the DET (aoDET)
233 \li Cross Correlation coefficient (CC)
234 \li Weighted Relative Accuracy (WRAcc)
235 \li Balanced Error (BAL)