Academic
Publications
LASSO, Iterative Feature Selection and the Correlation Selector: Oracle inequalities and numerical performances

LASSO, Iterative Feature Selection and the Correlation Selector: Oracle inequalities and numerical performances,10.1214/08-EJS288,Electronic Journal o

LASSO, Iterative Feature Selection and the Correlation Selector: Oracle inequalities and numerical performances   (Citations: 3)
BibTex | RIS | RefWorks Download
We propose a general family of algorithms for regression estimation with quadratic loss, on the basis of geometrical considerations. These algorithms are able to select relevant functions into a large dictionary. We prove that a lot of methods that have already been studied for this task (LASSO, Dantzig selector, Iterative Feature Selection, among others) belong to our family, and exhibit another particular member of this family that we call Correlation Selector in this paper. Using general properties of our family of algorithm we prove oracle inequalities for IFS, for the LASSO and for the Correlation Selector, and compare numerical performances of these estimators on a toy example.
Journal: Electronic Journal of Statistics , vol. 2, no. 2008, pp. 1129-1152, 2008
Cumulative Annual
View Publication
The following links allow you to view full publications. These links are maintained by other sources not affiliated with Microsoft Academic Search.
    • ...Also, note that this choice for s is not necessarily the best choice in practice: it requires the knowledge of σ 2 , and it is usually too large (see the experiments in [2] for the case K =1 ). Moreover it is not data-driven...
    • ...We are now ready to give the proof of Theorem 1. The proof heavily uses the geometrical considerations given in [2]...
    • ...Using classical convex analysis result (see [2] for example), we have, for any b ∈ R p ,...
    • ...see for example [13] for the proof, and [2] for a discussion of the geometric role of the constraint � (1/n)X � (Y − Xb)� ∞ ≤ s . So, in some way, the estimator ˆ β(s, t) is really a variant of the LASSO, that tries to take into account the fact that β has blocks...

    Pierre Alquier. An Algorithm for Iterative Selection of Blocks of Features

    • ...In this work, we propose a generalized version both of the LASSO and the Dantzig Selector, based on the geometrical remarks about the LASSO in [Alq08, AH08]...
    • ...In the next section, we motivate the use of the studied family of estimators through geometrical considerations stated in [AH08]...
    • ...Based on these geometrical considerations, we proposed in [AH08] to study the following transductive estimator:  ...
    • ...Following our previous work [AH08], it is possible to provide such results for the Transductive LASSO...

    Pierre Alquieret al. Transductive versions of the LASSO and the Dantzig Selector

Sort by: