|
|
| Record Number | 10263 |
| Reference Type | Journal Article |
| Author(s) | Hoffmann, H.;Schaal, S.;Vijayakumar, S. |
| Year | 2009 |
| Title | Local dimensionality reduction for non-parametric regression |
| Journal/Conference/Book Title | Neural Processing Letters |
| Keywords | locally weighted learning, dimensionality reduction,correlation, dimensionality reduction, factor analysis, incremental learning, kernel function, locally-weighted regression, partial least squares, principal component analysis, principal component regres |
Abstract |
Locally-weighted regression is a computationally-efficient technique for non-linear regression.
However, for high-dimensional data, this technique becomes numerically brittle and computationally
too expensive if many local models need to be maintained simultaneously. Thus, local linear
dimensionality reduction combined with locally-weighted regression seems to be a promising solution.
In this context, we review linear dimensionality-reduction methods, compare their performance on nonparametric
locally-linear regression, and discuss their ability to extend to incremental learning. The
considered methods belong to the following three groups: (1) reducing dimensionality only on the input
data, (2) modeling the joint input-output data distribution, and (3) optimizing the correlation between
projection directions and output data. Group 1 contains principal component regression (PCR); group
2 contains principal component analysis (PCA) in joint input and output space, factor analysis, and
probabilistic PCA; and group 3 contains reduced rank regression (RRR) and partial least squares
(PLS) regression. Among the tested methods, only group 3 managed to achieve robust performance
even for a non-optimal number of components (factors or projection directions). In contrast, group 1
and 2 failed for fewer components since these methods rely on the correct estimate of the true intrinsic
dimensionality. In group 3, PLS is the only method for which a computationally-efficient incremental
implementation exists. Thus, PLS appears to be ideally suited as a building block for a locally-weighted
regressor in which projection directions are incrementally added on the fly.
|
| Notes | clmc |
| URL(s) | http://www-clmc.usc.edu/publications/H/hoffmann-NPL2009.pdf
|
| Short Title | Local dimensionality reduction for non-parametric regression |
| Papers are available as Adobe PDF ".pdf" files. Adobe Reader is available for free for all computer platforms.
|
|
|
|
|
Page last modified on August 10, 2006, at 06:47 PM
|
|
