Abstract
Some online algorithms for linear classification model the uncertainty in their weights over the course of learning. Modeling the full covariance structure of the weights can provide a significant advantage for classification. However, for high-dimensional, largescale data, even though there may be many second-order feature interactions, it is computationally infeasible to maintain this covariance structure. To extend second-order methods to high-dimensional data, we develop low-rank approximations of the covariance structure. We evaluate our approach on both synthetic and real-world data sets using the confidence-weighted (Dredze et al., 2008; Crammer et al., 2009a) online learning framework. We show improvements over diagonal covariance matrices for both low and high-dimensional data.
Original language | English (US) |
---|---|
Pages (from-to) | 493-500 |
Number of pages | 8 |
Journal | Journal of Machine Learning Research |
Volume | 9 |
State | Published - 2010 |
Event | 13th International Conference on Artificial Intelligence and Statistics, AISTATS 2010 - Sardinia, Italy Duration: May 13 2010 → May 15 2010 |
ASJC Scopus subject areas
- Software
- Control and Systems Engineering
- Statistics and Probability
- Artificial Intelligence