Matrix Computations and Scientific Computing Seminar: Sketched Ridge Regression

Seminar | January 25 | 11:10 a.m.-12 p.m. | 380 Soda Hall

 Shusen Wang, UC Berkeley

 Department of Mathematics

Previous work has applied matrix sketching to speed up the least squares regression (LSR) on the n >> d data. Theoretical analysis of the sketched LSR is has been well established and refined. How the results extend to the sketched ridge regression is yet unclear. In our recent work, we study two types of the sketched ridge regression—the classical sketch and the Hessian sketch--- from two perspectives— the optimization perspective and statistical perspective— and draw many useful conclusion. The optimization analysis shows that the sketched solutions can be nearly as good as the optimal; in contrast, the statistical analysis clearly indicates that the two sketched solutions significantly increases bias or variance. Our conclusion is that the practical usefulness of the sketched ridge regression may be very limited. We also propose a simple method which we call the model averaging to improve the quality of the sketched solution, both theoretically and empirically. We argue that model average has several very useful applications in practice.