Sparse Partially Linear Additive Models, Yin Lou, Jacob Bien, Rich Caruana, Johannes Gehrke, 7-17-14

Sparse Partially Linear Additive Models

Yin Lou, Jacob Bien, Rich Caruana, Johannes Gehrke

(Submitted on 17 Jul 2014)

The generalized partially linear additive model (GPLAM) is a flexible and interpretable approach to building predictive models. It combines features in an additive manner, allowing them to have either a linear or nonlinear effect on the response. However, the assignment of features to the linear and nonlinear groups is typically assumed known. Thus, to make a GPLAM a viable approach in situations in which little is known apriori about the features, one must overcome two primary model selection challenges: deciding which features to include in the model and determining which features to treat nonlinearly. We introduce sparse partially linear additive models (SPLAMs), which combine model fitting and both of these model selection challenges into a single convex optimization problem. SPLAM provides a bridge between the Lasso and sparse additive models. Through a statistical oracle inequality and thorough simulation, we demonstrate that SPLAM can outperform other methods across a broad spectrum of statistical regimes, including the high-dimensional (p≫N) setting. We develop efficient algorithms that are applied to real data sets with half a million samples and over 45,000 features with excellent predictive performance.

Subjects: Methodology (stat.ME); Learning (cs.LG); Machine Learning (stat.ML)

Cite as: arXiv:1407.4729 [stat.ME]

  (or arXiv:1407.4729v1 [stat.ME] for this version)