This week's Graduate Student Seminar speaker is Lucas Mentch.
Talk Title: Inference for Supervised Learning Schemes: Regression Trees and CLTs
Abstract
This talk presents initial steps towards inference using ensemble-tree methods. Ensemble methods based on bootstrapping have improved the predictive accuracy of individual trees, but fail to provide a framework in which distributional results can be easily determined. Instead of aggregating bootstrap samples, we consider predicting by averaging over trees built on subsamples of the training set and demonstrate that such an estimator takes the form of a U-statistic. As such, predictions for individual feature vectors are asymptotically normal, thereby allowing for confidence intervals to accompany predictions. We derive rates at which the subsample size may grow with total sample size to establish these results. Frequently, a subset of subsamples will be used for computational speed; here our estimators take the form of an incomplete U-statistic for which equivalent results can be derived. We end by demonstrating that this setup also provides a framework for testing significance of individual features.