Model selection curves

Abstract:

Many popular methods of model selection involve minimizing a penalized function of the data over a set of models. The penalty in the criterion function is controlled by a penalty multiplier _ which determines the properties of the procedure. In this talk, we first review model selection criteria of the simple form Loss + Penalty and then propose studying such model selection criteria as functions of the penalty multiplier. This approach can be interpreted as exploring the stability of model selection criteria through what we call model selection curves in Mueller and Welsh (to appear, ISR). It leads to new insights into model selection and allows the comparison of a range of model selection strategies. We use the stratified bootstrap as explained in Mueller and Welsh (2005, JASA) to enhance the basic model selection curve and develop convenient numerical and graphical summaries of the results. The methodology is illustrated using simple examples and simulations. If time permits issues of robustness are included.

About the speaker: Dr Samuel Muller is Senior Lecturer in the School of Mathematics and Statistics at the University of Sydney. His research interests include Statistical Model Selection, Extreme Value Theory and Robust Methods.