Regularization after retention in ultrahigh dimensional linear regression models

• DOI: 10.5705/ss.202015.0413
• Publication Date: 2017-07-31T08:31:21Z
• Authors: Haolei Weng; Yang Feng; Xingye Qiao
• Keywords: Methodology (stat.ME); FOS: Computer and information sciences; 0301 basic medicine; 03 medical and health sciences; 0502 economics and business; 05 social sciences; Statistics - Methodology

In ultrahigh dimensional setting, independence screening has been both theoretically and empirically proved a useful variable selection framework with low computation cost. In this work, we propose a two-step framework by using marginal information in a different perspective from independence screening. In particular, we retain significant variables rather than screening out irrelevant ones. The new method is shown to be model selection consistent in the ultrahigh dimensional linear regression model. To improve the finite sample performance, we then introduce a three-step version and characterize its asymptotic behavior. Simulations and real data analysis show advantages of our method over independence screening and its iterative variants in certain regimes.