Adaptive Estimation in Multivariate Response Regression with Hidden Variables

• DOI: 10.48550/arxiv.2003.13844
• Publication Date: 2020-01-01
• Authors: Xin Bing; Ning Yang; Yaosheng Xu
• Keywords: Identifiability; Collinearity; Lasso; Matrix (chemical analysis)

This paper studies the estimation of coefficient matrix $\Ttheta$ in multivariate regression with hidden variables, $Y = (\Ttheta)^TX + (B^*)^TZ E$, where $Y$ is a $m$-dimensional response vector, $X$ $p$-dimensional vector observable features, $Z$ represents $K$-dimensional unobserved possibly correlated $X$, and $E$ an independent error. The number variables $K$ unknown both $m$ $p$ are allowed but not required to grow sample size $n$. Since only observable, we provide necessary conditions for identifiability $\Ttheta$. same set shown be sufficient when error homoscedastic. Our proof constructive leads novel computationally efficient algorithm, called HIVE. first step algorithm estimate best linear prediction given which exhibits additive decomposition dense originated from correlation between variable $Z$. Under row sparsity assumption on $\Ttheta$, propose minimize penalized least squares loss by regularizing via group-lasso penalty ridge penalty. Non-asymptotic deviation bounds in-sample established. second space $B^*$ leveraging covariance structure residual step. In last step, remove effect projecting onto complement estimated $B^*$. our final estimator model identifiability, parameter statistical guarantees further extended setting heteroscedastic errors.