论文标题
多任务高维线性模型中的噪声协方差估计
Noise Covariance Estimation in Multi-Task High-dimensional Linear Models
论文作者
论文摘要
本文研究了多任务高维线性回归模型，其中不同任务之间的噪声是相关的，在中等高的维度状态下，样本量$ n $ and Dimension $ p $的顺序是相同的。我们的目标是估计噪声随机向量的协方差矩阵，或等效地将噪声变量在任何两个任务的任何一对上的相关性。将回归系数视为令人讨厌的参数，我们利用多任务弹性网络和多任务套索估计器来估计滋扰。通过准确理解平方残留矩阵的偏差并纠正这种偏见，我们开发了一个新颖的噪声协方差估计器，该噪声协方差以frobenius norm的收敛，以$ n^{ - 1/2} $当时协变量是高斯。这个新颖的估计器是有效的计算。 在适当的条件下，提出的噪声协方差估计值与事先知道多任务模型回归系数的“甲骨文”估计器的收敛速率相同。本文获得的FROBENIUS误差界限还说明了该新估计量的优势，而不是试图估计滋扰的方法估计器。 作为我们技术的副产品，我们获得了多任务弹性网络和多任务套索估计器的概括误差的估计。进行了广泛的仿真研究，以说明该方法的数值性能。
This paper studies the multi-task high-dimensional linear regression models where the noise among different tasks is correlated, in the moderately high dimensional regime where sample size $n$ and dimension $p$ are of the same order. Our goal is to estimate the covariance matrix of the noise random vectors, or equivalently the correlation of the noise variables on any pair of two tasks. Treating the regression coefficients as a nuisance parameter, we leverage the multi-task elastic-net and multi-task lasso estimators to estimate the nuisance. By precisely understanding the bias of the squared residual matrix and by correcting this bias, we develop a novel estimator of the noise covariance that converges in Frobenius norm at the rate $n^{-1/2}$ when the covariates are Gaussian. This novel estimator is efficiently computable. Under suitable conditions, the proposed estimator of the noise covariance attains the same rate of convergence as the "oracle" estimator that knows in advance the regression coefficients of the multi-task model. The Frobenius error bounds obtained in this paper also illustrate the advantage of this new estimator compared to a method-of-moments estimator that does not attempt to estimate the nuisance. As a byproduct of our techniques, we obtain an estimate of the generalization error of the multi-task elastic-net and multi-task lasso estimators. Extensive simulation studies are carried out to illustrate the numerical performance of the proposed method.
