Regularized Estimation of Covariance Matrices for Longitudinal Data Through Smoothing and Shrinkage

This dissertation develops methods that efficiently estimate a covariance matrix of longitudinal data without making restrictive parametric assumptions, even when the dimension is large relative to the sample size. The high dimensionality is an important issue because the number of free elements in the covariance matrix increases quadratically with its dimension. Charles Stein pointed out in the 1950's that the unrestricted maximum likelihood estimator of the covariance matrix is statistically inefficient. To gain statistical efficiency and overcome the high dimensionality problem in covariance matrix estimation, an effective regularization scheme is needed. In this dissertation we design such schemes using the regression formulation of covariance matrix estimation advocated by Pourahmadi (1999, 2000). We propose penalized likelihood methods for producing statistically efficient estimator of a covariance matrix for longitudinal data. The approaches parameterize the covariance matrix through the modified Cholesky decomposition. For longitudinal data, the entries of the lower triangular and the diagonal matrix associated with the modified Cholesky decomposition can be interpreted as regression coefficients and prediction error variances when regressing a measurement on its predecessors. Many covariance matrix estimation methods have been developed based on two observations: first, there is usually some kind of continuity among neighboring elements in the lower triangular; second, the lower triangular is likely to have many off-diagonal elements that are zero or close to zero. Wu and Pourahmadi (2003) and Huang, Liu, and Liu (2007) proposed to smooth the long subdiagonals of the lower triangular using local polynomial or splines techniques while truncate the short subdiagonals to zeros. Their approaches are rather restrictive and can't be easily combined with shrinkage. Huang et al. (2006) proposed to shrink the elements of the lower triangular by imposing the L1 or L2 penalty using the penalized likelihood, but it completely ignores the possible smoothness in the modified Cholesky factor. Our first new approach relaxed these restrictions and employs roughness penalty to impose smoothness in the rows or subdiagonals of the lower triangular. Use of roughness has been well studied for nonparametric function estimation and the second-order roughness penalty can be viewed as an approximation of the integrated squared second derivative penalty. This proposed new smoothing method can be easily combined with the shrinkage. Our second new approach combines smoothing with shrinkage using penalized likelihood. Our method is general enough to include various combinations of the shrinkage penalty (no penalty, the L1 penalty, or the L2 penalty) and the roughness penalty (no smoothing, smoothing along the rows of the lower triangular matrix, or smoothing along the subdiagonals of the lower triangular matrix), while the best combination can be chosen using a data-driven method. It turns out that combination of shrinkage and smoothing does better and sometimes much better than using shrinkage or smoothing alone. Our simulation study shows the superior performance of this new method. Most of existing works on covariance matrix estimation are only directly applicable to balanced longitudinal data. In this dissertation we also consider the unbalanced case by treating it as a missing data problem and the EM algorithm can be easily applied to our proposed approach. The practical value of our methods is demonstrated in its application to efficient estimation of regression mean parameters for longitudinal data. It is well known that the regression mean...