Multivariate Statistics: High-Dimensional and Large-Sample Approximations

A comprehensive examination of high-dimensional analysis of multivariate methods and their real-world applications Multivariate Statistics: High-Dimensional and Large-Sample Approximations is the first book of its kind to explore how classical multivariate methods can be revised and used in place of conventional statistical tools. Written by prominent researchers in the field, the book focuses on high-dimensional and large-scale approximations and details the many basic multivariate methods used to achieve high levels of accuracy. The authors begin with a fundamental presentation of the basic tools and exact distributional results of multivariate statistics, and, in addition, the derivations of most distributional results are provided. Statistical methods for high-dimensional data, such as curve data, spectra, images, and DNA microarrays, are discussed. Bootstrap approximations from a methodological point of view, theoretical accuracies in MANOVA tests, and model selection criteria are also presented. Subsequent chapters feature additional topical coverage including: High-dimensional approximations of various statistics High-dimensional statistical methods Approximations with computable error bound Selection of variables based on model selection approach Statistics with error bounds and their appearance in discriminant analysis, growth curve models, generalized linear models, profile analysis, and multiple comparison Each chapter provides real-world applications and thorough analyses of the real data. In addition, approximation formulas found throughout the book are a useful tool for both practical and theoretical statisticians, and basic results on exact distributions in multivariate analysis are included in a comprehensive, yet accessible, format. Multivariate Statistics is an excellent book for courses on probability theory in statistics at the graduate level. It is also an essential reference for both practical and theoretical statisticians who are interested in multivariate analysis and who would benefit from learning the applications of analytical probabilistic methods in statistics.

Table of Contents:
Preface. Glossary of Notation and Abbreviations. 1 Multivariate Normal and Related Distributions. 1.1 Random Vectors. 1.1.1 Mean Vector and Covariance Matrix. 1.1.2 Characteristic Function and Distribution. 1.2 Multivariate Normal Distribution. 1.2.1 Bivariate Normal Distribution. 1.2.2 Definition. 1.2.3 Some Properties. 1.3 Spherical and Elliptical Distributions. 1.4 Multivariate Cumulants. Problems. 2 Wishart Distribution. 2.1 Definition. 2.2 Some Basic Properties. 2.3 Functions of Wishart Matrices. 2.4 Cochran's Theorem. 2.5 Asymptotic Distributions. Problems. 3 Hotelling's T2 and Lambda Statistics. 3.1 Hotelling's T2 and Lambda Statistics. 3.1.1 Distribution of the T2 Statistic. 3.1.2 Decomposition of T2 and D2. 3.2 Lambda-Statistic. 3.2.1 Motivation of Lambda Statistic. 3.2.2 Distribution of Lambda Statistic. 3.3 Test for Additional Information. 3.3.1 Decomposition of Lambda Statistic. Problems. 4 Correlation Coefficients. 4.1 Ordinary Correlation Coefficients. 4.1.1 Population Correlation. 4.1.2 Sample Correlation. 4.2 Multiple Correlation Coefficient. 4.2.1 Population Multiple Correlation. 4.2.2 Sample Multiple Correlation. 4.3 Partial Correlation. 4.3.1 Population Partial Correlation. 4.3.2 Sample Partial Correlation. 4.3.3 Covariance Selection Model. Problems. 5 Asymptotic Expansions for Multivariate Basic Statistics. 5.1 Edgeworth Expansion and its Validity. 5.2 The Sample Mean Vector and Covariance Matrix. 5.3 T2Statistic. 5.3.1 Outlines of Two Methods. 5.3.2 Multivariate t-Statistic. 5.3.3 Asymptotic Expansions. 5.4 Statistics with a Class of Moments. 5.4.1 Large-Sample Expansions. 5.4.2 High-Dimensional Expansions. 5.5 Perturbation Method. 5.6 Cornish-Fisher Expansions. 5.6.1 Expansion Formulas. 5.6.2 Validity of Cornish-Fisher Expansions. 5.7 Transformations for Improved Approximations. 5.8 Bootstrap Approximations. 5.9 High-Dimensional Approximations. 5.9.1 Limiting Spectral Distribution. 5.9.2 Central Limit Theorem. 5.9.3 Martingale Limit Theorem. 5.9.4 Geometric Representation. Problems. 6 MANOVA Models. 6.1 Multivariate One-Way Analysis of Variance. 6.2 Multivariate Two-Way Analysis of Variance. 6.3 MANOVA Tests. 6.3.1 Test Criteria. 6.3.2 Large-Sample Approximations. 6.3.3 Comparison of Powers. 6.3.4 High-Dimensional Approximations. 6.4 Approximations Under Nonnormality. 6.4.1 Asymptotic Expansions. 6.4.2 Bootstrap Tests. 6.5 Distributions of Characteristic Roots. 6.5.1 Exact Distributions. 6.5.2 Large-Sample Case. 6.5.3 High-Dimensional Case. 6.6 Tests for Dimensionality. 6.6.1 Three Test Criteria. 6.6.2 Large-Sample and High-Dimensional Asymptotics. 6.7 High-Dimensional Tests. Problems. 7 Multivariate Regression. 7.1 Multivariate Linear Regression Model. 7.2 Statistical Inference. 7.3 Selection of Variables. 7.3.1 Stepwise Procedure. 7.3.2 Cp Criterion. 7.3.3 AIC Criterion. 7.3.4 Numerical Example. 7.4 Principal Component Regression. 7.5 Selection of Response Variables. 7.6 General Linear Hypotheses and Confidence Intervals. 7.7 Penalized Regression Models. Problems. 8 Classical and High-Dimensional Tests for Covariance Matrices. 8.1 Specified Covariance Matrix. 8.1.1 Likelihood Ratio Test and Moments. 8.1.2 Asymptotic Expansions. 8.1.3 High-Dimensional Tests. 8.2 Sphericity. 8.2.1 Likelihood Ratio Tests and Moments. 8.2.2 Asymptotic Expansions. 8.2.3 High-Dimensional Tests. 8.3 Intraclass Covariance Structure. 8.3.1 Likelihood Ratio Tests and Moments. 8.3.2 Asymptotic Expansions. 8.3.3 Numerical Accuracy. 8.4 Test for Independence. 8.4.1 Likelihood Ratio Tests and Moments. 8.4.2 Asymptotic Expansions. 8.4.3 High-Dimensional Tests. 8.5 Tests for Equality of Covariance Matrices. 8.5.1 Likelihood Ratio Test and Moments. 8.5.2 Asymptotic Expansions. 8.5.3 High-Dimensional Tests. Problems. 9 Discriminant Analysis. 9.1 Classification Rules for Known Distributions. 9.2 Sample Classification Rules for Normal Populations. 9.2.1 Two Normal Populations with S1 = S2. 9.2.2 Case of Several Normal Populations. 9.3 Probability of Misclassifications. 9.3.1 W-Rule. 9.3.2 Z-Rule. 9.3.3 High-Dimensional Asymptotic Results. 9.4 Canonical Discriminant Analysis. 9.4.1 Canonical Discriminant Method. 9.4.2 Test for Additional Information. 9.4.3 Selection of Variables. 9.4.4 Estimation of Dimensionality. 9.5 Regression Approach. 9.6 High-Dimensional Approach. 9.6.1 Penalized Discriminant Analysis. 9.6.2 Other Approaches. Problems. 10 Principal Component Analysis. 10.1 Definition of Principal Components. 10.2 Optimality of Principal Components. 10.3 Sample Principal Components. 10.4 MLEs of the Characteristic Roots and Vectors. 10.5 Distributions of the Characteristic Roots. 10.5.1 Exact Distribution. 10.5.2 Large-Sample Case. 10.5.3 High-Dimensional Case. 10.6 Model Selection Approach for Covariance Structures. 10.6.1 General Approach. 10.6.2 Models for Equality of the Smaller Roots. 10.6.3 Selecting a Subset of Original Variables. 10.7 Methods Related to Principal Components. 10.7.1 Fixed-Effect Principal Component Model. 10.7.2 Random-Effect Principal Components Model. Problems. 11 Canonical Correlation Analysis. 11.1 Definition of Population Canonical Correlations and Variables. 11.2 Sample Canonical Correlations. 11.3 Distributions of Canonical Correlations. 11.3.1 Distributional Reduction. 11.3.2 Large-Sample Asymptotic Distributions. 11.3.3 High-Dimensional Asymptotic Distributions. 11.3.4 Fisher's z-Transformation. 11.4 Inference for Dimensionality. 11.4.1 Test of Dimensionality. 11.4.2 Estimation of Dimensionality. 11.5 Selection of Variables. 11.5.1 Test for Redundancy. 11.5.2 Selection of Variables. Problems. 12 Growth Curve Analysis. 12.1 Growth Curve Model. 12.2 Statistical Inference: One Group. 12.2.1 Test for Adequacy. 12.2.2 Estimation and Test. 12.2.3 Confidence Intervals. 12.3 Statistical Methods: Several Groups. 12.4 Derivation of Statistical Inference. 12.4.1 A General Multivariate Linear Model. 12.4.2 Estimation. 12.4.3 LR Tests for General Linear Hypotheses. 12.4.4 Confidence Intervals. 12.5 Model Selection. 12.5.1 AIC and CAIC. 12.5.2 Derivation of CAIC. 12.5.3 Extended Growth Curve Model. Problems. 13 Approximation to the Scale-Mixted Distributions. 13.1 Introduction. 13.1.1 Simple Example: Student's t-Distribution. 13.1.2 Improving the Approximation. 13.2 Error Bounds Evaluated in Sup-Norm. 13.2.1 General Theory. 13.2.2 Scale-Mixed Normal. 13.2.3 Scale-Mixed Gamma. 13.3 Error Bounds Evaluated in L1-Norm. 13.3.1 Some Basic Results. 13.3.2 Scale-Mixed Normal Density. 13.3.3 Scale-Mixed Gamma Density. 13.3.4 Scale-Mixed Chi-square Density. 13.4 Multivariate Scale Mixtures. 13.4.1 General Theory. 13.4.2 Normal Case. 13.4.3 Gamma Case. Problems. 14 Approximation to Some Related Distributions. 14.1 Location and Scale Mixtures. 14.2 Maximum of Multivariate Variables. 14.2.1 Distribution of the Maximum Component of a Multivariate Variable. 14.2.2 Multivariate t-Distribution. 14.2.3 Multivariate F-Distribution. 14.3 Scale Mixtures of the F-Distribution. 14.4 Non-Uniform Error Bounds. 14.5 Method of Characteristic Functions. Problems. 15 Error Bounds for Approximations of Multivariate Tests. 15.1 Multivariate Scale Mixture and MANOVA Tests. 15.2 A Function of Multivariate Scale Mixture. 15.3 Hotelling's T²0 Statistic. 15.4 Wilk's Lambda Distribution. 15.4.1 Univariate Case. 15.4.2 Multivariate Case. Problems.  16 Error Bounds for Approximations to Some Other Statistics. 16.1 Linear Discriminant Function. 16.1.1 Representation as Location and Scale Mixture. 16.1.2 Large-Sample Approximations. 16.1.3 High-Dimensional Approximations. 16.1.4 Some Related Topics. 16.2 Profile Analysis. 16.2.1 Parallelism Model and MLE. 16.2.2 Distributions of γ. 16.2.3 Confidence Interval for γ. 16.3 Estimators in the Growth Curve Model. 16.3.1 Error Bounds. 16.3.2 Distribution of the Bilinear Form. 16.4 Generalized Least Squares Estimators. Problems. Appendix. A.1 Some Results on Matrices. A.1.1 Determinants and Inverse Matrices. A.1.2 Characteristic Roots and Vectors. A.1.3 Matrix Factorizations. A.1.4 Idempotent Matrices. A.2 Inequalities and Max-Min Problems. A.3 Jacobians of Transformations. Bibliography. Index.