Variable Selection Methods with Applications to Shape Restricted Regression

This dissertation consists of four major projects. Two of these projects develop and extend Bayesian variable selection methods. The remaining projects apply existing variable selection and quadratic programming methods to the problem of fitting a shape-restricted regression curve. We summarize each of these projects below. High correlation among predictors has long been an annoyance in regression analysis. The crux of the problem is that the linear regression model assumes each predictor has an independent effect on the response that can be encapsulated in the predictor's regression coefficient. When predictors are highly correlated, the data do not contain much information on the independent effects of each predictor. The high correlation among predictors can result in large standard errors for the regression coefficients and coefficients with signs opposite of what is expected based on a priori, subject-matter theory. We propose a Bayesian model that accounts for correlation among the predictors by simultaneously performing selection and clustering of the predictors. Our model combines a Dirichlet process prior and a variable selection prior for the regression coefficients. In our model, highly correlated predictors can be grouped together by setting their corresponding coefficients exactly equal. Similarly, redundant predictors can be removed from the model through the variable selection component of our prior. We demonstrate the competitiveness of our method through simulation studies and analysis of real data. The literature is replete with variable selection techniques for the classical linear regression model. It is only relatively recently that authors have begun to explore variable selection in fully nonparametric and additive regression models. One such variable selection technique is a generalization of the LASSO called the group LASSO. In this work, we demonstrate a connection between the group LASSO and Bayesian inference in additive models with a multivariate Laplace prior for model parameters similar to the connection between the LASSO and Bayesian inference in the linear model with a univariate Laplace prior for regression coefficients. We use this connection to derive approximate posterior model probabilities for additive models. We use the concept of regular and nonregular models to reduce the size of the model space and avoid costly computations. The simple regression problem in statistics consists of determining the relationship between a response variable and a single predictor variable through a regression function. Prior information is often available that suggests the regression function should have a certain shape (e.g. monotonically increasing or concave) but not necessarily a specific parametric form. Recently, Bernstein polynomials have been used to impose certain shape restrictions on regression functions. In this work, we demonstrate a connection between the monotonic regression problem and the variable selection problem in the linear model. We develop a Bayesian procedure for fitting the monotonic regression model by adapting the variable selection procedure of previous authors. We demonstrate the effectiveness of our method through simulations and the analysis of real data. The workhorse in statistical inference is the linear regression model. However, in empirical research, the assumptions implicit in the linear regression model are often too restrictive. The literature contains many nonparametric methods for fitting regression curves that only make minimal smoothness assumptions about the regression curve. However, in many situations substantive subject-matter information exists on the general...