## Keywords (10)

Publications
Regression and Classification Using Gaussian Process Priors

# Regression and Classification Using Gaussian Process Priors,RADFORD M. NEAL

Regression and Classification Using Gaussian Process Priors
SUMMARY Gaussian processes are a natural way of specifying prior distributions o ver functions of one or more input variables. When such a function defines the mean response in a regression model with Gaussian errors, inference can be done using matrix computations, which are feasible for datasets of up to about a thousand cases. The covariance function of the Gaussian process can be given a hierarchical prior, which allows the model to discover high-level properties of the data, such as which inputs are relevant to predicting the response. Inference for these covariance hyperparameters can be done using Markov chain sampling. Classification models can be defined using Gaussian processes for underlying latent values, which can also be sampled within the Markov chain. Gaussian processes are in my view the simplest and most obvious way of defining flexible Bayesian regression and classification models, but despite some past usage, they appear to have been rather neglected as a general-purpose technique. This may be partly due to a confusion between the properties of the function being modeled and the properties of the best predictor for this unknown function. In this paper, I hope to persuade you that Gaussian processes are a fruitful way of defining prior distributions for flexible regression and classification models in whic h the regression or class probability functions are not limited to simple parametric forms. The basic idea goes back many years in a regression context, but is nevertheless not widely appreciated. The use of general Gaussian process models for classification is more recent, and t o my knowledge the work presented here is the first that implements an exact Bayesian approach. One attraction of Gaussian processes is the variety of covariance functions one can choose from, which lead to functions with different degrees of smoothness, or different sorts of additive structure. I will describe some of these possibilities, while also noting the limitations of Gaussian processes.
Published in 1999.
View Publication
 The following links allow you to view full publications. These links are maintained by other sources not affiliated with Microsoft Academic Search.
 ( www.cs.toronto.edu ) ( www.cs.utoronto.ca )

## Citation Context (25)

• ...Related work has addressed this problem by modeling C−1 outputs to imply the missing output [16], [18]...

### Alistair Reid, et al. Multi-class classification of vegetation in natural environments using...

• ...This can be performed straightforwardly using Hamiltonian Monte Carlo as described by Neal (1998)...

### Ryan Prescott Adams, et al. Tractable Nonparametric Bayesian Inference in Poisson Processes with G...

• ...Neal (1998) described the use of Markov chain Monte Carlo (MCMC) approximation for GPs...

### Shirish Krishnaj Shevade, et al. Validation-Based Sparse Gaussian Process Classifier Design

• ...O'Hagan 2 and Neal 3 used a GP prior for functions in regression analysis...

### Alexandra M. Schmidt, et al. Investigating the sensitivity of Gaussian processes to the choice of t...

• ...Classification algorithms based on Gaussian Processes (GPs) have become very popular since the influential papers by Neal (1998); Williams and Barber (1998) which motivated the development of posterior approximations which are computationally appealing alternatives to the Markov Chain Monte Carlo approach...
• ...Figure 1 also demonstrates the Automatic Relevance Determination (ARD) process (Neal, 1998) which forces the two informative covariates to small scale parameters while penalizing the other eight noisy input parameters...

## References (19)

### Gaussian Processes for Bayesian Classification via Hybrid Monte Carlo(Citations: 31)

Conference: Neural Information Processing Systems - NIPS , pp. 340-346, 1996

### Bayesian Gaussian processes for regression and classification(Citations: 91)

Published in 1997.

### Adaptive rejection sampling for gibbs sampling(Citations: 540)

Published in 1992.

### A Bayesian Analysis of Kriging(Citations: 146)

Journal: Technometrics , vol. 35, no. 4, pp. 403-410, 1993
Sort by:

## Citations (53)

### MCMC Using Ensembles of States for Problems with Fast and Slow Variables such as Gaussian Process Regression

Published in 2011.

### Elliptical slice sampling(Citations: 4)

Published in 2010.

### Slice sampling covariance hyperparameters of latent Gaussian models(Citations: 1)

Conference: Neural Information Processing Systems - NIPS , pp. 1723-1731, 2010