GPs with Input Uncertainty

Gaussian processes can be trained on inputs with uncertainty.

class vanguard.uncertainty.GaussianUncertaintyGPController(train_x, train_x_std, train_y, y_std, kernel_class, mean_class=<class 'gpytorch.means.constant_mean.ConstantMean'>, likelihood_class=<class 'gpytorch.likelihoods.gaussian_likelihood.FixedNoiseGaussianLikelihood'>, marginal_log_likelihood_class=<class 'gpytorch.mlls.exact_marginal_log_likelihood.ExactMarginalLogLikelihood'>, optimiser_class=<class 'torch.optim.adam.Adam'>, smart_optimiser_class=<class 'vanguard.optimise.optimiser.SmartOptimiser'>, rng=None, **kwargs)

Bases: GPController

Allows the user to pass the standard deviation of the input values.

Base class for implementing the Heteroskedastic Noisy Input Gaussian Process (HNIGP). This is a generalised version of the Noisy Input Gaussian Process (NIGP) method in [Mchutchon11] and our implementation here exploits torch.autograd to circumvent any by-hand calculations of GP derivatives.

Parameters:

• train_x (Union[Tensor, ndarray[tuple[Any, ...], dtype[floating]]])

• train_x_std (Union[Tensor, ndarray[tuple[Any, ...], dtype[floating]], float, None])

• train_y (Union[Tensor, ndarray[tuple[Any, ...], dtype[integer]], ndarray[tuple[Any, ...], dtype[floating]]])

• y_std (Union[Tensor, ndarray[tuple[Any, ...], dtype[floating]], float])

• kernel_class (type[Kernel])

• mean_class (type[Mean])

• likelihood_class (type[Likelihood])

• marginal_log_likelihood_class (type[MarginalLogLikelihood])

• optimiser_class (type[Optimizer])

• smart_optimiser_class (type[SmartOptimiser])

• rng (Optional[Generator])

• kwargs (Any)

__init__(train_x, train_x_std, train_y, y_std, kernel_class, mean_class=<class 'gpytorch.means.constant_mean.ConstantMean'>, likelihood_class=<class 'gpytorch.likelihoods.gaussian_likelihood.FixedNoiseGaussianLikelihood'>, marginal_log_likelihood_class=<class 'gpytorch.mlls.exact_marginal_log_likelihood.ExactMarginalLogLikelihood'>, optimiser_class=<class 'torch.optim.adam.Adam'>, smart_optimiser_class=<class 'vanguard.optimise.optimiser.SmartOptimiser'>, rng=None, **kwargs)

Initialise self.

Parameters:

• train_x (Union[Tensor, ndarray[tuple[Any, ...], dtype[floating]]]) – (n_samples, n_features) The mean of the inputs (or the observed values).

• train_x_std (Union[Tensor, ndarray[tuple[Any, ...], dtype[floating]], float, None]) –

  The standard deviation of input noise:

  • array_like[float] (n_features,): observation std dev per input feature,

  • array_like[float] (n_samples, n_features): observation std dev input samples in each input dimension,

  • float: single input feature, or assumed homoskedastic noise amongst inputs,

  • None: heteroskedastic (among input dimension) std dev is inferred from given noisy inputs.

• train_y (Union[Tensor, ndarray[tuple[Any, ...], dtype[integer]], ndarray[tuple[Any, ...], dtype[floating]]]) – (n_samples,) or (n_samples, 1) The responsive values.

• y_std (Union[Tensor, ndarray[tuple[Any, ...], dtype[floating]], float]) –

  The observation noise standard deviation:

  • array_like[float] (n_samples,): known heteroskedastic noise,

  • float: known homoskedastic noise assumed.

• kernel_class (type[Kernel]) – An uninstantiated subclass of gpytorch.kernels.Kernel.

• mean_class (type[Mean]) – An uninstantiated subclass of gpytorch.means.Mean to use in the prior GP. Defaults to gpytorch.means.ConstantMean.

• gp_model_class – An uninstantiated subclass of a GP model from gpytorch.models. The default is vanguard.models.ExactGPModel.

• likelihood_class (type[Likelihood]) – An uninstantiated subclass of gpytorch.likelihoods.Likelihood. The default is gpytorch.likelihoods.FixedNoiseGaussianLikelihood.

• marginal_log_likelihood_class (type[MarginalLogLikelihood]) – An uninstantiated subclass of an MLL from gpytorch.mlls. The default is gpytorch.mlls.ExactMarginalLogLikelihood.

• optimiser_class (type[Optimizer]) – An uninstantiated torch.optim.Optimizer class used for gradient-based learning of hyperparameters. The default is torch.optim.Adam.

• smart_optimiser_class (type[SmartOptimiser]) – An uninstantiated subclass of SmartOptimiser, that wraps around the given optimiser_class to enable advanced features, for example early stopping.

• rng (Optional[Generator]) – Generator instance used to generate random numbers.

• kwargs (Any) – For a complete list, see GPController.

_sgd_round(n_iters=100, gradient_every=100)

Use gradient based optimiser to tune the hyperparameters.

Additive gradient noise is set before each call to the super method.

Parameters:

• n_iters (int) – The number of gradient updates.

• gradient_every (int) – How often (in iterations) to do HNIGP input gradient steps.

Return type:

float

Returns:

The training loss at the last iteration.

property gradient_variance: Tensor | None

Return the gradient variance.

Access the value that stores the \(n\times n\) tensor of covariance resulting from Taylor expansion added to training covariance matrix.

predict_at_point(x)

Doesn’t make sense for an uncertain controller.

Parameters:

x (Union[ndarray[tuple[Any, ...], dtype[floating]], Tensor])

Return type:

NoReturn