Multitask GPs

Multitask GPs can fit to multiple output values.

Multitask

Enabling multitask Gaussian processes.

The Multitask decorator converts a controller class into a multitask controller.

class vanguard.multitask.decorator.Multitask(num_tasks, lmc_dimension=None, rank=1, **kwargs)

Bases: Decorator

Make a GP multitask.

Example:

>>> from vanguard.base import GPController
>>>
>>> @Multitask(num_tasks=2)
... class MyController(GPController):
...     pass

Parameters:

• num_tasks (int)

• lmc_dimension (Optional[int])

• rank (int)

• kwargs (Any)

__init__(num_tasks, lmc_dimension=None, rank=1, **kwargs)

Initialise self.

Parameters:

• num_tasks (int) – The number of tasks (i.e. y-value dimension).

• lmc_dimension (Optional[int]) – If using LMC (linear model of co-regionalisation), how many latent dimensions to use. Bigger means a more complicated model. Should probably be at least as big as the number of tasks, unless you want to specifically make low-rank assumptions about the relationship between tasks. Default (None) means LMC is not used at all.

• rank (int) – The rank of the task-task covar matrix in a Kronecker product multitask kernel. Only relevant for exact GP inference.

• kwargs (Any)

property safe_updates: dict[type, set[str]]

Get a dictionary (class -> set[names]) of overrides/new methods that we consider “safe”.

verify_decorated_class(cls)

Verify that a class can be decorated by this instance.

Parameters:

cls (type[Any]) – The class to be decorated.

Raises:

• TypeError – If cls is not a subclass of the framework_class.

• TopmostDecoratorError – If cls is already decorated with a TopMostDecorator.

• MissingRequirementsError – If cls is missing a required decorator.

Return type:

None

Multitask Likelihoods

Contains GPyTorch likelihoods required in Vanguard but not implemented in GPyTorch.

class vanguard.multitask.likelihoods.FixedNoiseMultitaskGaussianLikelihood(noise, learn_additional_noise=False, batch_shape=(), **kwargs)

A multitask likelihood with heteroskedastic noise.

Combines gpytorch.likelihoods.MultitaskGaussianLikelihood with gpytorch.likelihoods.FixedNoiseGaussianLikelihood to give a multitask Gaussian likelihood where a fixed heteroskedastic observation noise can be specified for each training point and task, but there is covariance between the points or the tasks.

Parameters:

• noise (Tensor)

• learn_additional_noise (bool)

• batch_shape (Size)

• kwargs (Any)

__init__(noise, learn_additional_noise=False, batch_shape=(), **kwargs)

Initialise self.

Parameters:

• noise (Tensor) – (n_samples, n_tasks) The fixed observation noise.

• learn_additional_noise (bool) – If to learn additional observation (likelihood) noise covariance along with the specified fixed noise. Takes the same form as the covariance in gpytorch.likelihoods.MultitaskGaussianLikelihood.

• batch_shape (Size) – The batch shape of the learned noise parameter, defaults to empty Size.

• kwargs (Any)

property fixed_noise: Tensor

Get the fixed noise.

marginal(function_dist, *params, noise=None, **kwargs)

Return the marginal distribution.

If rank == 0, adds the task noises to the diagonal of the covariance matrix of the supplied gpytorch.distributions.MultivariateNormal or gpytorch.distributions.MultitaskMultivariateNormal. Otherwise, adds a rank rank covariance matrix to it.

To accomplish this, we form a new linear_operator.operators.KroneckerProductLinearOperator between \(I_{n}\), an identity matrix with size equal to the data and a (not necessarily diagonal) matrix containing the task noises \(D_{t}\).

We also incorporate a shared noise parameter from the base gpytorch.likelihoods.GaussianLikelihood that we extend.

There is also the fixed noise (supplied to __init__() as noise) represented as \(\sigma^*\) of length \(nt\) with task-contiguous blocks.

The final covariance matrix after this method is then \(K + D_{t} \otimes I_{n} + \sigma^{2}I_{nt} + diag(\sigma^*)\).

Parameters:

• function_dist (MultitaskMultivariateNormal) – Random variable whose covariance matrix is a linear_operator.LinearOperator we intend to augment.

• noise (Optional[Tensor]) – The noise (standard deviation) to use in the likelihood, None, to use the likelihoods’s own fixed noise.

• params (Any)

• kwargs (Any)

Return type:

MultitaskMultivariateNormal

Returns:

A new random variable whose covariance matrix is a linear_operator.LinearOperator with \(D_{t} \otimes I_{n}\), \(\sigma^{2}I_{nt}\) and \(diag(\sigma^*)\) added.