gpflux.sampling.sample

This module enables you to sample from (Deep) GPs using different approaches.

Module Contents

compute_A_inv_b(A: gpflow.base.TensorType, b: gpflow.base.TensorType) → tf.Tensor

Computes \(A^{-1} b\) using the Cholesky of A instead of the explicit inverse, as this is often numerically more stable.

Parameters:

• A – A positive-definite matrix with shape [..., M, M]. Can contain any leading dimensions (...) as long as they correspond to the leading dimensions in b.

• b – Tensor with shape [..., M, D]. Can contain any leading dimensions (...) as long as they correspond to the leading dimensions in A.

Returns:

Tensor with shape [..., M, D]. Leading dimensions originate from A and b.

class KernelWithFeatureDecomposition(kernel: gpflow.kernels.Kernel | NoneType, feature_functions: gpflow.keras.tf_keras.layers.Layer, feature_coefficients: gpflow.base.TensorType)

Bases: gpflow.kernels.Kernel

This class represents a kernel together with its finite feature decomposition:

\[k(x, x') = \sum_{i=0}^L \lambda_i \phi_i(x) \phi_i(x'),\]

where \(\lambda_i\) and \(\phi_i(\cdot)\) are the coefficients and features, respectively.

The decomposition can be derived from Mercer or Bochner’s theorem. For example, feature-coefficient pairs could be eigenfunction-eigenvalue pairs (Mercer) or Fourier features with constant coefficients (Bochner).

In some cases (e.g., [1] and [2]) the left-hand side (that is, the covariance function \(k(\cdot, \cdot)\)) is unknown and the kernel can only be approximated using its feature decomposition. In other cases (e.g., [3] and [4]), both the covariance function and feature decomposition are available in closed form.

[1]

Solin, Arno, and Simo Särkkä. “Hilbert space methods for reduced-rank Gaussian process regression.” Statistics and Computing (2020).

[2]

Borovitskiy, Viacheslav, et al. “Matérn Gaussian processes on Riemannian manifolds.” In Advances in Neural Information Processing Systems (2020).

[3]

Ali Rahimi and Benjamin Recht. Random features for large-scale kernel machines. In Advances in Neural Information Processing Systems (2007).

[4]

Dutordoir, Vincent, Nicolas Durrande, and James Hensman. “Sparse Gaussian processes with spherical harmonic features.” In International Conference on Machine Learning (2020).

Parameters:

• kernel –

  The kernel corresponding to the feature decomposition. If None, there is no analytical expression associated with the infinite sum and we approximate the kernel based on the feature decomposition.

  Note

  In certain cases, the analytical expression for the kernel is not available. In this case, passing None is allowed, and K() and K_diag() will be computed using the approximation provided by the feature decomposition.

• feature_functions – A Keras layer for which the call evaluates the L features of the kernel \(\phi_i(\cdot)\). For X with the shape [N, D], feature_functions(X) returns a tensor with the shape [N, L].

• feature_coefficients – A tensor with the shape [L, 1] with coefficients associated with the features, \(\lambda_i\).

property feature_functions: gpflow.keras.tf_keras.layers.Layer

Return the kernel’s features \(\phi_i(\cdot)\).

property feature_coefficients: tf.Tensor

Return the kernel’s coefficients \(\lambda_i\).

draw_conditional_sample(mean: gpflow.base.TensorType, cov: gpflow.base.TensorType, f_old: gpflow.base.TensorType) → tf.Tensor

Draw a sample \(\tilde{f}_\text{new}\) from the conditional multivariate Gaussian \(p(f_\text{new} | f_\text{old})\), where the parameters mean and cov are the mean and covariance matrix of the joint multivariate Gaussian over \([f_\text{old}, f_\text{new}]\).

Parameters:

• mean –

  A tensor with the shape [..., D, N+M] with the mean of [f_old, f_new]. For each [..., D] this is a stacked vector of the form:

  \[\begin{split}\begin{pmatrix} \operatorname{mean}(f_\text{old}) \;[N] \\ \operatorname{mean}(f_\text{new}) \;[M] \end{pmatrix}\end{split}\]

• cov –

  A tensor with the shape [..., D, N+M, N+M] with the covariance of [f_old, f_new]. For each [..., D], there is a 2x2 block matrix of the form:

  \[\begin{split}\begin{pmatrix} \operatorname{cov}(f_\text{old}, f_\text{old}) \;[N, N] & \operatorname{cov}(f_\text{old}, f_\text{new}) \;[N, M] \\ \operatorname{cov}(f_\text{new}, f_\text{old}) \;[M, N] & \operatorname{cov}(f_\text{new}, f_\text{new}) \;[M, M] \end{pmatrix}\end{split}\]

• f_old – A tensor of observations with the shape [..., D, N], drawn from Normal distribution with mean \(\operatorname{mean}(f_\text{old}) \;[N]\), and covariance \(\operatorname{cov}(f_\text{old}, f_\text{old}) \;[N, N]\)

Returns:

A sample \(\tilde{f}_\text{new}\) from the conditional normal \(p(f_\text{new} | f_\text{old})\) with the shape [..., D, M].

efficient_sample

A function that returns a Sample of a GP posterior.

class Sample

Bases: abc.ABC

This class represents a sample from a GP that you can evaluate by using the __call__ at new locations within the support of the GP.

Importantly, the same function draw (sample) is evaluated when calling it multiple times. This property is called consistency. Achieving consistency for vanilla GPs is costly because it scales cubically with the number of evaluation points, but works with any kernel. It is implemented in _efficient_sample_conditional_gaussian(). For KernelWithFeatureDecomposition, the more efficient approach following Wilson et al. [WBT+20] is implemented in _efficient_sample_matheron_rule().

See the tutorial notebooks Efficient sampling and Weight Space Approximation with Random Fourier Features for an in-depth overview.

abstract __call__(X: gpflow.base.TensorType) → tf.Tensor

Return the evaluation of the GP sample \(f(X)\) for \(f \sim GP(0, k)\).

Parameters:

X – The inputs, a tensor with the shape [N, D], where D is the input dimensionality.

Returns:

Function values, a tensor with the shape [N, P], where P is the output dimensionality.

__add__(other: Sample | Callable[[gpflow.base.TensorType], gpflow.base.TensorType]) → Sample

Allow for the summation of two instances that implement the __call__ method.

_efficient_sample_conditional_gaussian(inducing_variable: gpflow.inducing_variables.InducingVariables, kernel: gpflow.kernels.Kernel, q_mu: tf.Tensor, *, q_sqrt: gpflow.base.TensorType | None = None, whiten: bool = False) → Sample

Most costly implementation for obtaining a consistent GP sample. However, this method can be used for any kernel.

_efficient_sample_matheron_rule(inducing_variable: gpflow.inducing_variables.InducingVariables, kernel: gpflux.sampling.kernel_with_feature_decomposition.KernelWithFeatureDecomposition, q_mu: tf.Tensor, *, q_sqrt: gpflow.base.TensorType | None = None, whiten: bool = False) → Sample

Implements the efficient sampling rule from Wilson et al. [WBT+20] using the Matheron rule. To use this sampling scheme, the GP has to have a kernel of the KernelWithFeatureDecomposition type .

Parameters:

• kernel – A kernel of the KernelWithFeatureDecomposition type, which holds the covariance function and the kernel’s features and coefficients.

• q_mu – A tensor with the shape [M, P].

• q_sqrt – A tensor with the shape [P, M, M].

• whiten – Determines the parameterisation of the inducing variables.