gpflux.layers.basis_functions.fourier_features.random.base

Module Contents

_sample_students_t(nu: float, shape: gpflux.types.ShapeType, dtype: gpflow.base.DType) → gpflow.base.TensorType

Draw samples from a (central) Student’s t-distribution using the following:

BETA ~ Gamma(nu/2, nu/2) (shape-rate parameterization) X ~ Normal(0, 1/BETA)

then:

X ~ StudentsT(nu)

Note this is equivalent to the more commonly used parameterization

Z ~ Chi2(nu) = Gamma(nu/2, 1/2) EPSILON ~ Normal(0, 1) X = EPSILON * sqrt(nu/Z)

To see this, note

Z/nu ~ Gamma(nu/2, nu/2)

and

X ~ Normal(0, nu/Z)

The equivalence becomes obvious when we set BETA = Z/nu

class RandomFourierFeaturesBase(kernel: gpflow.kernels.Kernel, n_components: int, **kwargs: Mapping)

Bases: gpflux.layers.basis_functions.fourier_features.base.FourierFeaturesBase

The base class for all Fourier feature layers, used for both random Fourier feature layers and quadrature layers. We subclass tf.keras.layers.Layer, so we must provide :method:`build` and :method:`call` methods.

Parameters:

• kernel – kernel to approximate using a set of Fourier bases.

• n_components – number of components (e.g. Monte Carlo samples, quadrature nodes, etc.) used to numerically approximate the kernel.

build(input_shape: gpflux.types.ShapeType) → None

Creates the variables of the layer. See tf.keras.layers.Layer.build().

static rff_constant(variance: gpflow.base.TensorType, output_dim: int) → tf.Tensor

Normalizing constant for random Fourier features.

class RandomFourierFeatures(kernel: gpflow.kernels.Kernel, n_components: int, **kwargs: Mapping)

Bases: RandomFourierFeaturesBase

Random Fourier features (RFF) is a method for approximating kernels. The essential element of the RFF approach [RR07] is the realization that Bochner’s theorem for stationary kernels can be approximated by a Monte Carlo sum.

We will approximate the kernel \(k(\mathbf{x}, \mathbf{x}')\) by \(\Phi(\mathbf{x})^\top \Phi(\mathbf{x}')\) where \(\Phi: \mathbb{R}^{D} \to \mathbb{R}^{M}\) is a finite-dimensional feature map.

The feature map is defined as:

\[\begin{split}\Phi(\mathbf{x}) = \sqrt{\frac{2 \sigma^2}{\ell}} \begin{bmatrix} \cos(\boldsymbol{\theta}_1^\top \mathbf{x}) \\ \sin(\boldsymbol{\theta}_1^\top \mathbf{x}) \\ \vdots \\ \cos(\boldsymbol{\theta}_{\frac{M}{2}}^\top \mathbf{x}) \\ \sin(\boldsymbol{\theta}_{\frac{M}{2}}^\top \mathbf{x}) \end{bmatrix}\end{split}\]

where \(\sigma^2\) is the kernel variance. The features are parameterised by random weights:

• \(\boldsymbol{\theta} \sim p(\boldsymbol{\theta})\) where \(p(\boldsymbol{\theta})\) is the spectral density of the kernel.

At least for the squared exponential kernel, this variant of the feature mapping has more desirable theoretical properties than its counterpart form from phase-shifted cosines RandomFourierFeaturesCosine [SS15].

Parameters:

• kernel – kernel to approximate using a set of Fourier bases.

• n_components – number of components (e.g. Monte Carlo samples, quadrature nodes, etc.) used to numerically approximate the kernel.

_compute_bases(inputs: gpflow.base.TensorType) → tf.Tensor

Compute basis functions.

Returns:

A tensor with the shape [N, 2M] or [P, N, 2M].

_compute_constant() → tf.Tensor

Compute normalizing constant for basis functions.

Returns:

A tensor with the shape [] (i.e. a scalar).

class RandomFourierFeaturesCosine(kernel: gpflow.kernels.Kernel, n_components: int, **kwargs: Mapping)

Bases: RandomFourierFeaturesBase

Random Fourier Features (RFF) is a method for approximating kernels. The essential element of the RFF approach [RR07] is the realization that Bochner’s theorem for stationary kernels can be approximated by a Monte Carlo sum.

We will approximate the kernel \(k(\mathbf{x}, \mathbf{x}')\) by \(\Phi(\mathbf{x})^\top \Phi(\mathbf{x}')\) where \(\Phi: \mathbb{R}^{D} \to \mathbb{R}^{M}\) is a finite-dimensional feature map.

The feature map is defined as:

\[\begin{split}\Phi(\mathbf{x}) = \sqrt{\frac{2 \sigma^2}{\ell}} \begin{bmatrix} \cos(\boldsymbol{\theta}_1^\top \mathbf{x} + \tau) \\ \vdots \\ \cos(\boldsymbol{\theta}_M^\top \mathbf{x} + \tau) \end{bmatrix}\end{split}\]

where \(\sigma^2\) is the kernel variance. The features are parameterised by random weights:

• \(\boldsymbol{\theta} \sim p(\boldsymbol{\theta})\) where \(p(\boldsymbol{\theta})\) is the spectral density of the kernel

• \(\tau \sim \mathcal{U}(0, 2\pi)\)

Equivalent to RandomFourierFeatures by elementary trigonometric identities.

Parameters:

• kernel – kernel to approximate using a set of Fourier bases.

• n_components – number of components (e.g. Monte Carlo samples, quadrature nodes, etc.) used to numerically approximate the kernel.

build(input_shape: gpflux.types.ShapeType) → None

Creates the variables of the layer. See tf.keras.layers.Layer.build().

_compute_bases(inputs: gpflow.base.TensorType) → tf.Tensor

Compute basis functions.

Returns:

A tensor with the shape [N, M] or [P, N, M].

_compute_constant() → tf.Tensor

Compute normalizing constant for basis functions.

Returns:

A tensor with the shape [] (i.e. a scalar).