gpflux.layers.basis_functions.fourier_features.random#

A kernel’s features and coefficients using Random Fourier Features (RFF).

Package Contents#

class RandomFourierFeatures(kernel: gpflow.kernels.Kernel, n_components: int, **kwargs: Mapping)[source]#

Bases: RandomFourierFeaturesBase

Random Fourier features (RFF) is a method for approximating kernels. The essential element of the RFF approach [RR07] is the realization that Bochner’s theorem for stationary kernels can be approximated by a Monte Carlo sum.

We will approximate the kernel \(k(\mathbf{x}, \mathbf{x}')\) by \(\Phi(\mathbf{x})^\top \Phi(\mathbf{x}')\) where \(\Phi: \mathbb{R}^{D} \to \mathbb{R}^{M}\) is a finite-dimensional feature map.

The feature map is defined as:

\[\begin{split}\Phi(\mathbf{x}) = \sqrt{\frac{2 \sigma^2}{\ell}} \begin{bmatrix} \cos(\boldsymbol{\theta}_1^\top \mathbf{x}) \\ \sin(\boldsymbol{\theta}_1^\top \mathbf{x}) \\ \vdots \\ \cos(\boldsymbol{\theta}_{\frac{M}{2}}^\top \mathbf{x}) \\ \sin(\boldsymbol{\theta}_{\frac{M}{2}}^\top \mathbf{x}) \end{bmatrix}\end{split}\]

where \(\sigma^2\) is the kernel variance. The features are parameterised by random weights:

  • \(\boldsymbol{\theta} \sim p(\boldsymbol{\theta})\) where \(p(\boldsymbol{\theta})\) is the spectral density of the kernel.

At least for the squared exponential kernel, this variant of the feature mapping has more desirable theoretical properties than its counterpart form from phase-shifted cosines RandomFourierFeaturesCosine [SS15].

Parameters:

  • kernel – kernel to approximate using a set of Fourier bases.

  • n_components – number of components (e.g. Monte Carlo samples, quadrature nodes, etc.) used to numerically approximate the kernel.

_compute_bases(inputs: gpflow.base.TensorType) → tf.Tensor[source]#

Compute basis functions.

Returns:

A tensor with the shape [N, 2M] or [P, N, 2M].

_compute_constant() → tf.Tensor[source]#

Compute normalizing constant for basis functions.

Returns:

A tensor with the shape [] (i.e. a scalar).

class RandomFourierFeaturesCosine(kernel: gpflow.kernels.Kernel, n_components: int, **kwargs: Mapping)[source]#

Bases: RandomFourierFeaturesBase

Random Fourier Features (RFF) is a method for approximating kernels. The essential element of the RFF approach [RR07] is the realization that Bochner’s theorem for stationary kernels can be approximated by a Monte Carlo sum.

We will approximate the kernel \(k(\mathbf{x}, \mathbf{x}')\) by \(\Phi(\mathbf{x})^\top \Phi(\mathbf{x}')\) where \(\Phi: \mathbb{R}^{D} \to \mathbb{R}^{M}\) is a finite-dimensional feature map.

The feature map is defined as:

\[\begin{split}\Phi(\mathbf{x}) = \sqrt{\frac{2 \sigma^2}{\ell}} \begin{bmatrix} \cos(\boldsymbol{\theta}_1^\top \mathbf{x} + \tau) \\ \vdots \\ \cos(\boldsymbol{\theta}_M^\top \mathbf{x} + \tau) \end{bmatrix}\end{split}\]

where \(\sigma^2\) is the kernel variance. The features are parameterised by random weights:

  • \(\boldsymbol{\theta} \sim p(\boldsymbol{\theta})\) where \(p(\boldsymbol{\theta})\) is the spectral density of the kernel

  • \(\tau \sim \mathcal{U}(0, 2\pi)\)

Equivalent to RandomFourierFeatures by elementary trigonometric identities.

Parameters:

  • kernel – kernel to approximate using a set of Fourier bases.

  • n_components – number of components (e.g. Monte Carlo samples, quadrature nodes, etc.) used to numerically approximate the kernel.

build(input_shape: gpflux.types.ShapeType) → None[source]#

Creates the variables of the layer. See tf.keras.layers.Layer.build().

_compute_bases(inputs: gpflow.base.TensorType) → tf.Tensor[source]#

Compute basis functions.

Returns:

A tensor with the shape [N, M] or [P, N, M].

_compute_constant() → tf.Tensor[source]#

Compute normalizing constant for basis functions.

Returns:

A tensor with the shape [] (i.e. a scalar).

class OrthogonalRandomFeatures(kernel: gpflow.kernels.Kernel, n_components: int, **kwargs: Mapping)[source]#

Bases: gpflux.layers.basis_functions.fourier_features.random.base.RandomFourierFeatures

Orthogonal random Fourier features (ORF) [YSC+16] for more efficient and accurate kernel approximations than RandomFourierFeatures.

Parameters:

  • kernel – kernel to approximate using a set of Fourier bases.

  • n_components – number of components (e.g. Monte Carlo samples, quadrature nodes, etc.) used to numerically approximate the kernel.