gpflux.layers.basis_functions.fourier_features.random#

Package Contents#

class RandomFourierFeatures(kernel: gpflow.kernels.Kernel, n_components: int, **kwargs: Mapping)[source]#

Bases: RandomFourierFeaturesBase

Random Fourier features (RFF) is a method for approximating kernels. The essential element of the RFF approach [RR07] is the realization that Bochner’s theorem for stationary kernels can be approximated by a Monte Carlo sum.

We will approximate the kernel $k(\mathbf{x},{\mathbf{x}}^{\prime })$ by $\mathrm{\Phi }(\mathbf{x}{)}^{\mathrm{\top }}\mathrm{\Phi }({\mathbf{x}}^{\prime })$ where $\mathrm{\Phi }:{\mathbb{R}}^D\to {\mathbb{R}}^M$ is a finite-dimensional feature map.

The feature map is defined as:

$$\begin{array}{r}\mathrm{\Phi }(\mathbf{x})=\sqrt{\frac{2{\sigma }^2}{\ell }}\left[\begin{array}{c}\cos ({\mathit{\theta }}_1^{\mathrm{\top }}\mathbf{x})\\ \sin ({\mathit{\theta }}_1^{\mathrm{\top }}\mathbf{x})\\ ⋮\\ \cos ({\mathit{\theta }}_{\frac{M}{2}}^{\mathrm{\top }}\mathbf{x})\\ \sin ({\mathit{\theta }}_{\frac{M}{2}}^{\mathrm{\top }}\mathbf{x})\end{array}\right]\end{array}$$

where ${\sigma }^2$ is the kernel variance. The features are parameterised by random weights:

• $\mathit{\theta }\sim p(\mathit{\theta })$ where $p(\mathit{\theta })$ is the spectral density of the kernel.

At least for the squared exponential kernel, this variant of the feature mapping has more desirable theoretical properties than its counterpart form from phase-shifted cosines RandomFourierFeaturesCosine [SS15].

Parameters:

• kernel – kernel to approximate using a set of Fourier bases.

• n_components – number of components (e.g. Monte Carlo samples, quadrature nodes, etc.) used to numerically approximate the kernel.

_compute_bases(inputs: gpflow.base.TensorType) → tf.Tensor[source]#

Compute basis functions.

Returns:

A tensor with the shape [N, 2M] or [P, N, 2M].

_compute_constant() → tf.Tensor[source]#

Compute normalizing constant for basis functions.

Returns:

A tensor with the shape [] (i.e. a scalar).

class RandomFourierFeaturesCosine(kernel: gpflow.kernels.Kernel, n_components: int, **kwargs: Mapping)[source]#

Bases: RandomFourierFeaturesBase

Random Fourier Features (RFF) is a method for approximating kernels. The essential element of the RFF approach [RR07] is the realization that Bochner’s theorem for stationary kernels can be approximated by a Monte Carlo sum.

We will approximate the kernel $k(\mathbf{x},{\mathbf{x}}^{\prime })$ by $\mathrm{\Phi }(\mathbf{x}{)}^{\mathrm{\top }}\mathrm{\Phi }({\mathbf{x}}^{\prime })$ where $\mathrm{\Phi }:{\mathbb{R}}^D\to {\mathbb{R}}^M$ is a finite-dimensional feature map.

The feature map is defined as:

$$\begin{array}{r}\mathrm{\Phi }(\mathbf{x})=\sqrt{\frac{2{\sigma }^2}{\ell }}\left[\begin{array}{c}\cos ({\mathit{\theta }}_1^{\mathrm{\top }}\mathbf{x}+\tau )\\ ⋮\\ \cos ({\mathit{\theta }}_M^{\mathrm{\top }}\mathbf{x}+\tau )\end{array}\right]\end{array}$$

where ${\sigma }^2$ is the kernel variance. The features are parameterised by random weights:

• $\mathit{\theta }\sim p(\mathit{\theta })$ where $p(\mathit{\theta })$ is the spectral density of the kernel

• $\tau \sim \mathcal{U}(0,2\pi )$

Equivalent to RandomFourierFeatures by elementary trigonometric identities.

Parameters:

• kernel – kernel to approximate using a set of Fourier bases.

• n_components – number of components (e.g. Monte Carlo samples, quadrature nodes, etc.) used to numerically approximate the kernel.

build(input_shape: gpflux.types.ShapeType) → None[source]#

Creates the variables of the layer. See tf.keras.layers.Layer.build().

_compute_bases(inputs: gpflow.base.TensorType) → tf.Tensor[source]#

Compute basis functions.

Returns:

A tensor with the shape [N, M] or [P, N, M].

_compute_constant() → tf.Tensor[source]#

Compute normalizing constant for basis functions.

Returns:

A tensor with the shape [] (i.e. a scalar).

class OrthogonalRandomFeatures(kernel: gpflow.kernels.Kernel, n_components: int, **kwargs: Mapping)[source]#

Bases: gpflux.layers.basis_functions.fourier_features.random.base.RandomFourierFeatures

Orthogonal random Fourier features (ORF) [YSC+16] for more efficient and accurate kernel approximations than RandomFourierFeatures.

Parameters:

• kernel – kernel to approximate using a set of Fourier bases.

• n_components – number of components (e.g. Monte Carlo samples, quadrature nodes, etc.) used to numerically approximate the kernel.