trieste.models.gpflux#

This package contains the primary interface for deep Gaussian process models. It also contains a number of TrainableProbabilisticModel wrappers for GPflux-based models. Note that currently copying/saving models is not supported, so in a Bayes Opt loop track_state should be set False. Note as well that tf.keras.backend.set_floatx() should be used to set the desired float type, consistent with the GPflow float type being used.

Package Contents#

build_vanilla_deep_gp(data: trieste.data.Dataset, search_space: trieste.space.SearchSpace, num_layers: int = NUM_LAYERS, num_inducing_points: Optional[int] = None, inner_layer_sqrt_factor: float = INNER_LAYER_SQRT_FACTOR, likelihood_variance: float = LIKELIHOOD_VARIANCE, trainable_likelihood: bool = True) → gpflux.models.DeepGP[source]#

Build a DeepGP model with sensible initial parameters. We found the default configuration used here to work well in most situation, but it should not be taken as a universally good solution.

Note that although we set all the relevant parameters to sensible values, we rely on build_constant_input_dim_deep_gp from architectures to build the model.

Parameters

  • data – Dataset from the initial design, used to estimate the variance of observations and to provide query points which are used to determine inducing point locations with k-means.

  • search_space – Search space for performing Bayesian optimization. Used for initialization of inducing locations if num_inducing_points is larger than the amount of data.

  • num_layers – Number of layers in deep GP. By default set to NUM_LAYERS.

  • num_inducing_points – Number of inducing points to use in each layer. If left unspecified (default), this number is set to either NUM_INDUCING_POINTS_PER_DIM``*dimensionality of the search space or value given by ``MAX_NUM_INDUCING_POINTS, whichever is smaller.

  • inner_layer_sqrt_factor – A multiplicative factor used to rescale hidden layers, see Config for details. By default set to INNER_LAYER_SQRT_FACTOR.

  • likelihood_variance – Initial noise variance in the likelihood function, see Config for details. By default set to LIKELIHOOD_VARIANCE.

  • trainable_likelihood – Trainable likelihood variance.

Returns

A DeepGP model with sensible default settings.

Raise

If non-positive num_layers, inner_layer_sqrt_factor, likelihood_variance or num_inducing_points is provided.

class GPfluxPredictor(optimizer: KerasOptimizer | None = None)[source]#

Bases: trieste.models.interfaces.SupportsGetObservationNoise, abc.ABC

A trainable wrapper for a GPflux deep Gaussian process model. The code assumes subclasses will use the Keras fit method for training, and so they should provide access to both a model_keras and model_gpflux. Note: due to Keras integration, the user should remember to use tf.keras.backend.set_floatx() with the desired value (consistent with GPflow) to avoid dtype errors.

Parameters

optimizer – The optimizer wrapper containing the optimizer with which to train the model and arguments for the wrapper and the optimizer. The optimizer must be an instance of a Optimizer. Defaults to Adam optimizer with 0.01 learning rate.

property model_gpflux(self) → gpflow.base.Module#

The underlying GPflux model.

property model_keras(self) → tensorflow.keras.Model#

Returns the compiled Keras model for training.

property optimizer(self) → trieste.models.optimizer.KerasOptimizer#

The optimizer wrapper for training the model.

predict(self, query_points: trieste.types.TensorType) → tuple[trieste.types.TensorType, trieste.types.TensorType]#

Note: unless otherwise noted, this returns the mean and variance of the last layer conditioned on one sample from the previous layers.

abstract sample(self, query_points: trieste.types.TensorType, num_samples: int) → trieste.types.TensorType#

Return num_samples samples from the independent marginal distributions at query_points.

Parameters

  • query_points – The points at which to sample, with shape […, N, D].

  • num_samples – The number of samples at each point.

Returns

The samples. For a predictive distribution with event shape E, this has shape […, S, N] + E, where S is the number of samples.

predict_y(self, query_points: trieste.types.TensorType) → tuple[trieste.types.TensorType, trieste.types.TensorType]#

Note: unless otherwise noted, this will return the prediction conditioned on one sample from the lower layers.

get_observation_noise(self) → trieste.types.TensorType#

Return the variance of observation noise for homoscedastic likelihoods.

Returns

The observation noise.

Raises

NotImplementedError – If the model does not have a homoscedastic likelihood.

class DeepGaussianProcess(model: DeepGP | Callable[], DeepGP], optimizer: KerasOptimizer | None = None, num_rff_features: int = 1000, continuous_optimisation: bool = True)[source]#

Bases: trieste.models.gpflux.interface.GPfluxPredictor, trieste.models.interfaces.TrainableProbabilisticModel, trieste.models.interfaces.HasReparamSampler, trieste.models.interfaces.HasTrajectorySampler

A TrainableProbabilisticModel wrapper for a GPflux DeepGP with GPLayer or LatentVariableLayer: this class does not support e.g. keras layers. We provide simple architectures that can be used with this class in the architectures.py file. Note: the user should remember to set tf.keras.backend.set_floatx() with the desired value (consistent with GPflow) so that dtype errors do not occur.

Parameters

  • model – The underlying GPflux deep Gaussian process model. Passing in a named closure rather than a model can help when copying or serialising.

  • optimizer – The optimizer configuration for training the model. Defaults to KerasOptimizer wrapper with Adam optimizer. The optimizer argument to the wrapper is used when compiling the model and fit_args is a dictionary of arguments to be used in the Keras fit method. Defaults to 400 epochs, batch size of 1000, and verbose 0. A custom callback that reduces the optimizer learning rate is used as well. See https://keras.io/api/models/model_training_apis/#fit-method for a list of possible arguments.

  • num_rff_features – The number of random Fourier features used to approximate the kernel when calling trajectory_sampler(). We use a default of 1000 as it typically performs well for a wide range of kernels. Note that very smooth kernels (e.g. RBF) can be well-approximated with fewer features.

  • continuous_optimisation – if True (default), the optimizer will keep track of the number of epochs across BO iterations and use this number as initial_epoch. This is essential to allow monitoring of model training across BO iterations.

Raises

ValueError – If model has unsupported layers, num_rff_features is less than 0, or if the optimizer is not of a supported type.

property model_gpflux(self) → gpflux.models.DeepGP#

The underlying GPflux model.

property model_keras(self) → tensorflow.keras.Model#

Returns the compiled Keras model for training.

sample(self, query_points: trieste.types.TensorType, num_samples: int) → trieste.types.TensorType#

Return num_samples samples from the independent marginal distributions at query_points.

Parameters

  • query_points – The points at which to sample, with shape […, N, D].

  • num_samples – The number of samples at each point.

Returns

The samples. For a predictive distribution with event shape E, this has shape […, S, N] + E, where S is the number of samples.

reparam_sampler(self, num_samples: int) → trieste.models.interfaces.ReparametrizationSampler[trieste.models.gpflux.interface.GPfluxPredictor]#

Return a reparametrization sampler for a DeepGaussianProcess model.

Parameters

num_samples – The number of samples to obtain.

Returns

The reparametrization sampler.

trajectory_sampler(self) → trieste.models.interfaces.TrajectorySampler[trieste.models.gpflux.interface.GPfluxPredictor]#

Return a trajectory sampler. For DeepGaussianProcess, we build trajectories using the GPflux default sampler.

Returns

The trajectory sampler.

update(self, dataset: trieste.data.Dataset) → None#

Update the model given the specified dataset. Does not train the model.

Parameters

dataset – The data with which to update the model.

optimize(self, dataset: trieste.data.Dataset) → None#

Optimize the model with the specified dataset. :param dataset: The data with which to optimize the model.