markovflow.likelihoods.likelihoods
Module containing base classes for likelihoods.
Likelihood
Bases: tf.Module, abc.ABC
tf.Module
abc.ABC
Abstract class for likelihoods.
A likelihood defines the observation model relating the observed variables \(Y\) to the latent variables \(F\) of a generative model. The observation model is specified through its conditional density \(p(Y|F)\).
In order to perform variational inference with non-Gaussian likelihoods, a ‘variational expectation’ should be computed under a Gaussian distribution \(q(F) ~ N(μ, Σ)\). This can be defined as:
Note that the predictive density:
…is a useful metric to evaluate the quality of a Gaussian approximation \(q(F) ~ N(μ, Σ)\) to the posterior density \(p(F|Y)\).
Note
Implementations of this class should typically avoid performing computation in their __init__ method. Performing computation in the constructor conflicts with running in TensorFlow’s eager mode (and computation of gradients etc).
__init__
log_probability_density
Compute the log probability density \(log p(Y|F)\).
fs – A conditioning variable, with shape batch_shape + [num_data, obs_dim].
batch_shape + [num_data, obs_dim]
observations – A conditioned variable, with shape batch_shape + [num_data, obs_dim].
A tensor representing \(log p(yᵢ | fᵢ)\), with shape batch_shape + [num_data].
batch_shape + [num_data]
variational_expectations
Calculate a variational expectation for each observation:
…where \(q(f) ~ N(μ, P)\).
Note that \(p(y |f)\) is defined by the type of likelihood function, as specified by the observation model.
This term is used when calculating the evidence lower bound (ELBO):
f_means – The marginal \(f\) means for each state of the StateSpaceModel, with shape batch_shape + [num_data, obs_dim].
StateSpaceModel
f_covariances – The marginal \(f\) covariances for each state of the StateSpaceModel, with shape batch_shape + [num_data, obs_dim, obs_dim].
batch_shape + [num_data, obs_dim, obs_dim]
observations – The \(y\) values at which to evaluate the log probability, with shape batch_shape + [num_data, obs_dim].
A tensor with shape batch_shape + [num_data].
predict_density
Predict the density.
That is, calculate \(∫ q(F) p(Y|F) dF\) of a Gaussian approximation \(q(F) ~ N(μ, Σ)\) to the posterior density \(p(F|Y)\).
predict_mean_and_var
Compute the means and covariances of the posterior predictive distribution over outputs \(y*\) at \(x*\).
The (in most case intractable) density is given by:
…where:
\(p(f* | x*, x, y)\) is Gaussian with moments f_means and f_covariances \(p(y* | f*)\) is defined by the likelihood
\(p(f* | x*, x, y)\) is Gaussian with moments f_means and f_covariances
f_means
f_covariances
\(p(y* | f*)\) is defined by the likelihood
f_means – The marginal \(f\) means for some arbitrary predicted time points, with shape batch_shape + [num_data, obs_dim].
f_covariances – The marginal \(f\) covariances for some arbitrary predicted time points, with shape batch_shape + [num_data, obs_dim, obs_dim].
A tuple of tensors containing observation means and covariances, with respective shapes batch_shape + [num_time_points, obs_dim], batch_shape + [num_time_points, obs_dim, obs_dim].
batch_shape + [num_time_points, obs_dim]
batch_shape + [num_time_points, obs_dim, obs_dim]
PEPScalarLikelihood
Bases: gpflow.likelihoods.ScalarLikelihood
gpflow.likelihoods.ScalarLikelihood
Wrapper around GPflow likelihoods, adding functionality to compute Power Expectation Propagation updates
base – base likelihood object
num_gauss_hermite_points – number of Gauss-Hermite points
kwargs – additional arguments dictionary
_scalar_log_prob
Compute log p(Y|F). :param F: function evaluation Tensor, with shape […, latent_dim] :param Y: observation Tensor, with shape […, latent_dim]
_scalar_alpha_prob
Compute p(Y|F) :param F: function evaluation Tensor, with shape […, latent_dim] :param Y: observation Tensor, with shape […, latent_dim] :param alpha: scalar
log_expected_density
Compute log ∫ p(y=Y|f)ᵃ q(f) df, where q(f) = N(Fmu, Fvar) :param Fmu: mean function evaluation Tenself._quadrature_reduction(
self.quadrature.logspace(self._scalar_log_prob, Fmu, Fvar, Y=Y)
)sor, with shape […, latent_dim] :param Fvar: variance of function evaluation Tensor, with shape […, latent_dim] :param Y: observation Tensor, with shape […, observation_dim]: :param alpha: scalar
grad_log_expected_density
Noting I(q) = log ∫ p(y=Y|f)ᵃ q(f) df, where q(f) = N(Fmu, Fvar), this computes ∇I(q) and ∇∇I(q), where the gradient is wrt Fmu. :param Fmu: mean function evaluation Tensor, with shape […, latent_dim] :param Fvar: variance of function evaluation Tensor, with shape […, latent_dim] :param Y: observation Tensor, with shape […, observation_dim]: :param alpha: scalar
_conditional_mean
The conditional mean of Y|F
_conditional_variance
The conditional variance of Y|F
PEPGaussian
Bases: PEPScalarLikelihood
Wrapper around the univariate Gaussian Likelihood.
base – A Gaussian Likelihood object
kwargs – dictionary of additional parameters
Compute log ∫ p(y=Y|f)ᵃ q(f) df, where q(f) = N(f;Fmu, Fvar)
log ∫ p(y=Y|f)ᵃ q(f) df = log ∫ N(y; f, σ²) ᵃ N(f; Fmu, Fvar) df = log N(y; Fmu, σ² + Fvar)
Fmu – mean function evaluation Tensor, with shape […, latent_dim]
Fvar – variance of function evaluation Tensor, with shape […, latent_dim]
Y – observation Tensor, with shape […, latent_dim]
alpha – scalar
check_input_shapes
Check that the shapes of inputs to likelihood methods are valid.
f_means – A tensor with shape batch_shape + [num_data, obs_dim].
observations – A tensor with shape batch_shape + [num_data, obs_dim].
expected_obs_dim – The expected number of dimensions.
f_covariances – A tensor with shape batch_shape + [num_data, obs_dim, obs_dim].