Multitask multioutput GPy Coregionalized Regression with non-Gaussian Likelihood and Laplace inference function

Question

I want to perform coregionalized regression in GPy, however I am using a Bernoulli likelihood and then to estimate that as a Gaussian, I use Laplace inference. The code below shows how I would usually run a single-output GP with this set up (with my custom PjkRbf kernel):

likelihood  = GPy.likelihoods.Bernoulli()
laplace_inf = GPy.inference.latent_function_inference.Laplace()
kernel      = GPy.kern.PjkRbf(X.shape[1])
m = GPy.core.GP(X, Y, kernel=kernel, likelihood=likelihood, inference_method=laplace_inf)

Now I am trying to run the same set up but as multi-output. This is something I have not been able to do.

I have tried using the GPCoregionalizedRegression class with an ICM kernel, as shown in the code below:

likelihood1  = GPy.likelihoods.Bernoulli()
likelihood2 = GPy.likelihoods.Bernoulli()
laplace_inf = GPy.inference.latent_function_inference.Laplace()
K = GPy.kern.PjkRbf(X.shape[1])
icm = GPy.util.multioutput.ICM(input_dim=2,num_outputs=2,kernel=K)
m = GPy.models.GPCoregionalizedRegression([X,X],[Y1,Y2],
                                          kernel=icm, 
                                          likelihoods_list=[likelihood1, likelihood2])

Running this code throws an AssertionError, with a long stack-trace, but the last section shows the following. The likelihood cannot be asserted as a Gaussian.

~\Anaconda3\lib\site-packages\GPy\likelihoods\mixed_noise.py in gaussian_variance(self, Y_metadata)
     22 
     23     def gaussian_variance(self, Y_metadata):
---> 24         assert all([isinstance(l, Gaussian) for l in self.likelihoods_list])
     25         ind = Y_metadata['output_index'].flatten()
     26         variance = np.zeros(ind.size)

AssertionError: 

This is because I am not able to pass the Laplace inference to the GPCoregionalizedRegression model.

Can anyone offer advice on how I can resolve this, or if there is a different model I can use to perform multioutput regression with a Bernoulli likelihood and Laplace inference method?

Answer 1

The issue isn't that you can't provide Laplace inference to GPCoregionalizedRegression; it's that your GPy.likelihoods. Bernoulli chances aren't supported.
In the documents it is given that Most likelihood classes inherit directly from GPy.likelihoods.likelihood, although an intermediary class GPy.likelihoods.mixed_noise.MixedNoise is used by GPy.likelihoods.multioutput_likelihood.
The underlying likelihood is a GPy.likelihoods.mixed noise because you set up a multi-output problem. In GPy, the MixedNoise object only supports lists of GPy.likelihoods. Objects with a Gaussian distribution.
The source code for comparison:

class MixedNoise(Likelihood):
def __init__(self, likelihoods_list, name='mixed_noise'):
    #NOTE at the moment this likelihood only works for using a list of gaussians
    super(Likelihood, self).__init__(name=name)

As a result, the assert statement in \GPy\likelihoods\mixed noise.py, which tests that the likelihood_list items are Gaussian, fails, as shown by the stacktrace you provided.

As of right moment, this hasn't been implemented. I believe that the development of GPy is given less importance than that of its replacement, GPflow. There appear to be no constraints on the type of likelihoods in the appropriate functions (according to the code used in this example) (even if the example again uses only Gaussian likelihoods)