Evidence lower bound

It's not learnable.

That's the usual ELBO objective.

And recall that these decoders are parameterized, are all Gaussians.

And they have this simple form, where to sample xt minus 1, you would take xt, you pass it through some neural network to get the mean, and then they will have fixed variance.

But kind of like, if you parameterize the network in terms of an absolute network that takes Xt as an input.

And then tries to predict the noise that was added to xt and subtracts it from, subtracts this estimated noise from xt to get a guess of what xt minus 1 should be.

Then you can show that this ELBO objective is actually equivalent to the usual denoising score matching loss.

So, Minimizing the negative elbow or maximizing the lower bound on the average plug likelihood is actually the same as trying to estimate the scores of these noise perturbed data distributions.