Expectation–maximization algorithm

In this module, I'm going to talk about the

EM algorithm for

learning

Bayesian networks when we have unobserved variables in our training data.

So what can we do in this case? Intuitively, it seems kind of hopeless.

How can we learn a Bayesian network relating G and R when we don't even see examples of G? But we'll show that this is actually possible in many cases, but certainly not all cases.

And that's kind of the magic of EM and unsupervised learning in general.

So let's try to approach this problem top down.

Today, we're going to continue in unsupervised learning.

We're going to see the details of the EM algorithm.

And then we're going to transition to its non-probabilistic analog, PCA.

And PCA is a workhorse algorithm.

And so it is kind of a workhorse canonical algorithm that you should have in your toolkit.

OK.

So we'll first discover this EM that discovers our ad hoc algorithm that we did before.

This GMM algorithm, we're gonna explain how this is a principled element of what we're doing.