Naive Bayes classifier

So let's look at some other Bayesian networks with parameter sharing.

So we already looked at the Naive Bayes before, but just to anchor it in this notation, we have some, let's say we have a genre and we have a movie review, and we have a Bayesian network which generates each word independently conditioned on the genre.

And so the joint distribution over everything is equal to probability under genre of Y times for each word, the probability of a particular word given Y.

So the parameters of this Bayesian network are the genre and P word.

So secondly, we looked at the maximum likelihood principle, which is this kind of high-minded principle that says maximize the likelihood of your data.

And we show that this is equal to this very a pragmatic and simple intuitive principle of counting and normalizing.

And this is the simplicity which makes Bayesian networks, especially Naive Bayes, still very practical, useful, and interpretable.

That's the end.

And already from here, you can see that there's lots of space to explore.

If I wanted to try naive Bayes or a support vector machine, I would just write a different model wrapper.

If I wanted to explore different feature functions, I would just write new functions of trees, uh, and that would be very quick in terms of evaluating.

And by default, this is going to evaluate on random train test splits that are drawn from the training data, um, and you could periodically test against the dev set to see how well you're doing in reality.

there's no end of these things

essentially.

If you go on to scikit-learn and just look like in the linear model package, that things like logistic regression or if you look at the support vector machines in there or the Naive Bayes models, they have dozens of things that are hyperparameters.

And you can see them codified in code because it says like, here are all the keyword arguments, they have some defaults, but that's just one of the many values that you could explore for each one of those things.

The importance of generative models, you know, let's say arguably dwarfs the discriminative from the point of view of excitement, maybe not of utility yet.

And then we'll talk about Naive Bayes, which is, like, one of the simplest generative models.

Um, and we'll do, like, a spam filter because I think there's still some, like, law that we have to teach you how to build a spam filter.

All right.

And if something magical happens is we can set the derivatives to zero and solve it and get a closed form solution.

Okay? This isn't necessary for Gaussian models, but GDA does have this nice feature, and that is a feature worth pointing out because often people would use these generative models in situations when they wanted to compute something relatively cheaply.

And we'll see probably the, the archetype of this, something called Naive Bayes at the end of this lecture, which, like, is a very good first model.

It's a count-based model.

So I looked into that, and I said that, "Why don't you just use...

There's a Naive Bayes algorithm." Naive Bayes is a famous algorithm that can be used.

And so they said, "Can we use that?" I said, "It's a very simple chat.

It's...

Just try to use it." So they used Naive Bayes, and it was faster, it was cheaper, and the cost was zero because we own the algorithm, right? So the cost from thousands of dollars, it reduced to zero.

And to my surprise, the results were better.

The confusion matrix was better.