Perceptron

He was very, very skeptical of neural nets.

In fact, wrote a book called Perceptrons that killed off neural net research for a decade.

In this interview, he was saying, look, it's not that I'm prejudiced about it, but I tried these neural nets and I saw that they're not really working well.

I concluded that either they don't work or else you would need a neural net at some ridiculous scale, millions of times larger than what we're doing now.

But let's take a step back because I understand the engineering mindset.

And I think people are like, people who don't really know the history of artificial intelligence don't realize that what we have as artificial intelligence as of today, the transformer paper in 2017 on right is pretty much was witchdoctory or pseudoscience for all for for decades if you go back to alan turing in 1950 saying that someday you know machines would be able to talk in ways that would fool humans essentially the turing test but the 1956 dartmouth conference where the term artificial intelligence was coined right for i think It was 57 when the first neural network type of architecture in Perceptron, if I get that right.

Yeah, Frank Rosenblum.

I have to keep notes to remember all this crap.

because the

perceptron itself has such a dramatic history.

It traces back through Frank Rosenblatt's single-layer perceptron, then collapsed after Minsky and Papert's book Perceptrons exposed the limits of those systems on nonlinear problems.

That helped trigger the first AI winter, when funding and excitement just dried up.

And the revival came in the 1980s with backpropagation, but real momentum waited for more data and more compute.