Claude Elwood Shannon

As long as you can power your computation, information is power.

Claude Shannon, the father of information theory, was deeply inspired by the close connection between entropy and information.

He defined a new type of entropy, Shannon entropy.

It can be interpreted as the amount of previously hidden information that can potentially be revealed by the observation of a random event.

In a sense, it's a measure of the uncertainty of the event.

So for example, the roll of a die has more entropy than the flip of a coin.

There are more potential outcomes in the former, so more uncertainty has been eliminated when the die is cast.

Shannon used the term entropy for this measure because his formula was almost identical to the formula for thermodynamic entropy.

In the 1940s, McCullough and Pitts introduced the first mathematical neural network model in 1943.

Claude Shannon's 1948 information theory paper gave probabilistic language tools, and early computers like ENIAC provided the hardware that made AI research possible.

Those early ideas were just the beginning.

Want to hear how they evolved into the AI we know today?

Yeah.

So everyone was aware of, I think it was Ed Thorke and Claude Shannon back in the 1960s.

Ah, yeah.

Developing this computer that allowed them to.

And, uh, the book is great because, um, you know, the art's good, but it's also just a fantastic story, and it's a way to learn about Kasparov because of course we go into his career, his life, his first match with Deep Blue in 1996, uh, and then we learn about the, uh, the creators of Deep Blue, the engineers, uh, where, where it came from.

Actually, just passing, in, in passing, I learned why Claude is named Claude.

I, I'd forgotten or never read th- that it's named after Claude Shannon, the, uh, one of the progenitors of computer science and intelligent machines.

Uh, so you learn about AI, um, but you also get this fantastic story about, uh, one of, uh, a truly remarkable, uh, individual, Garry Kasparov, and his fight, uh, with the super intelligence.

Relevant today, of course, but amidst all the doom, uh, saying for, for AI, but with figures like Kasparov, you get the sense that the human spirit as well as the human intellect are, are up to any, uh, challenge.

And not just any binary structures.

There is a real-world concept developed by theoretical physicist James Gates, who found equations in string theory that are indistinguishable from Claude Shannon's block codes.

Okay, what are Claude Shannon's block codes?

They are error-correcting codes.

They are the exact same algorithms that human computer scientists use to prevent data corruption in web browsers and digital communications.

Right.

this gets into thermodynamics this gets into entropy and like information and so then i said okay well is it just people is this a socio thing or is this like information theoretical and like all things are bound by um you know information processing And then I discovered that there's actually Shannon, Claude Shannon, who worked at IBM and was a scientist and developed Shannon entropy.

And that's like used in a ton of communications.

And then you have Landauer and Landauer's principle and that to change or erase a bit of data costs like an actual amount of like thermodynamic energy to do.