Attention Is All You Need
BookWikipedia
20
MENTIONS
18
EPISODES
18
PODCASTS
Search complete. 20 mentions across 18 episodes found for "Attention Is All You Need".
Nov 4, 2026
Marissa MayerGUEST
And when you start to think about the upcoming dawn of wearables, right, eventually what we're going to have is we're all going to have millions of frames a day.
Marissa MayerGUEST
But from that, you know, the Transformers paper headline, attention is all you need.
Marissa MayerGUEST
Your camera roll is literally what you've turned your attention towards.
Marissa MayerGUEST
In fact, you turn so much attention towards that you actually use that camera button to capture it.
Nico LaganHOST
Cohere has an LLM or a chatbot similar to ChatGPT, Gemini, Cloud, and Grok.
Nico LaganHOST
But one thing worth mentioning before I explain to you what the difference is between Cohere and the chatbot that you're used to, that you've heard of before, is that Aiden Gomez, their CEO... was 20 years old a 20 year old student when he co-authored with seven other students a paper a research paper by the name of attention is all you need doesn't sound very it doesn't sound like much but the matter that the point and the matter is that This research paper introduced the transformer architecture, basically what made LLMs like ChatGPT, Gemini, Claude, and Grok possible.
Nico LaganHOST
They laid out the model needed, the technical foundation for those chatbots.
Nico LaganHOST
So we're talking, the CEO, we're talking about a guy that wrote one of the most important, if not the most important research paper on AI when he was just 20 years old.
Chris KavanaghHOST
No, I have sympathy for this.
Chris KavanaghHOST
And Matt, before you respond, just let me add in that he is making a point that that paper, attention is all you need, that we read together, for example, a breakthrough in hindsight.
Chris KavanaghHOST
something that does seem quite simple, right? Even from the point of view of it being built on a lot of complexity, right? But as an insight, it doesn't seem like something that should have took that long to work out.
Chris KavanaghHOST
And yeah, very, very helpful.
Scott PhoenixHOST
That's right.
Scott PhoenixHOST
Yeah, reverse attention is all you need.
Scott PhoenixHOST
16 years into the bet, what has the leap changed his mind about? And what has he held on to? So how have your personal beliefs changed?
Dileep GeorgeGUEST
One, I never thought LLMs would be this good at coding.
Ram AhluwaliaHOST
The hard technologies are finding the next leap forward in these LLM technologies.
Ram AhluwaliaHOST
The great breakthrough was the white paper, attention is all you need.
Ram AhluwaliaHOST
And after that, you had some reinforcement learning that was introduced, which was an obvious innovation, and then you are throwing compute and data and trillion parameter models to make the LLMs better.
Ram AhluwaliaHOST
That's pretty much been the history of LLMs.
Lee McCabeGUEST
Just carry on moving fast.
Kyle MatthewsHOST
A lesson in that, because we'll get into AI is, and again, you, you probably know more than I do, but [clears throat] my understanding is Google, was it 2017, the white paper, uh, Attention Is All You Need, really was the one that discovered the GBT, you know, the General Pre-trained Transformer that led to language models.
Kyle MatthewsHOST
They wrote this white paper, right?
Lee McCabeGUEST
Yep.
Alexandros MarinosHOST
And speaking of scarcity here, an important element to know about character AI is that one of the founders was Noam Shazier.
Alexandros MarinosHOST
Noam Shazier was one of the authors of the Attention is All You Need paper from Google.
Alexandros MarinosHOST
So he was at Google.
Alexandros MarinosHOST
Okay, you got to keep track of the graph here.
Carl QuintanillaHOST
And joining us this morning in a first on CNBC interview is Aidan Gomez, the co-founder and CEO of Cohere, which he started after leaving Google Brain.
Carl QuintanillaHOST
At his time at Google, he co-authored the paper Attention Is All You Need, one of the foundational pieces for the AI industry.
Carl QuintanillaHOST
Uh, Aidan, it's so good to have you.
Carl QuintanillaHOST
Thanks for joining us.
Thomas FriedmanGUEST
Eight Google computer scientists, uh, produce what is... will probably go down certainly as one of the most, if not the most transformative, uh, innovations ever.
Thomas FriedmanGUEST
The paper was called Attention Is All You Need, and it's what created large language modules.
Thomas FriedmanGUEST
By the way, as an aside, seven of the eight engineers were born and the eighth was the son of a new immigrant.
Thomas FriedmanGUEST
Just saying.
8 more episodes mention Attention Is All You Need.
Create an account to see the whole feed, search across every transcript, and follow the entities you care about.