MapReduce
Computer programWikipedia
34
MENTIONS
14
EPISODES
14
PODCASTS
Search complete. 34 mentions across 14 episodes found for "MapReduce".
Oct 19, 2026
speaker_0HOST
Okay.
speaker_1HOST
If you write a traditional MapReduce style script to process 100 files through a three-step process, audit, verify, report, the script will typically wait for all 100 files to finish the audit phase.
speaker_0HOST
Like an artificial barrier.
speaker_1HOST
Exactly.
Stephen IbarakiHOST
fellows, Sanjay and Jeff, and they both left, right? And in fact, they even saved Google in the year 2000, right, when Google's search engine had a problem.
Stephen IbarakiHOST
And every major kind of technical achievement, including MapReduce, They left.
Yan SolihinGUEST
Yeah.
Stephen IbarakiHOST
So, you know, we can say in the interim, we have these sort of goals we want from an education standpoint.
Sam NewmanGUEST
And so literally, it-- when a computer g- fails, the software doesn't really care.
Sam NewmanGUEST
I mean, this is way back in the days of MapReduce, right? You'd slide a tray out.
Sam NewmanGUEST
You'd find, okay, that's the hard drive.
Sam NewmanGUEST
You'd unzip the Velcro, throw it in a bin, put another hard drive in, tape on the Velcro, put it in.
John PlattGUEST
It could be a very data-heavy problem because people at Google are very, very good at large data processing.
John PlattGUEST
I mean, dating back to MapReduce and all of the large-scale computing that we have.
John PlattGUEST
And now we have lots of TPUs and AI.
John PlattGUEST
So if there's problems that we think are... scientific problems where AI can do very, very well, then we try to attack those.
Chris GambillGUEST
Well, now we have AI.
Chris GambillGUEST
So it makes those things all of a sudden possible and kind of in the window of opportunity again, which increases that mountain of work, right? That's what big data did, right? That's what Hadoop and MapReduce did, right? It increased the mountain of work.
Chris GambillGUEST
It never decreased the mountain of work.
Chris GambillGUEST
That's what cloud compute did.
Brooke HopkinsGUEST
So I like to kind of take a similar task and break it down into like paralyzed.
Brooke HopkinsGUEST
It's like MapReduce, but for humans, it's like break it down into a bunch of parallel workflows that you're like still working on the same thing, but it's doing different pieces.
Brooke HopkinsGUEST
And so you're kind of like checking in on different pieces of it.
Amir BormandHOST
Absolutely.
speaker_4HOST
If you look back, we used to rely on these incredibly clunky JVM-based Hadoop tools to do the heavy data lifting.
speaker_3HOST
And Hadoop relied on MapReduce.
speaker_4HOST
Exactly.
speaker_3HOST
Yeah.
David CrawshawGUEST
I, I wanted to do something with lots of computers or very large computers because all of my work had been basically solo open source stuff, which meant my universe was constrained to the PC or the Mac or some machine on that scale.
David CrawshawGUEST
And, uh, I had recently been doing a data analysis project with my now father-in-law, who's a professor at Haas, and, you know, I'd, I'd built this thing because the data set didn't fit in one machine, and so we found 10 machines somewhere at the university, and I, I tried to build something across them and, you know, hacked up some very crude MapReduce-looking thing-
Matt KleinHOST
Yeah
David CrawshawGUEST
... based on what I'd read about MapReduce.
Michael SlaterGUEST
Right.
Michael SlaterGUEST
And then we had Spark and MapReduce was the technical term of that day.
Michael SlaterGUEST
That was the AI of that day.
David O'HaraHOST
Yeah.
4 more episodes mention MapReduce.
Create an account to see the whole feed, search across every transcript, and follow the entities you care about.