Skip to main content

Pete Hunt

Sep 23, 2026

20:54
Let's talk a little bit of evolution, modern data stack, and talk about tooling, like how you've seen that evolve, and then maybe where you see it headed.
21:01
Sure, yeah.
21:02
I mean, I think, you know, we were talking about running 20-minute queries on Hive back in the day.
21:09
That was definitely the pre-modern data stack.
21:12
In many ways, big data was still a challenge, right? I would say that even in the high of the era, like big, like we had these big data tools, but big data was not a solved problem yet.
21:23
We couldn't like easily compute over like unbounded sets of data, you know, in a reasonable way or like effectively unbounded anyway.
21:31
And so I would say with the arrival of tools like that Peregrine thing that I told you about, but also like Snowflake, Databricks, BigQuery, you really got to like almost interactive query speeds, right? And to me, you know, right around the time that like BigQuery came out and I think the, you know, it's like the Google Dremel paper, right? Like that to me, it was when like big data kind of became a solved problem.

6 MINS LATER

28:07
So I think it'd be interesting to talk about even maybe specifically, maybe generally with orchestrators, but specifically with Dagster, like how do you see those coming together? And then on top of that, we've got this new component of the AI piece that also makes that a little bit more complicated as far as like what the roles might even look like.
6:53
I'm a big believer in like, and maybe this is like a little
6:56
old school, but I think that you let the people use the tools they want to use and do work the way they want to do the work.
7:03
You got to have some standardization so people can work together, obviously.
7:06
So it's not like completely chaotic.
7:09
For the most part, like if you didn't care what IDE people use, like why do you care like how they use AI? So like what we tried to do was tell people about it, tell them about the best practices.
7:21
And then what ended up happening was just like the pace.
7:24
Certain teams would just like take off and have like really fast pace and other teams would be slower.

16 MINS LATER

23:56
So what works for you when you're building that brand and that trust with your customers? What works and maybe even more importantly, what doesn't?

We value your privacy

We use cookies to understand how you use our platform and to improve your experience. Click “Accept All” to consent, or “Decline non-essential” to opt out of non-essential cookies. Read our Privacy Policy.