BigQuery

Welcome back to Dev Interrupted, brought to you by LinearB.

My guest today is Jordan Tagani, co-founder and CEO of Motherduck, and founding engineer on Google BigQuery.

He used to write more SQL than anyone at his company, and now, like so many of us, he can't remember the last time he wrote a query.

We covered Steve Ige's Gastown here on the show, and Jordan wrote the data version.

You know, you're the CEO and co-founder of a pretty cool company right now, in my opinion.

There's a lot of really innovative things coming out of Mother Duck.

And your background as a founding engineer at Google BigQuery is, you know, just like a driving force, I think, in all of this change in narrative.

And it makes so many folks look to you to understand like how data science is transforming.

So we were forced onto figuring out a long-term solution to data, to read data engineer what we had into something sustainable.

And that path was BigQuery.

So my focus for the last eight to 10 months have been to transition our, all the data that had been living on the sheet into a system.

And subsequently through that, those data in that system, how to bring those along with our workday data into Google BigQuery, into a place where we can do all that data transformation, do all those validations so that we have a real time source of truth that can be used for like enabling future agentic use cases and other things that we have in mind but um but that's for another podcast probably

yes there's you're definitely a treasure of content colette we could dig deeper and deeper but yes just wrapping this up what's that one check now that you run before you trust what the ai or whatever you build hands you based on your experience going back and forth to get the data foundations right what's that one good requirement

it probably sounds like something on the internet already but my one quick requirement is that just acknowledge that your requirement is probably stupid and it's probably too like it probably can be simplified the reason I say this is it's it's easy to it's easy to want to have a lot of different options and use cases and cover all the things but without doing the hard thing of just like cutting the requirement in half and thinking from a first principle way of what is really important, you will start just automating stupid things.

[whooshing] The last one from OpenAI is their new data agent for ChatGPT Work.

The agent is designed for handling proprietary data within an organization with connections to data providers like Amazon Redshift, Datadog, Google BigQuery, ClickHouse, Databricks, MongoDB, and Snowflake.

OpenAI says the features can be used to ingest sales data and generate insights around core metrics like sales conversions and retention.

The goal is to realize the promise of being able to talk to your data and perform real analysis without needing to touch additional tools.

And actually, I come from the experience of running some of these beefy systems.

I think Pedro mentioned MonetDB or Postgres, but also like the BigQuery, Databricks, Snowflakes, right? Which is like, let's say, a second generation kind of system where the experience is quite good, but the cost is quite expensive.

And, uh, I mean, one of the, the things actually that, uh, prompted me to, to join MDD was that, uh, Tag Lake was released actually.

And I, I listened to this podcast, uh, that announced Tag Lake, uh, with Hannes and Mark.