Data build tool

Sources for today's episode.

O'Reilly Radar, Katie Nuggets, The New Stack, Towards Data Science, MIT Sloan Management Review, DBT Labs.

Vendor.

Data tooling.

Some of our software stack, like, is very much pulled from the rest of the world.

Like, a lot of data engineers outside Jane Street use DBT, which stands for Data Build Tool.

Data Build Tool is, like, a tool for, like, describing and orchestrating transformations like in your data warehouse usually like using SQL queries but in like a code reviewable testable composable good software engineering practices way

sounds

great so I have seen 300 line postgres um views and functions uh in my time at chain street and we don't want any of that like dbt lets you break down the data transformation problem into smaller components and like test the properties of the data at each stage so that you can like rely on it