Semantic Coupling & the Interconnected AI Stack with Eugene Wu, Professor at Columbia University

point.Yeah, I think for me, the interesting thing about this moment is that there's, um, a lot of, to your point, blurring between the lines, and so systems thinking is really critical because effectively you have to look at every part of the system to guarantee quality of the output.

And what's hard, I think, for many people that are either classically trained in programming or, or, or, you know, that have sort of thought about architecture is that you always think about these stacks, you think about layers, and you think about, like, discrete handoffs.

And it-- you know, you really-- that doesn't actually-- that, that construct almost doesn't even make sense because in many cases you sort of can't define the quality of one layer without understanding the outcome or the output of the layer above and what happens underneath.

And so, so these layers bleed, and that bleed is not something that people either care...

Like, you know, people want to just do the job and then hand it off to the next person, and that kind of thinking just does not work.

Have you seen people deal with sort of that fundamental intellectual challenge and, and do people, uh, you know, I, I...

Certainly in industry, how, how do architects, classically trained architects think about and respond to some of your work?

Yeah.

I think it's a fairly new way of thinking about things, right? Because like you said, like, a lot of us, you know, are trained in a particular kind of area, and that's what we kind of traditionally focus on.

It's very difficult because, you know, what LLMs are really good at is, like, learning expertise in one area and then transferring it over to a completely different domain, right? And so that's something that, you know, LLMs and kinda, like, AI is very good at, but it's not something that kinda, like, a person might be very good at, right? And so I feel like for a variety of different reasons, it could be kind of, like, organizational kinda, like, reasons or it could be kind of existing legacy technology reasons.

Like, typically kind of what we've found is that people are kinda, like, reconfiguring existing systems, but trying to, like, do more in the harness or in the agent workflow structure in order to kinda, like, deal with the lack of reliability in the underlying environment, right? So that's generally what we see, and it kind of makes sense because, y-you know, like, o-one example, right? You guys, you know, like, Alation is doing kind of, like, like, managing, like, huge amounts of kinda, like, data and, and, and cataloging and so on, right? Like, if the quality of the metadata is not very good, then you can't really trust the search results.

And if you can't trust the search results, then any application you build on top of it needs to expose this information to the user, right? And the user has to make a judgment call.

And that places a huge cognitive load on the end user trying to use this tool, and then they have to then make a decision, like, is it worth it or not, right? And, and so, like, that's kind of one example where, like, something very, very, you know, low in kinda, like, the stack, just indexing, can permeate all the way up to the end user, right, and their experience and what they can expect.

Yeah.

And there's lots of different ways to solve that problem.

I mean, we-

Right

... we've, we've been known as a catalog, and what we realized is that-- and, and so our view on this is that agents ultimately, you know, we-- when people use these things like catalogs and-

Yeah

... and the like, they're doing it, you know, either for the purpose of trust or they're doing it for the purpose of discovery at the, at the end.

And, and what we, what we've come to believe is that basically whether you're building trust by describing or sort of somehow doing something with the data that allows you to sort of tag it or, or improve it or, or make it better described or you're consuming the data, you're gonna do that, in our view, using agents because, you know, like many problems, it's just you don't have enough resources in order to sort of analyze this stuff or, or improve it over, over time.

And so what we, what we came to realize was that, you know, the catalog is sort of a thing.

And to your point, whatever is there is there.

It's like, you know, if it's, if it's wrong, then agents or when people are gonna both, like, take the information and conclude wrong things.

So we ended up building sort of this abstraction which we, which we're calling sort of a data product alongside these ontologies which, you know, are basically what agents use then in order to be able to, to sort of do their work.