GCP AI Cost Optimization: Vertex AI Model Selection Guide

So for your experience in GCP and with AI, One of the factors is to select a good model is pricing, okay? So depending on the pricing, you select one model or the other based on your case study, you don't always need the more expensive, near the cheapest one.

So how do you think, how relevant is the pricing in the model selection?

I think it is very relevant.

Because, well, I think that we are currently in the beginning of the AI revolution.

And obviously many companies want to be as much as, you know, on top of things and implement AI models, AI agents in their products or services.

So what we are seeing now a lot is just people going over everything, all the models or testing the priciest models for all use cases and stuff like that.

And it is still very hard to break down all those costs.

We still see the AI spend like a black box that we really want to turn it into a white box and open it up to analyze it.

And, you know, we all hear of those stories of like a developer just prompting, just playing with one AI model.

And then in the end of the month, you get like a bill of thousands of dollars.

So this is where I think we came to a realization that we have to implement FinOps in AI, you know? Yeah.

Yeah, that's true, because I think we've overcome the phase of where everyone was experimenting and trying out new things, and you always wanted to try the latest model and everything.

But it was more, I think it changed when the AI availability was for everyone.

So before, and it's also a debate that I have in FinOps, it's like, The cloud, for example, just to make a similarity, is available for a certain side of the company, which is IT people, and especially IT people focusing

cloud.

But AI is available for everyone now.

That's a

very good point.

And then you need to control it to everyone.

So you need to apply FinOps as soon as you can, because cloud has been there for a while, but AI has been super short time, and then we are already handling token budgets, the token maxing thing, and all the quotas thing.

So it's super relevant.

And, you know, following up on this, we know that, okay, in the model selection, there are other factors that you have because you cannot always select based on price.