Season 3: Episode 13 - CTO Confessions: How to Survive a Million Line Codebase Build at 20x Velocity

What have you got for us in the news this week, Spud Pete?

Well, I've got quite a lot in the news.

And actually, to your point of what we're likely to hear from Greg later in the episode.

So the BBC have put out a little bit of advice on making the most of tokenomics.

So obviously, this is a topic that we've discussed fairly extensively.

You know, we've been going back and forth for what feels like forever about rising costs in terms of AI use trends in kind of moving towards token maximization and sort of pivoting and token minimization.

But I think, and I don't know, David, if you agree with this, a lot of this has been, for me, quite a theoretical argument.

And this is one of the first stories I've seen come out with some actual practical advice that tech leaders can take into work on Monday morning and say, this is how we are going to get the best ROI from our use of AI.

So, I mean, obviously, there's been movement.

We kind of initially had some subscription-based pricing.

We moved to token-based payment.

We've seen costs go down, but we have seen token use go up.

We've seen some firms who started to introduce kind of token leaderboards to really encourage people to embrace the use of AI.

But I think we've seen a little bit of a rebound on that front.

Interesting stat from Goldman Sachs.

They think that token use will increase by 24-fold by the time we get to 2030, which is trillions and trillions of tokens.

But of course, because the models themselves and the way that tokens are used is non-deterministic, that therefore means that the credit cost or the financial cost itself is non-deterministic.

So I have got, well, I'm going to relay for you six tips from the BBC in terms of what you can do to maximize your kind of financial positioning when it comes to using AI.

So the first is to avoid tying your customer into a cost model for a long period, whether that's six months, 12 months, a couple of years.

Because what this means is that you can lose out if you're taking a kind of a standard fixed payment from your customer, but actually costs associated with tokens and credits start to rise, which I think everyone is predicting that's something that's going to be happening.

Or your customer will start to be dissatisfied if costs drop and they start to see, well, actually your competitors are charging quite a bit less for the same service, but I'm locked in on this price and I'm very unhappy about that.

And I think another thing that's really relevant here is in your kind of design of your product is not to have a sort of uncapped token payment model at your end, but a fixed subscription flat fee at the customer.

And so the best example, and David, I know you've spent some time in the health world, so you'll have seen this too, is these, you know, a health chat bot and it promises 24-7 access and ask as many questions as you need.

I haven't seen any examples of usage-based payments for any of those, but of course, it'll be a usage-based payment at the provider side.

Yeah, of course.