Cloud API

It also has a merchant agent for people running the store, doing sales analysis, inventory flags, and promo recommendations, with a human approving anything before it goes live.

It deploys on the Cloud API, Amazon Bedrock, Microsoft Foundry, or Google Vertex.

There's also a Cloud Code plugin.

Wix says its engineers had an agent taking prompts in 15 minutes.

And as I understand it, that again is about self-hosting.

When you say LLMs in production, you don't mean calling the OpenAI, the Cloud API.

You mean doing it yourself.

So that's what we're here.

You know, if you're...

So our current stack is we target with Clay, we enrich with a Cloud API integrated with Clay, use...

NE10 to kind of transfer certain data onto a Google sheet where we have a different cloud API and Phantom Buster to scrape information from that prospect's LinkedIn.

And then the API uses our own SOPs that we've sort of made with the client to actually write the message.

And then there's a quality assurance aspect layered on top of that.

And then what they do next is they go to production with something finally after... six, 12, 18 months.

They go to production with something and then they go, wow, what if the Gemini API is down? What if the Cloud API is down? We should fail over to something else.

So then they have two frontier models in production.

And that lasts for a while until they get traction and they go, wow, we're spending $50,000 a month on inference.

So that is a lot of companies, and now you need to search for all of them and then pick up all the news from everywhere about all these companies and kind of aggregate them and all that stuff.

So because this is all Gentic, it's actually using Cloud API to run some of these things in search of API through Brave and other pieces.

The costs start adding up, and...

I got to a point where, yeah, 280 bucks a day, I got to start optimizing.

Everything is possible, as long as you define it and think through it.

I was wondering if that expense you had also includes the Cloud API.

Because I find myself, I might spend like five pence or cents calling Cloud API with Sonnet.

Just curious if you're using that at all.

No, I don't use.

About seventy percent of my agent's work runs on the local three B model.

The other thirty percent goes to a cloud API.

The split is not random.

It's based on which task types each model handles well.

But the bigger reason is latency.

A local model on the Pi responds in two to five seconds.

The cloud API takes eight to fifteen seconds, sometimes longer during peak hours.

When the agent is executing a ten-step deployment checklist, ten cloud calls at twelve seconds each is two minutes.