what made this challenge like inference optimization what made it hard for you guys in the very beginning like what were some of those earlier hurdles that you had to try to overcome

Yeah, sure.

So I think one of the biggest challenges, as the old saying, everything is different when 10 times of the scale.

So 10 times is different.

So that means when we actually serve a small number of requests, And then we just go there, grab a bunch of GPU and then run the model.

We don't care too much about, you know, whether the GPU is fully utilized because, you know, even we optimize 50% is not a big saving.

And also like for a large scale, there are a lot of corner cases.

From here, looking forward, where are we going in this realm of either inference optimization or inference serving, the general inference category as a whole?