Simon Mo
3
APPEARANCES
2
PODCASTS
012
DEC 30
JAN 6
JAN 13
JAN 20
JAN 27
FEB 3
FEB 10
FEB 17
FEB 24
MAR 3
MAR 10
MAR 17
MAR 24
MAR 31
APR 7
APR 14
APR 21
APR 28
MAY 5
MAY 12
MAY 19
MAY 26
JUN 2
JUN 9
JUN 16
JUN 23
JUN 30
JUL 7
JUL 14
JUL 21
JUL 28
AUG 4
AUG 11
AUG 18
AUG 25
SEP 1
SEP 8
SEP 15
SEP 22
SEP 29
OCT 6
OCT 13
OCT 20
OCT 27
NOV 3
NOV 10
NOV 17
NOV 24
DEC 1
DEC 8
DEC 15
DEC 22
DEC 29
JAN 5
JAN 12
JAN 19
JAN 26
FEB 2
FEB 9
FEB 16
FEB 23
MAR 2
MAR 9
MAR 16
MAR 23
MAR 30
APR 6
APR 13
APR 20
APR 27
MAY 4
MAY 11
MAY 18
MAY 25
JUN 1
JUN 8
JUN 15
JUN 22
JUN 29
JUL 6
JUL 13
JUL 20
JUL 27
AUG 3
AUG 10
AUG 17
AUG 24
AUG 31
SEP 7
SEP 14
SEP 21
SEP 28
OCT 5
OCT 12
OCT 19
OCT 26
NOV 2
NOV 9
NOV 16
Aug 6, 2026
Inside vLLM: The Engine Powering Open-Source AI
16:33
16:52
17:10
17:17
E
16:24Elena BurgerHOST
So in those cases, kind of what is the point of running them, and kind of what do we learn architecturally in the, in the course of running them?

Simon MoGUEST
So first, on cost, it's not necessarily they are as expensive as a proprietary model, but rather, first, the cost, cost discourse has been discussed over and over again with even Geo and five point two a few months back.

Simon MoGUEST
And but for this model, there's a big sort of s-step change where we're bridging almost a ten x gap, but strike somewhere in the middle, where Kimi K three is not as expensive as Claude or GPD Sol, but it is a lot more expensive than Geo and five point two.

Simon MoGUEST
Why is that? And I do believe this is the point of we're pricing intelligence with the market correctly and understanding where it is.

Simon MoGUEST
And but then the majority part of the discourse should be focusing on, wow, this model is bringing a Opus four point eight level model to our own infrastructure that I can use, I can run, I can fine-tune, I can be able to understand exactly how many tokens do I need, understand the exact performance profile.
21 MINS LATER
How Open-Source AI Became Critical Infrastructure
16:33
16:52
17:19
17:37
E
16:24Elena BurgerHOST
So in those cases, kind of what is the point of running them and kind of what do we learn architecturally in the, in the course of running them?

Simon MoGUEST
So first on cost, it's not necessarily they are as expensive as a proprietary model, but rather, first the cost, cost discourse has been discussed over and over again with even Geo 5.2 a few months back.

Simon MoGUEST
And but for this model, there's a big sort of s-step change where we're bridging almost a ten x gap but strike somewhere in the middle, where Kimi 3K is not as expensive as Claude or GPD Sol, but it is a lot more expensive than Geo 5.2. Why is that? And I do believe this is the point of we're pricing intelligence with the market correctly and understanding where it is and...

Simon MoGUEST
But then the majority part of the discourse should be focusing on, wow, this model is bringing a Opus 4.8 level model to our own infrastructure that I can use, I can run, I can fine-tune, I can be able to understand exactly how many tokens do I need, understand the exact performance profile.

Simon MoGUEST
The reason here, for example, is for proprietary models, there is a regular mode and fast mode.
21 MINS LATER
Inferact: Building the Infrastructure That Runs Modern AI
28:40
28:45
28:53
29:02
29:22
29:31

Matt BornsteinHOST
Do you think this means more, there will be more state managed in the inference layer over time?

Simon MoGUEST
As before, the paradigm has been texting, text out, and then just single request response.

Simon MoGUEST
But as we evolve into the year and the decade of agents, we're seeing multi-turn conversation turning into... hundreds and thousands of terms.

Simon MoGUEST
And then these terms also involves external tool use, like interacting with sandbox, performing web searches, running Python script or any programming languages, and be able to have this kind of long iterative process where LLM is involved, but also external environment interaction is involved.

Simon MoGUEST
And this really kicked off a huge wave of co-optimizing a Genentech architecture with inference architecture.

Simon MoGUEST
So just to give an example, it is very important for VLM to understand whether or not the conversation is still happening.
10 MINS LATER
