Skip to main content
Mingyu Liu

Mingyu Liu

Vice President of the Cosmos Lab at NVIDIA and IEEE Fellow, leading world foundation model research for Physical AI.

Oct 1, 2026

1:56
I'm wondering, Mingyu, if you could help us understand, maybe from your perspective, from Nvidia's perspective, why in today's, uh, world, uh, is open, i- is the idea of having open models and promoting open models, why is that such a critical thing in the ecosystem?
2:15
Yes.
2:16
Uh, okay.
2:17
So, uh, when I was a student, uh, I learned a lot about computer vision and machine learning.
2:24
At that time, there are great researchers, uh, students open source, uh, their code and model.
2:32
And looking at their code and model actually helped me understand the concept and helped me to innovate, right? And, and th-this, the academy always worked like that, uh, and had been years, you know, even before GPT, and GPT also have open source model in the past.
2:51
A-and that had been, um, you know, push the field forward, uh, around innovation.

26 MINS LATER

29:06
If we go all the way back to kind of one of those concrete examples like the robot or the, uh, manufacturing case or the automotive case, where does the world model come into play as you're creating the models that are actually, you know, driving decision-making or movement or actions in the, in the robot, in the physical world? Where does that fit in versus some other things like you mentioned, uh, policy, you know, models, et cetera?
5:51
World models are a different game, so can you introduce for us what are world models, how do we get to this point with world models, and what is Cosmos like?
6:01
People have some model of the world before they have language.
6:09
I have a one-year-old, so I know this.
6:16
Language is very nice.
6:18
It's symbolic, it's semantic, it's a document how people think about everything.
6:27
So when GPT, I think it's the first major AI breakthrough, it makes sense it happened in the language domain.
6:36
So enormous amount of information there encapsulates and learn from how human deal with things and then can start have chat with you, now can help you with the code generation review, right? So one model is from a different perspective, right? So people build one model, try to accomplish different tasks.

27 MINS LATER

33:37
It already is on the software side.
13:49
'Cause, you know, in reinforcement learning, we have the sim to real gap, and could you have a situation where the policy might learn to exploit features inside the simulator?
13:58
Yeah.
13:58
It, it's possible.
14:00
The way I l-look at how, uh, this, uh, neural simulator, uh, progress is that, uh, I think it will be first, um, it will be more useful for policy verification in the beginning.
14:13
Uh, policy verification is a task that, uh, you, you know, i-if you're a model builder, uh, any model running, you're gonna have, uh, a lot of checkpoints.
14:22
And you might do a version study, and you're gonna... every, you know, variance, you're gonna have many checkpoints.
14:28
And how do you know which one is better, right? Uh, uh, ideal case, uh, is to deploy each, uh, policy in your real robot, in your real environment, and ch-measure the complete rates, right? Uh, success rates.

5 MINS LATER

19:45
And can you tell me about Cosmos Dreams?

We value your privacy

We use cookies to understand how you use our platform and to improve your experience. Click “Accept All” to consent, or “Decline non-essential” to opt out of non-essential cookies. Read our Privacy Policy.