Vision-language-action model

And he's saying we got to do better as a field in 2026 and stop treating reproducibility and scientific discipline as second-class citizens.

And three, VLM-based VLA fields are wrong.

This I thought was quite interesting.

So we've seen a few of these VLAs, VLMs announced, right? So VLA stands for vision and language action.

So similar how we have LLMs, large language model, we can also have VLMs, a vision language model, and VLA is a vision language action model, right? So you take a pre-trained VLM checkpoint and graft an action model on top.

Google has been rolling out a few of these.

They look really impressive so far, as far as I can tell.

And two, visual encoders are actively tuned to discard low level details because question answers only require high level understanding.

It's one of the key use cases we foresee to enhance also the user experience and also provide some opportunities to the operators to provide compute as a service.

Because when you deal with the robotics and teleoperation, you need to use VLA models and the VLA's involve and imply split compute.

between the robot, that is an intelligence device, actually, an intelligent device.

And in order, you need to split the compute and offload to the cloud.

These days, you can do it for videos, for robotics applications.

Actually, most of VLA, the Vision Language Action Model, are using diffusion models for generating the actions.

So they are kind of like quite different from a typical large language model to some degree.

And the training of the models requires some version of inference or the EM algorithm.

It writes itself a set of small tasks inside that copy, writes the program that moves a simulated arm through each one, checks that every run succeeded, and exports thousands of practice runs.

Those go into fine tuning an ordinary VLA, a vision language action model, and on the real table, its success went from roughly half to about three quarters.

I mean, in that example, the frontier model makes the training data.

and an ordinary network does the driving.

Obviously, we have a different approach.

Our, our belief is that while VLAs are very helpful, they too have a ceiling.

You're working with a lot of these existing customers in a lot of cases with off-the-shelf hardware.

So why ultimately is the company developing its own hardware?

Um, and, and, and that's, that's, you know, been a, a big part of the progress over the last few years.

Obviously, we have a different approach.

Our, our belief is that while VLAs are very helpful and they've shown results that, you know, uh, perhaps go beyond what traditional robotics can do, uh, that they too have a ceiling.

And we think that ceiling comes from the fact that, that there's just not enough teleop data that you can collect, and, and more importantly, that you can't get enough diversity in that data set to really learn a good, uh, general prior on how the world moves.

Most of the robot policies on this show are trained for the job.

A VLA, a Vision Language Action Model, takes a camera image and an instruction and puts out the next movement of the arm, and it learns that from thousands of hours of robots doing tasks.

A general model like the ones behind a chatbot has never seen any of that.

Then in September, an outside group ran OpenAI's GPT-6 Astra as a robot policy with no robot training at all, and it came out above every public robot policy on a simulated two-arm benchmark called RoboDojo.

And it's also widely available that you can query for like general image and video understanding and semantic understanding.

Now you can think of Gemini robotics as the VLA that is like controlling the actions model, right? How the robot moves and what it should do given a certain intention from either a user or another system tool model.

And you can think of the Gemini robotics on device model as a much smaller version of that that kind of fits on the robot's computer that doesn't live in the cloud and do It is comparable.

It's essentially doing what Gemini robotics can do, but it's a smaller model that fits on device.

Practically, you need to be more efficient with how you manage your RAMs, right? And you kind of want to have the most important information represented.

And sometimes that can take the form of also representing information in text, which is a little bit more compressible.

And this is also now related to the, like the hardness between like ER and the VLA, like the ER tries to parse the intentions from a human and via text and other modalities kind of talks to the VLA.

And so there, like the bandwidth there is much more interpretable modalities.

It operates on an omnidirectional base with a dual arm manipulator.

What makes it revolutionary is that it is powered by a vision language action or VLA foundation model.

Right.

Traditional robotics rely on rigid, pre-programmed spatial coordinates.

If an object is slightly out of place, the robot fails.

A VLA model maps high-level language concepts directly to visual data and physical actuation in real time.

And Taku is deployed in the hospitality and healthcare sectors, where it handles commercial laundry.

Imagine a hotel laundry room.

So the problem is from the observation of some small fine-tuning of the, like, models, like Pi Zero or Pi zero point five, we found that current generative robot policies are kind-- are kind of tightly coupled of the training-specific configurations, so which, which caused some failing to decouple the reusable motor skills from the normal task constraints.

Uh, like for example, here we have some cubes on the table, and if you want to stack some up to a tower with a spec numbers, you may call the Pi zero five pre-trained VLA policy to stack some cubes.

Uh, like i-if you want to stack two cubes and using the termina- uh, termination signal, like the pushing the red button on the table, it can do very well because you-- because for this specific task, you can fine-tune the base policy a bit, like stack two cubes and push the button or stack four cubes to push the button.

And, but what if at test time you ask the base policy to stack three cubes? It's never, uh, it's never contained in the fine-tuning data set, so the base policy can learn how to stack two cubes or stack four cubes and push the button, but it will never know how to st- how to stack three cubes and push the button.

So the objective is sta-static and also is very distribution dependent, which means after, after fine-tuning, your base policy can only capture the spatial and semantic correlations on the training manifold.

If you change the observation or language at test time, the base policy may confuse.

This observation is clear at your daily work because if you take one pre-trained, uh, VLA policy, uh, you, you just Don't want to, like, use it in the in-distribution tasks.

You may change the scenario, you may change the language to, to the VLA.