Recurrent neural network

AI driven autonomy.

Machine and deep learning making decisions and adapting behavior in real time.

The jump that matters is the last one.

GPS waypoints made drones programmable.

Visual Inertial Odometry fuses camera and IMU data to hold an accurate position estimate when GPS drops out.

This is the machinery that lets a drone stay localized in a warehouse, a tunnel, or a jammed environment.

The eye and machine learning core, two families of learning do most of the work, deep learning for perception and understanding.

CNNs handle object detection and scene understanding.

Yo.

Parakeet is a family of ASR models from NVIDIA built on what's called a fast conformer encoder paired with one of three decoders: CTC, RNN-T, or TDT, which is token and duration transducer.

They're trained and shipped through NVIDIA NeMo, and the checkpoints live on Hugging Face under the NVIDIA namespace.

And the specific ones Daniel's using.

It sat at number one on the Hugging Face Open ASR leaderboard as of May twenty twenty-five.

Then V3 is the multilingual successor covering twenty-five European languages.

There are also CTC and RNN-T siblings at .6 and one point one billion parameters, a unified English model, a streaming multi-speaker model.

It's a real family, not a single checkpoint.

And there are fundamental reasons for why diffusion models are better than autoregressive models at inference time.

So even if you think about the story of autoregressive models, there was an inflection point in 2017 when people switched from RNNs to transformers, right? And what was that? The problem was that RNNs had to essentially process tokens sequentially, one at a time, and training was very slow.

And so people came up with this idea of let's have an architecture that allows you to process many tokens at the same time in parallel.

And that was the transformer.

You're spending most of your time moving around weights across the memory hierarchy, and you're doing very little arithmetic.

And that's a fundamental problem of autoregressive models.

And so what's the equivalent? If you think RNNs, transformers, Autoregressive models, the equivalent at inference time is a diffusion model.

Because a diffusion model is built to have at inference time a workload where you process many tokens at the same time.

And, and do you think TRM can be a good solution for, uh, continual learning?

RNN, yeah, or linear attention, this kind of technique could be possible, but we need different training paradigm, I think, uh, rather than, uh... 'Cause we're just feeding a ton of random data, a ton of random shit to, to them, so it's not, uh, it's not conductive to long-term, uh, memorization and, and experience.

And do you think so, like, right, like, everyone are talking about, like, test-time compute, right? And kind of like TRM, the test-time compute with just tiny models instead of like, of a giant one.

So why you think like the, the, the field, like, default is still the giant version?

The second thing it heard was the suitcase, and then it had it.

Well, with RNN, it would be very easy to determine as the

LLM that you're talking about the

trophy.

when you get to it because that's kind of the the order of things with uh and you know humans know we we have context clues to know what we're talking about obviously the trophy is what's going to fit in the suitcase with uh with the transformer and attention we're able to um to read all that in parallel and assign kind of strings between each token.

And from that we can determine where the transformer, the LLM determines that what we're actually talking about there is the suitcase being big enough for the trophy to fit in.

So all that, you know, all, all this incredible jump forward is mostly just because we are able to, instead of doing RNNs, which is, you know, one token at a time and sequential, We're able to do it all in parallel across, you know, now.

And now we're, you know, we're training all of the human knowledge into, you know, a trillion parameter models.

So they weren't actually interested in attention mechanisms whatsoever to start with.

So they were replacing the RNNs with ConvNets, and then they were doing a machine translation task and trying to get good behavior, and hopefully at least as good, maybe a bit better behavior, uh, but better performance, um, than, than the existing, um, tasks.

And gradually, they accumulated this sort of cobbled-together pieces.

ConvNets were one of them.

We're talking like 2015, 2016.

Um, there was a, a lot of hullabaloo about RNNs, CNNs, neural networks-

Mm-hmm

... and the potential for computers to, to think like we do in a, in a, in a useful way.

He also argues that if something truly better appeared, research culture would eventually move.

And he makes the point that old approaches like RNNs weren't magically immune to hallucinations either.

Now, his work at Meta is really practical.

He's working on an EMG wristband for controlling AR glasses.

OK, break that down for us.

So first, they feed those raw neuron spikes into an RNN, a recurrent neural network.

Right.

Now, the RNN's only job is to estimate phones.

It is not looking for vocabulary or full words yet.

Just the sounds.

A completely normal sentence.

Exactly.

22.

Transformers are RNNs.

Fast Autoregressive Transformers with Linear Attention.

Catharopolis, Vias, Pappus and Floret.