Neural Engine

Um, uh, first of all, I wanna mention that our show notes can be found at macvoices.com/show notes.

The very first one tonight, um, points out that the 512 gig Mac, uh, iPad Mini gets an extra CPU core and dual neu- neural engine.

Right.

Um, and this has been a bit of controversy over the last, what, 24 to 36 hours, that you had to buy the top-end iPad Mini to get a n- a not a binned chip.

So I'm not sure that that restriction or that is a big issue, but that's me.

That's the way I think of using a Mini.

Does anybody else feel like this would be a serious horsepower machine that they would need the, the, the dual neural engine and the extra CPU core?

I really don't.

So you have to spend at least $900.

to get the nano option if you get the 512 gigabyte option which is the most it doesn't come with it doesn't come in a terabyte yet or at all it never has it and it has it does it now so but the 512 which costs a thousand dollars uh 1100 if you get the nano thing you do get a better chip I don't know why it matters, but the 128 and 256 have a chip with a five core CPU and a single 16 core neural engine.

The 512 gigabyte model has a six core CPU and a dual 16 core neural engine for whatever that's worth.

So probably good.

128 gigs is pretty small these days.

Like in the iPhone Air, the A19 Pro had one less GPU core and stuff.

So we've seen that.

But it's interesting that they're also doing that thing where instead of the doubled up neural engine, you've only got the normal 16 core, the same as it has had.

The one more is CoreML.

There's an open source Swift package called Fluid Audio, and it already runs stock Parakeet TDT on the neural engine, the chip in the phone built for this.

It needs iOS 17 or later.

You're doing the voice again.

A Japanese parakeet conversion reportedly fell back to the main processor on an A15 chip, roughly an order of magnitude slower.

Fell back? Quietly, I assume? Quietly.

Some operations don't map onto the neural engine, so the phone runs them on the general processor and tells nobody.

That's the scariest thing you've said all week, and you read papers for fun.

So Apple and Parakeet, those two models, do almost all of their work on the CPU.

They're using the neural engine, but they're not using that for most of the things that they're doing.

Most of that work has been done on the CPU, and notably, both of them are single-threaded jobs.

So they will max out a single CPU core, typically whatever your fastest core in that device is, but they don't scale with more cores.

The M4 Pro or Macs, your transcription with Apple's model and Parakeet's model as well is going to be largely the same no matter which one you get.

Yeah.

Meanwhile, Whisper, Whisper is even more interesting because Whisper does most of its work on the neural engine with just a little bit of CPU usage there.

So Whisper is almost entirely locked to how good the neural engine is in your Mac.