Generative Recursive Reasoning

This paper is called Generative Recursive Reasoning.

This is out of Quebec AI Institute, NYU, University of Montreal, and KIST, which I'm not familiar with.

But the paper caught my interest mainly because it's a fairly novel approach, at least from what I can tell.

Novel to me, at least.

I haven't heard a lot about this approach.

There is cited prior research, but not prevalent as far as I can tell.

So there's sort of like a developing research thread along this trajectory, but it's still in its infancy, I would say.

It's a new generative type of model.

And specifically, this paper adds a variant on this new type of generative model and makes it stochastic.

And so it becomes non-deterministic, sort of like an LLM where you're sampling logits versus some kind of deterministic output, broadly speaking.

So I will talk through, like usual, I'll talk through each section of the paper.

So let's just start off with the abstract.

So the abstract, I'll summarize a bit.

So recursive reasoning models offer promising alternative to autoregressive sequence extension by performing iterative latent state refinement with shared transition functions.

So this, like I mentioned, is sort of an alternative to existing generative models.

The applications in this paper aren't language specifically.

They use things like Sudoku and Chess and other things like that, but could be presumably extended to language modeling tasks.

It sort of functions like a kind of diffusion unit architecture type of thing, but also has callbacks to like an RNN recurrence mechanism.

So they mentioned Graham models, generative recursive reasoning models.

So that's the title of the paper.