How would you go about making an eval for this problem of continual learning?

I think the basic structure would be that there is a context that the model can learn something about.

And you give the model...

You ask the model to work in that context for a long time, whatever that means.

And then over time, you expect it to accumulate some kind of learning, which allows its performance on future tasks in the same context to get better.

And there is a... of how does the model accumulate more learning.

An eval should probably be agnostic about that.

Do you know, like, why this