Skip to main content
Aden Barton

Aden Barton

Oct 19, 2026

speaker_1HOST
1:50
Any interesting results you can share with us?
1:53
Yeah.
1:54
So the motivation of the study was we see benchmark scores going up every day, and it's very hard to know what that actually means.
1:59
What does a 15% on APEX mean? And so one way to answer that is to give those tasks to humans, because at least you know, well, maybe Claude outperforms a human or underperforms a human.
2:10
And so we took tasks from APEX Accounting, which is a benchmark we made with RAMP, and we gave those tasks, simplified versions of those tasks, to real accountants, as you mentioned.
2:19
And we saw kind of two big things.
2:22
Just in level terms, models are really, really good today.
speaker_1HOST
5:09
And so it just ceases to actually be a good benchmark.

We value your privacy

We use cookies to understand how you use our platform and to improve your experience. Click “Accept All” to consent, or “Decline non-essential” to opt out of non-essential cookies. Read our Privacy Policy.