CMS Proposed Thresholds and Patient Reported Outcomes after Total Joint Arthroplasty

We'll now move into the headlines.

My headline is comparison of large language models with rules-based natural language processing algorithms for extracting data from orthopedic notes.

This study was done by Yang and colleagues.

It does come with a comment, so you don't have to take it from me.

You can hear what someone else thinks about this paper.

And it's also 30 days free, so no excuses.

Everyone should be able to access this regardless, at least for the next 30 days.

So this study is looking to compare two different approaches to extracting data from medical records.

And that is the large language model with a rule-based natural language processing algorithm.

The large language models may be a little bit more AI-reliant, although I'm sure you can do natural language processing algorithms using AI programming at this point.

When my team and I have done them in the past, they've been a little bit more individual programmer-heavy algorithms.

It's somewhat of an interesting concept, and I think it's different when you're looking at these things for extracting data for research purposes versus extracting data for maybe reporting to a registry or something to that effect.

It's not entirely clear to me exactly which route these authors are going in terms of their intent.

Honestly, what's good for one may be good for the other, the exception being that this does in some ways essentially cut corners.

From a research standpoint, my opinion, take it for what it's worth, is if you're abstracting data, nothing beats a good old-fashioned chart biopsy.

It's a lot of work.

I know it is.

But that's going to be the highest quality.

If you're outsourcing it to something else that's just going to pull stuff for you based on a large language model and populate something... it's likely that there are going to be errors.

The question is, how bad and how many? Their rationale is really just asking the question, if you compare LLM to NLP, which one is better? They took 1,000 primary total hip arthroplasty cases performed by 47 surgeons over a 23-year time window.

The notes came from three hospitals located, they say, at three geographically distinct sites in the United States, but it's still all within the same health system.

This is the Mayo Clinic.

Everybody here has the Mayo Clinic imprimatur.

It's not a community hospital where you'll have a couple of different private practices or something like that.

Everybody here is Mayo Clinic.