AI in Mental Health: Why Clinical Judgment Still Matters with Dr. Caitlin Stamatis

Yeah.

Yeah, you know, that's really interesting, especially mentioning Therabot.

I was talking to a group of graduate students yesterday, psychology graduate students, and one of the questions that came up was, you know, "Is there, is there a role for us in the future?" You know, because of things like Therabot demonstrating that, that these tools and applications can be helpful.

So how would you respond to that, right? When we're looking at the future of AI and looking at the things that these technologies could do, what remains still or what should remain still with the psychologist?

Yeah.

I think, you know, the thing-- I-if anything, I have become more optimistic about the future of our profession in healthcare with, with-- the more I get into the weeds building an AI system.

And I think for me, the thing that really has to remain central is clinical judgment.

Um, and what I mean by, by clinical judgment is really, I guess, the thing that separates good therapy from therapy that's just okay, right? Like, knowing when do I push someone? When should I interrupt a conversation that's warm and, and productive to ask about safety because something flagged for me? Am I really supporting someone in processing right now, or is this, I don't know, just kind of reinforcing someone's rumination spiral, right? It's, it's actually very hard to pick that up from a transcript, and it's also very hard to learn from the internet.

And-

Yeah

... right, like, what's on the internet is, I think, certainly the vocabulary of therapy, right? Which is why a typical chatbot can sound like a therapist.

But while sounding like a therapist, that doesn't mean that it is making the right calls, right? And we see that time and again with models that validate everything, you know, are not responding appropriately to the subtle ways that clinical risk shows up in a mental health context and then, you know, I-I think is hesitant to do the kind but not nice thing, right? Like, that a, that a good friend would call us on.

So, you know, I think if we, if we think about AI as part of an integrated care system, I do think that clinical judgment has to show up in a lot of places, right? And, you know, it's, it's part-- parts of it are, are teaching AI to do similar things to what we do in teaching human trainees how to be therapists, right? You never expect them to read a bunch of manuals and be competent.

They need to see and really understand and experience what good looks like.

And so a-and I think translated to AI, that looks like clinician judgment in pre-training, right? Like, something that's absent on the internet, for obvious reasons, again, are these large repositories showing what does good therapy look like, what does bad therapy look like, right? And, and, and that's something that I think gets overlooked in the conversations about what these models are doing and how.

And the second thing, at the risk of rambling, that I think is really important to preserve, and this is gonna, like, at the risk also of sounding like a capitalist cog in the machine, I think we need to be preserving the question of, like, who pays for this tool? That is, like, the central question that has to be asked with any, I mean, I guess any tool or anything, but especially in healthcare.

With therapy, our incentives, they're imperfectly aligned, but they are structurally aligned with the, the client's welfare, right? Like, we have a license to lose.

There's an ethics code.

Yeah.

A termination where a client gets better is a success.

But with a consumer product, you have none of that by default, and having a clinician in the company doesn't change that unless someone is deliberately building the obligation in, because a system like that has no license to lose, right? There's just not the same accountability structure.

So, and, and related to that, with a consumer product, what's being paid for is engagement, right? Like, we want someone to stick with it at all costs.

Um, and that is often not the best thing from a therapeutic perspective.

So I think that's where attention to how a company is making money on a product like this is actually really critical.

With healthcare, increasingly, obviously, what's being paid for is outcomes.

Yeah.

Yeah, you know, that's really interesting, especially mentioning Therabot.

I was talking to a group of graduate students yesterday, psychology graduate students, and one of the questions that came up was, you know, "Is there, is there a role for us in the future?" You know, because of things like Therabot demonstrating that, that these tools and applications can be helpful.

So how would you respond to that, right? When we're looking at the future of AI and looking at the things that these technologies could do, what remains still or what should remain still with the psychologist?

Yeah.

I think, you know, the thing-- I-if anything, I have become more optimistic about the future of our profession in healthcare with, with-- the more I get into the weeds building an AI system.

And I think for me, the thing that really has to remain central is clinical judgment.

Um, and what I mean by, by clinical judgment is really, I guess, the thing that separates good therapy from therapy that's just okay, right? Like, knowing when do I push someone? When should I interrupt a conversation that's warm and, and productive to ask about safety because something flagged for me? Am I really supporting someone in processing right now, or is this, I don't know, just kind of reinforcing someone's rumination spiral, right? It's, it's actually very hard to pick that up from a transcript, and it's also very hard to learn from the internet.

And-

Yeah

... right, like, what's on the internet is, I think, certainly the vocabulary of therapy, right? Which is why a typical chatbot can sound like a therapist.

But while sounding like a therapist, that doesn't mean that it is making the right calls, right? And we see that time and again with models that validate everything, you know, are not responding appropriately to the subtle ways that clinical risk shows up in a mental health context and then, you know, I-I think is hesitant to do the kind but not nice thing, right? Like, that a, that a good friend would call us on.

So, you know, I think if we, if we think about AI as part of an integrated care system, I do think that clinical judgment has to show up in a lot of places, right? And, you know, it's, it's part-- parts of it are, are teaching AI to do similar things to what we do in teaching human trainees how to be therapists, right? You never expect them to read a bunch of manuals and be competent.

They need to see and really understand and experience what good looks like.

And so a-and I think translated to AI, that looks like clinician judgment in pre-training, right? Like, something that's absent on the internet, for obvious reasons, again, are these large repositories showing what does good therapy look like, what does bad therapy look like, right? And, and, and that's something that I think gets overlooked in the conversations about what these models are doing and how.

And the second thing, at the risk of rambling, that I think is really important to preserve, and this is gonna, like, at the risk also of sounding like a capitalist cog in the machine, I think we need to be preserving the question of, like, who pays for this tool? That is, like, the central question that has to be asked with any, I mean, I guess any tool or anything, but especially in healthcare.

With therapy, our incentives, they're imperfectly aligned, but they are structurally aligned with the, the client's welfare, right? Like, we have a license to lose.

There's an ethics code.

Yeah.

A termination where a client gets better is a success.

But with a consumer product, you have none of that by default, and having a clinician in the company doesn't change that unless someone is deliberately building the obligation in, because a system like that has no license to lose, right? There's just not the same accountability structure.

So, and, and related to that, with a consumer product, what's being paid for is engagement, right? Like, we want someone to stick with it at all costs.

Um, and that is often not the best thing from a therapeutic perspective.

So I think that's where attention to how a company is making money on a product like this is actually really critical.

With healthcare, increasingly, obviously, what's being paid for is outcomes.