Buyer beware: the limitations of psychedelics research - Eiko Fried

To set the scene, because I'm one of your followers, academic followers, I know that a lot of your work is dedicated to something that we formally call meta-science, researching how we can make science better in a sense.

And meta-science touches every scientific subject and psychedelic research as well.

Would

you please,

in a few words, tell our listeners what it means to do this kind of research and why is it important?

Yeah, thanks for having me.

I'm excited to be here.

Shout out to Michiel van Elk also already, who was the first author on that paper, of course.

We talked about him briefly before this started.

So my friend Anna van Tver, who's also at Leider, always says, meta-science is research on research.

That's sort of, I think, a very short definition.

And I think meta-science is sort of like taking a lot of papers together and looking at them with tools that we have at our disposal as scientists, trying to see if there are potential problems in the literature.

For example, a very famous meta-science paper in 2015 in my field in psychology is where authors conducted 100 studies that they had been conducted before, and they wanted to see how many of those replicate, and they found out only a third of them actually replicated with similar findings, and that's sort of a meta-scientific finding casting doubt on the prior literature.

Another topic is reproducibility.

That means if authors with a given dataset and a given statistical software find a certain result, would I find the same result given the same software and the same dataset? And there's a couple other topics.

For us today, we'll talk a little bit about publication bias probably, which is another meta-scientific insight.

And that means that if you look at the published findings, they look too good to be true.

All of them are good.

And what you often see is that the smaller the samples get, the better the results, for example, for psychedelics or some other drugs.

And that is largely due to publication bias, meaning you publish significant positive studies, but you don't publish the non-significant ones.

Wonderful.

Thank you so much.

So it's important to look at how research is conducted before we can pretty much draw a definitive conclusion with respect to how robust the results are.

And

so with the context specific to psychedelic research, this is a bit of an unusual type of of subject and of component to clinical research studies because of the effects that it has right that psychedelics have and that involve a lot of very very personal unique experiences that are very hard to standardize and we are looking for standardization as much as possible when we do science, but with psychedelics, this challenge kind of takes it up a notch.