How to talk about failure

One of the takeaways from this example that I've heard is that funders like GiveWells and CoefficientGivings, when they're deciding what to push resources to and coming up with their internal calculus of how effective an intervention is likely to be, that they should be giving more weight than they currently do to exactly these things that you're talking about, the implementation science of it all versus the sort of, you know, what you can pull out of a randomized control trial about, you know, whether a specific intervention itself is likely to be effective in a sort of theoretical world that you have to make sure that you're factoring in these different complexities around bias and monitoring and evaluation and all of that into that initial calculation.

Do you agree sort of generally with that?

Yeah, I think that's fair.

It's actually far beyond monitoring.

It's just when you look at an intervention, how complex this is going to be to implement.

And so when we continue to speak with GiveWell about this and other programs, and we talk to others who care about effectiveness, there is definitely an increasing conversation around What are all the ways this could go wrong? I think that's always been the case and it's even more the case now.

You know, again, as I was sharing about dispensers, there were so many things that are required for the water to ultimately be safe to drink.

Behavior change is a proxy we're looking for across interventions as well.

If there's a lot of behavior change required, it's likely to be much harder to sustain results.

In so many of the countries in which we work, there is a lot of volunteer labor, especially post-USA.

So we also are thinking increasingly thoughtfully, not to say we're not going to use whatever form the government is using of people who are are delivering health interventions, community health workers, and often there are some who are paid and some who are not paid.

That is the government system, especially today.

So if you're working in government systems, you're working with what there is.

And motivation is likely to decline over time.

And that was also one of our lessons with dispensers.

So how do you factor that in from the start that you're expecting those types of drop-offs? And even if you added incentives and, you know, if you're trying to manage a program cost effectively, be realistic about the challenges that are likely to arise.

And so we definitely turn that lens internally because we want to ensure our programs have longevity and you know, that they not only are cost effective in the first years, but are cost effective for the long run.

But I'm certainly seeing this in our funders as well, looking at these kinds of questions.

And I think that's a positive shift.

What's the wrong lesson to learn from this?

I think a wrong lesson would be don't try to do the hard things.

Working in this particular area is truly neglected, as we've also been looking at, well, who else will provide water in these communities, is not an obvious answer because no one else is doing it.

And that is really sad.

So these heavily neglected areas don't go away.

And, you know, the wrong answer for our whole movement would be let's not try to do the hard things.