Data-driven charity

Some ex-hedge fund analysts recently started a non-profit devoted to evaluating the effectiveness of hundreds of charities, and apparently have been making waves (NYT). A few interesting reports have been posted on their website, givewell.net -- they make recommendations for which charities where donors' money is used most efficiently for saving lives or helping the disadvantaged.

(Does anyone else have interesting data on charity effectiveness? I've heard that evaluations are the big thing in philanthropy world now, and certainly the Gates Foundation talks a lot about it.)

Obviously this sort of evaluation is tricky, but it has to be the right approach. The NYT article makes them sound like they're a bit arrogant, which is too bad; on the other hand, any one who makes claims to have better empirical information than the established wisdom will always end up in that dynamic. (OK, so I love young smart people who come up with better results than a conservative, close-minded establishment. Or at least I'm a sucker for that story.)

This particular methodological criticism (from the article) struck me as odd:

“I think in general it’s a good thing,” said Thomas Tighe, president and chief executive of Direct Relief International, an agency that GiveWell evaluated but did not recommend. Like others in the field, however, Mr. Tighe has reservations about GiveWell’s method, saying it tends to be less a true measure of a charity’s effectiveness than simply a gauge of the charity’s ability to provide data on that effectiveness.

I think it's fine to penalize an organization for failing to provide data on its effectiveness. Isn't the burden of proof on them, to show that they're actually doing something useful? I guess it comes down to whether you believe empirical evaluation is necessary for organizational effectiveness. I believe this wholeheartedly.

The GiveWell people have an interesting argument that altruistic actions have a particularly poor feedback loop, which kills learning/optimization; therefore, you need to undertake explicit evaluative efforts. From their blog:

Now imagine an activity that consists of investing without looking at your results. In other words, you buy a stock, but you never check whether the stock makes money or loses money. You never read the news about whether the company does well or does poorly. How much would you value someone with this sort of experience - buying and selling stocks without ever checking up on how they do? Because that’s what “experience in philanthropy” (or workforce development, or education) comes down to, if unaccompanied by outcomes evaluation.

The peculiar thing about philanthropy is that because you’re trying to help someone else - not yourself - you need the big expensive study, or else you literally have no way of knowing whether what you did worked. And so, no way of learning from experience.

I really like this point -- which is easier to notice, that you're bankrupt or that someone else is? That your own business is doing well/badly, or that your beneficiaries are doing well/badly? Self-regarding actions get automatic evaluation but altruistic actions don't, presumably because, even if we care enough to give to others, we do not care enough to expend energy evaluating their outcomes down the line. But we really care about our own personal outcomes. Yet another example of human preferences being more selfish than altruistic; well, what's new?