Last June, the editors of Money Sense magazine issued a rating of 100 top Canadian charities. The magazine used a four-part rating system and assigned an overall letter grade to each charity. But what does it tell us about the effectiveness of the charities being rated? Probably very little.

Ranking systems have often been criticized for their arbitrariness. In the February 21 issue of The New Yorker, Malcolm Gladwell examines the limitations of college rankings. He picks apart the faulty assumptions of the best-known US college rating system and its reliance on flimsy data. He also point out the paradox of rating systems: they are self-fulfilling prophesies – rigged to produce certain results.

Fed by online tax return data posted by Canada Revenue Agency, there has been an explosion of charity rating systems in the last five year. There are now philanthropic consultants who specialize in separating naughty from nice charities. What most charity rating systems ignore are the personal experience of the donor, as well as location, culture and leadership of the charity. Are we in danger of reducing complex community organizations to caricatures, ignoring public benefit in our rush to look at fundraising costs and operating reserves?

When planning a gift by will to charity conducting due diligence in advance is recommended. My bias is personal investigation. Call the charity and ask questions. The Charity Checklist in the Canadian Donor’s Guide provides some tips for understanding your potential beneficiaries better, not merely judging them.