Sanity and Statistics

The Corruption Perception Index released by Transparency International has the whole country in a tizzy – the government’s detractors are delighted to have “evidence” of the government’s corruption and governance failures while the government is on the legal and political offensive, launching a slanderous attack on TI in Pakistan, passing resolutions against it in the provincial legislatures and having minister’s issue ludicrous proclamations promising the eradication of corruption from government departments in a week. Such political dramatics and embarrassment could have been avoided if the government had decided to employ a statistician to look over TI’s methodology rather than a team of lawyers to launch a legal attack on the organization criticizing the government.
 
Transparency’s methodology lacks the mathematical rigour required to provide a robust estimate of corruption in Pakistan. The CPI should have been opposed on mathematical and intellectual grounds not moral and legal ones.
 
Consider this – Pakistan ranks 143 in the list. Its CPI score is 2.3. However, if you care to dig a little deeper you will notice that there is this neat little concept of a confidence interval – this means that the number you see presented in front of you is really not the “real” number but there is a range within which this number could actually lie. Sp if most policy pundits had cared to click the extra box that reveals the confidence intervals, TI would have revealed all of this great wisdom to them – TI doesn’t really hide it – it’s on the website. But why bother looking at confidence intervals or standard deviations when we can scream tiny numbers and do clean little comparisons with our favorite comparison country? Because one number to harp on and India to hate on make more pleasant reading and television than statistical concepts.

So if you did even the tinist of digging on the transparency website two facts would become apparent – the CPI cannot be used to compare scores between years. So everyone who’s screaming that look at Pakistan’s X point drop from last year would have surely flunked stats at college and possibly even basic reading and comprehension because this is cleatly stated in the methodology documents of the CPI.

Also, the score that everyone is shouting about – isnt actually an accurate estimate of the perceptions of Corruption. In statistics – single numbers seldom are. The ever so cautious statisticians often choose a range to report scores rather than pick one and call this range a confidence interval. Therefore a 90% confidence interval would suggest that there is a 90% chance that the true score is somewhere between that range. The neat little numbers you see as “scores” are conveniently reported mid-points of these intervals.

So there is  a 90% chance that Pakistan’s score is not precisely 2.3 – but lies somewhere between 1.8 and 3.3. (this is the confidence interval around the reported score). Incidentally, India’s score is reported as 3.3 but there is a 90% chance this score actually falls between 2.6 and 3.9. So Pakistan might be perceived to be either more corrupt that India or less. That does not seem to say much. And based on an analysis of the methodology of the CPI – the government could have easily concluded that the CPI is not really the gold standard of corruption indicators. Or had they been fortunate enough to take Stats 101 with my professor at Princeton, they could have learnt that statistics are like bikinis – what they hide is often more interesting than what they show.
 
But one has to be willing to dig deeper and make an intellectual commitment to assessing  such performance indicators on their intellectual and methodological merit. However, the reactions of the Pakistani policy makers and policy pundits reveal a deeply disturbing trend of pegging policy debates and decisions on little else than rhetoric and bikini statistics – not the best guides in the often treacherous and difficult policy terrain.