Many readers of the blog know that there was a major cock-up over child mortality figures for Iraq. In fact, exaggerated child mortality figures have been used to justify the 2003 invasion of Iraq, both prospectively and retrospectively.
Here I won’t repeat the basics one more time, although anyone unfamiliar with this debacle should click on the above link which, in turn, offers further links providing more details.
Today I just inject one new point into this discussion – the dataset for the UNICEF survey that wildly overestimated Iraq’s child mortality rates is not available. (To be clear, estimates from this dataset are available but the underlying data you need to audit the survey are hidden.)
The hidden survey is called the Iraq Child and Maternal Mortality Survey (ICMMS). This graph (which you can enlarge on your screen) reveals the ICMMS as way out of line with no fewer than four subsequent surveys, all debunking the stratospheric ICMMS child mortality estimates. The datasets for three of the four contradicting surveys are publicly available and open to scrutiny (I will return to the fourth of the contradicting surveys in a future blog post.)
But the ICMMS dataset is nowhere to be found – and I’ve looked for it.
For starters, I emailed UNICEF but couldn’t find anyone there who had it or was willing to share it.
I also requested the dataset multiple times from Mohamed Ali, the consulting statistician on the survey who now is at the World Health Organization (WHO).
At one point Mohamed directed me to the acting head of the WHO office in Iraq who blew me off before I had a chance to request the data from him. But, then, you have to wonder what the current head of the WHO office in Iraq has to do with a 1990’s UNICEF survey, anyway.
I persisted with Mohamed who then told me that if he still has the data it would be somewhere on some floppy disk. This nostalgic reminder of an old technology is kind of cute but doesn’t let him off the hook for the dataset which I never received on a floppy disk or otherwise.
There is a rather interesting further wrinkle on this saga of futility. The ICMMS dataset was heavily criticized in research commissioned for the UN’s oil for food report:
It is clear, however, that widely quoted claims made in 1995 of 500,00 deaths of children under 5 as a result of sanctions were far too high;
John Blacker, Mohamed Ali and Gareth Jones then responded to this criticism with a 2007 academic article defending the ICMMS dataset:
A response to criticism of our estimates of under-5 mortality in Iraq, 1980-98.
According to estimates published in this journal, the number of deaths of children under 5 in Iraq in the period 1991-98 resulting from the Gulf War of 1991 and the subsequent imposition of sanctions by the United Nations was between 400,000 and 500,000. These estimates have since been held to be implausibly high by a working group set up by an Independent Inquiry Committee appointed by the United Nations Secretary-General. We believe the working group’s own estimates are seriously flawed and cannot be regarded as a credible challenge to our own. To obtain their estimates, they reject as unreliable the evidence of the 1999 Iraq Child and Maternal Mortality Survey–despite clear evidence of its internal coherence and supporting evidence from another, independent survey. They prefer to rely on the 1987 and 1997 censuses and on data obtained in a format that had elsewhere been rejected as unreliable 30 years earlier.
For the record, the Blacker, Ali and Jones article is weak and unconvincing and I may make it the subject of a future blog post. But today I just concentrate on the (non)availability of the ICMMS dataset so I won’t wander off into a critique of their article.
Thinking purely in terms of data availability, the 2007 article raises some interesting questions. Was Mohamed Ali still working off of floppy disks in 2007 when he published this article? Surely he must have copied the dataset onto a hard disk to do the analysis. And what about his co-authors? They must have the dataset too, no?
Unfortunately, John Blacker has passed away but Gareth Jones is still around so I emailed him asking for the ICMMS dataset which he had defended so gamely.
He replied that he never had the dataset.
Let that point sink in for a moment. Jones co-authored an article in an academic journal, the only point of which was to defend the quality of a dataset. Yet, he never saw the dataset that he defended? Sorry but this doesn’t work for me. As far as I’m concerned when you write an article that is solely about the quality of a dataset then you need to at least take a little peek at the dataset itself.
I see two possibilities here and can’t decide which is worse. Either these guys are pretending that they don’t have a dataset that they actually do have because they don’t want to make it public or they have been defending the integrity of a dataset they don’t even have. Either way, they should stop the charade and declare that the ICMMS was just a big fat mistake.
I have known for a long time that the ICMMS was crap but the myth it generated lives on. It is time for the principle defenders of this sorry survey to officially flush it down the toilet.