The conflict in Iraq since the US-led invasion of March 2003 has claimed many thousands of lives. But how many, exactly?

Some widely-quoted studies have cited figures of 601,000, one million, even 1.2 million. Others have put the deaths much lower, at around 100,000 to 150,000. All these are large figures, but the gap between them is huge: and while certainty is elusive in such a violent struggle, these estimates are not even within shouting distance of each other.

Professor Michael Spagat of the Department of Economics at Royal Holloway College, London, who has appeared on these pages before, believes the higher figures are gross exaggerations. In two recent papers he has launched a broadside at the way they were reached, arguing that overstating the deaths can do immense harm by entrenching hatreds and misunderstandings and inciting further violence.

His first target is a study (1) that appeared in The Lancet in October 2006 from a team led by Gilbert Burnham of the Bloomberg School of Public Health at John Hopkins University in Baltimore, Maryland. This paper (the second published by much the same team, and referred to as the Lancet II study) calculated that Iraq had suffered 601,000 violent deaths, four times as many as estimated by the Iraq Family Health Survey Study Group.

Professor Spagat’s assertion is that such a high figure can only have been produced through fabrication and falsification.The Burnham team carried out a household cluster study in which small neighbourhoods across Iraq were chosen by a random method, and then 40 contiguous households in each neighbourhood were approached for interview. Respondents were asked about how their houshold composition had changed, including deaths since 2002 and the causes of those deaths. The precise questionnaire has never been published. 

Professor Spagat raises many ethical issues about the way it which the survey was conducted. The survey teams, wearing white coats, used children in each neighbourhood to spread the word that a survey was going on and to dispel suspicion, possibly placing them in danger from local militias.

Whatever the exact questions asked, the forms differed from the original protocol in at least one important way. The interviewers recorded the names of respondents, when the protocol said they would not – exposing them, too, to possible reprisals. The Lancet paper expressly said that such identifying data had not been taken. Last year, as a result of this violation, the Bloomberg School of Public Health suspended Dr Burnham’s privileges to serve as a principle investigator on studies involving human subjects.

This review did not cover the sampling methods or the statistical approach used, but Professor Spagat does.The working schedule – 40 interviews a day – was implausibly onerous, expecially in a country racked by violence where travel itself was dangerous.

In some parts of the country – the South and Baghdad, for example - the team’s estimates of violent deaths matched fairly closely the estimates made by the Iraq Living Conditions Survey, whose coverage period overlapped with Lancet II. But in other places the estimates differed hugely, as the table below shows (Professor Spagat's underlining):

 
These differences, of scale, location and timing, are, says Professor Spagat, “consistent with fabrication”. He lists nine arguments for believing that some at least of the data published in The Lancet were fabricated. A few of these anomalies could arise by chance, “but it is extremely unlikely that all of them could have occured randomly and simultaneously”.
 
Professor Spagat’s 40-page analysis, in Defence and Peace Economics, is preceded by an editor’s note, which reads: “The authors of the Lancet II study were given the opportunity to reply to this article. No reply has been forthcoming.”
 
In a second paper, (3) Professor Spagat and colleague Josh Dougherty of Iraq Body Count take aim at an even higher estimate of Iraq dead, made by the British opinion polling company ORB. In September 2007 ORB said that a survey it had carried out in Iraq indicated that 1.2 million Iraqis had been killed, a figure it subsequently lowered to 1 million. 
 
ORB interviewed 1,720 adults in Iraq, of whom 1,499 agreed to answer questions about how many members of their household had died. The survey took place between August 12 and 17, 2007. Further sampling in rural areas caused ORB to lower its estimate by almost 200,000 deaths, and in January 2008 it said that it believed just over 1 million had in fact died (1,033,000), with a confidence interval of 946,000 to 1,120,000. The total number of respondents in these two surveys was 2,163.
 
The principal anomaly identified in the ORB data is that many respondents reported more deaths in members of their own household than they did among their extended families, in spite of the fact that extended families are far larger than households. To extrapolate to a national figure, only the household data can be used (to avoid double-counting) yet in four governates in central Iraq which account for more than 80 per cent of the 1 million deaths, the household deaths exceed extended family deaths that ORB had recorded in a poll taken just six months earlier.
 
To take another example, ORB estimated that there had been 130,000 deaths from car-bombings in Baghdad alone, in a period in which the international press, largely based in Baghdad, reported 5,500 such deaths in the city. This implies that 95 per cent of lethal car bombings went unreported – “exceptionally implausible” say Spagat and Dougherty, expecially as the deaths recorded by the Iraqi Ministry of Health and the Baghdad morgue are close to those estimated from press reports.
 
ORB’s figures in total suggest that 600,000 people died in Baghdad alone, 80 per cent of them men. If so, say Spagat and Dougherty, a quarter of the Baghdad male population would have been killed, and the sex ratio would have been 1.3 to one by August 2007. Yet ORB itself found the sex ratio close to one.
 
In a discussion published in the same issue of Survey Research Methods, Johnny Heald of ORB rejects the claim that the survey was politically motivated, and accuses Spagat and Dougherty of “pursuing an agenda”. He denies that ORB has been uncooperative with them, but does not address the detailed points and acknowledges that the the ORB figure is “only an estimate”. The true number of casualties, he says, “will likely never been known”.
 
Spagat and Doughery retort that the issue cannot be shrugged off so easily and is not simply, as Heald suggests, a minor academic spat. “ORB’s intervention is likely to lead to entrenchement of hatreds and possibly even incitement to future violence”, they say.
 
“Is Iraq’s future doomed to feature different groups selecting favoured estimates or war deaths from widely divergent claims and condeming each other as exaggerarators or deniers?” The work of the Research and Documentation Centre of Sarajevo in the aftermath of the Balkan conflcits shows that this is not inevitable, they conclude.
 
References
 
(1) Burnham G, Lafta R, Doocy S and Roberts L, Mortality after the 2003 invasion of Iraq: a cross-sectional cluster sample survey, The Lancet, 368 (9495) October 1421 -1428
 
(2) Spagat M, Ethical and data-integrity problems in the second Lancet survey of mortality in Iraq, Defence and Peace Economics, 21:1, 1-41
 
(3) Spagat M and Dougherty J, Conflict Deaths in Iraq: a methodological critique of the ORB survey estimate, Survey Research Methods, Vol4, No 1, pp3-15